Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
view the rest of the comments
No ECC is (and should be) a complete deal breaker for any self hosted data store.
There are just so many budget friendly options that do support it these days, I can't imagine making that sacrifice.
If you're talking about requiring ECC for ZFS, your point is arguable.
With that said, could you point me to the newer AM4/AM5, LGA1200/LGA1700 boards which support ECC?
ZFS doesn't need ECC more than any other filesystem. Technically it needs it less. But what it does do is expose just how common memory errors are.
ECC exists for a reason at the enterprise level. A very important reason. You need to be able to trust that the data that the CPU put in memory is the same as the data written to disk.
Would you have any suggestions for newer consumer boards which support ECC?
Don't buy a consumer board for a server unless you aren't using it to store important data. Like, say, a GPU cloud gaming or Plex server would be fine on consumer hardware ifyouf dont need out of band management.
Also ECC RDIMM is much easier to come by than ecc udimm but only works with epyc or Xeon chips. I say if you're going for storage then definitely buy enterprise gear and if you're going for raw CPU/GPU compute you should be fine with consumer hardware.
If you just want am4/5 ryzen chips, asrock rack makes some good boards with IPMI.
here's their x570 board you can browse their site and they also have am5 boards and they dont have three gimped x16@x1 sockets.
The problem with enterprise hardware is:
Consumer hardware solves both of these problems. Yes, we don't have iLO, but if someone is really motivated, they can use PiKVM. I am yet to figure out if I can run PiKVM without the hats on a different SBC but I think it can be done.
For me personally, I'll be using said board in a NAS. With this board, I would no longer need an LSI HBA hogging my x16 port, which means if I ever decide to train ML models, I can get a GPU for myself.
The Asrock Rack series is exciting, but from what I have seen, that line of motherboards are really expensive. I'll keep a look out though, I find one of their motherboards under $150 which fits my needs, and it will become my number 1 choice.
I do not see why I absolutely need ECC memory for a NAS. I'm not going to store PBs of media/documents, it'll likely be under 30TB (that's a conservative estimate). I thought ECC memory is a nice-to-have (this is no enterprise workload).
Cheers
Efficiency is relative. I'm not suggesting you get a high clock much core server chip (though you could technically limit the clock speed and TDP and it would use as much power as a typical desktop) as there's plenty of low power options that are 'old' (read:~4 years old is not that old). Maybe look into some Xeon-D embedded boards solely for your storage system. Many of those boards were made specifically for storage appliances. They can also be had pretty cheap on ebay or wherever.
I'd say $120 is too expensive for this motherboard. Seems like it should be ~$60 with those specs and not to mention it being last gen. So even though you're buying new you have an upgrade ceiling so why not buy a year or two older gear with more features and expansion.
FYI iLO is HPs out of band (IPMI) implementation. PiKVM is definitely cool but its just adding more cost and another point of failure in your setup.
If you want to train models and other gpu compute stuff like that, I would definitely shoot for a more current gen box just for that. In my (good) opinion, you should not run heavy compute loads on a server that is also serving/backing up your data,
ESPECIALLY if you're not going to use ECC memory. No reason to put your important data at risk of corruption like that. I highly recommend holding out for something simple with DDR5 and a discrete GPU of your choosing for any actual compute workloads like that.
Prices for newer hardware like this may fall before you're even ready to build this system, so keep that in mind. You'll also have a much easier time selling it over older gen hardware in the future if you change your mind about whatever.
from original post. why would you want to do this in a server? if you got a different board with sockets that werent x1 you could just get a 2.5gbe card... or you know, 10gbit.
Thank you for the tip. I will look into Xeon-D integrated motherboards. I will not be running very heavy loads (other than a Suricata instance for an IDS/traffic analyser - I would love suggestions which might be lighter on compute - which might be heavy). The idea for training ML models was just a remote possibility.
My apologies, I kept saying iLO/iDRAC when I meant IPMI.
Why do you suggest having separate devices for storage/compute?
My idea was to run FreeBSD on a ZFS mirror of NVME drives as the base, and run VMs/Jails on a pool of SATA SSDs. These would exist alongside HDDs but would otherwise not affect their functioning. In this scenario, how does having 2 machines make my infrastructure more reliable, other than FreeBSD not running as intended?
Have you had instances of memory corruption because you didn't use ECC? I was under the impression from r/selfhosted that this problem was blown out of proportion.
The reason I mentioned the E-key slot is because that way, I don't have to use a PCIe slot for the adapter, which I might use for something else. I have no need for 10Gbe.
Thanks!
because storage never needs to be upgraded beyond drive capacities, unless you need for a bunch of NVMe storage which requires more pcie lanes. The only reason you should have to change a board or cpu in a storage server is if it dies. If you need a new piece of hardware for its new features, it would be much easier to upgrade a different system rather than taking your storage offline to do it. whatever gpu you put in there now is going to be dated in a couple years when you may want to upgrade.
no because i dont run big storage pools on desktop hardware. you may be able to run non ecc memory for a long time and not get any data corruptions, but that doesn't mean you wont. also it's not always obvious when there's corruption especially in older data.
what are you going to do on those two x1 sockets? theyre really not good for anything other than usb and 1gbit (maybe 2.5gbit?) networking, or maybe a sound card. those m.2 adapters are more suited for minipcs that dont have any other pcie expansion options. not saying you can't or shouldn't do it, but why? especially when 10gbit options are much cheaper if you buy used.
Thanks for your comment. I'll keep this in mind :)