The original post: /r/datahoarder by /u/waaaggh on 2024-12-10 05:51:25.
Hey guys I'm looking to have a permanent long term storage solution for my business. My research into this yielded the following and I wanted to see if you guys have any recommendations.
I edit on my Macbook Pro off a 4TB Samsung T9 SSD. My shoots net about 500gb per job and I do plan to keeping my RAWS and footage forever. In the wedding industry it's probably best to never delete anything. Most likely I'll be compressing my RAW files to lossy DNG sometime in the future unless I just buy more HDDs.
I keep a old gaming PC running 24/7 and would ideally use a DAS so I can use Backblaze's backup service as an offsite cloud solution. The desktop just runs Windows 10 and I already have four 4TB drives inside as a Storage Space volume hosting my own personal files. They are setup with parity which as I understand it is basically just RAID 5.
During the black Friday sale, I bought four Ironwolf Pro 16TB drives and a QNAP TR-004. I have since set it up in hardware RAID 5 and just plugged it into my desktop.
My plan for data workflow:
- All my cameras run dual SD or CF-Express cards
- When I get home, I copy my memory cards to the T9 SSD and over my network (just a pleb 1Gb/s network) to the QNAP DAS network drive on my PC overnight
- Backblaze backs up everything to the cloud off the desktop
- I edit off the T9 SSD and keep the files on the T9 until final deliverables are sent. I rarely work on more than 5 jobs at once so the 4TB has been working fine for me
- Once or twice a year I plan on dumping the files off the DAS onto an external USB HDD and just buy more of them as I fill them up as an onsite backup. At the current 43TB off my DAS I'd eventually buy 4-5 USB HDDs.
This setup give or take some complexity is what I've seen other photographers using for long term storage. Some others use a NAS but then you'd loose out on the cheap Backblaze backup and I don't edit off these drives so I shouldn't see much benefit of a NAS right?
Is this a solid solution for my use case? Any recommendations and tips?
When I was learning about the different RAID options some people have said RAID 5 is bad. Should I use something else?
Is there a better enclosure that you guys would recommend? The QNAP seems to be one of the few hardware RAID enclosures I've found. I don't foresee having to move the enclosure to a different computer too often but that would be the main advantage of the built in RAID controller. In case my desktop dies or has issues, this seems to be the easiest solution to still have access to my DAS instead of fixing/buying a new desktop.
Would a software controlled dumb enclosure be better than the QNAP TR-004 as far as reliability? If the enclosure dies do I have to rebuild the RAID and loose all my files in either a dumb enclosure or the hardware raid one?
So many questions as I delve into this rabbit hole of long term data storage. Really appreciate any insight!