It's A Digital Disease!

The original post: /r/datahoarder by /u/Gnomefort on 2024-12-10 01:46:43.

Hi all! So I am running into an issue that is a bit beyond my expertise. The important bits:

• 32 TB RAID5 array (3x 16TB drives) hosting my personal Plex server. Drive is at 50% capacity.

• Started getting errors saying "File or directory is corrupt or unreadable" while altering files on the drive. Frequently accompanied by a BSOD (long as I don't touch the drive I don't seem to crash)

• I used SoftRAID and it started to rebuild automatically, says 22 days (wtf!)

• I had a few thousand video files on the drive, separated into a two folders. In one of the folders, I can only see about 10% of the files that that should be there. Haven't noticed issues in the other folder yet. However, if I load up my Plex client I can still stream everything just fine even though I can't see it on the drive. A small handful of files (all relatively recent additions) do show as "unavailable" though.

--

I assume others have run into similar issues? This is a bit beyond me. Should I just let the rebuild finish up and hope for the best? Is there something else I should be doing? Obviously the concern is data loss, but the fact that I can still stream most of it leads me to believe not all hope is lost, or am I being overly optimistic?

Definitely seems like I want to switch to a mirrored backup approach after this.

The original post: /r/datahoarder by /u/West-Solid9669 on 2024-12-10 01:17:13.

As title states archiving all consoles and there games pre gamecube era and I need help finding complete game sets for everything but NES, SNES, GameBoy, GameBoy Color, Virtual Boy, Genesis, Game Gear, Atari 2600 and Atari 5200.

The original post: /r/datahoarder by /u/GTRacer1972 on 2024-12-10 00:50:23.

When I am copying whole folders it will tell me if any of the files are different sizes and I can make decisions based on that, but the one feature I want is missing: if I have two copies of the same file, but different sizes, I want to set it to automatically keep both of those, while overwriting everything that's the same file name with the same size. The way it is now I have to manually hit keep both for every collision and it's annoying.

The original post: /r/datahoarder by /u/rahulkadukar on 2024-12-10 00:41:23.

I recently started backing up my data to LTO and saw this on the Ultrium 5 tape flyer "Store Media vertically".

https://imgur.com/a/88z8QNx

I have not seen the same on a Fujifilm Tape, so my question is for long term storage should I store the tapes vertically OR horizontally.

The original post: /r/datahoarder by /u/perecastor on 2024-12-10 00:12:28.

I currently buy a pair number of drives, one for the data and one for the mirror backup. I use rsync to do the sync once a month. Do you have a better alternative? The drives are not mounted all the time so I think it has to be a manual sync

I use something that looks like a toaster to mount SATA hard drives

The original post: /r/datahoarder by /u/kaitlyn2004 on 2024-12-10 00:07:56.

I have an internal hard drive in my Windows desktop.

I now have a MacBook Pro, along with a 14TB USB3 drive. I expect to format that as APFS (unless I hear differently?!)

Looking to move across the Windows internal drive contents in its entirety, which is 99% RAW photo files.

Thinking I'll expose it as a temporary network shared drive, then access it from my laptop with the external drive plugged in? Contents is about 4TB. Given the sheer volume of files along with average filesize of like ~30MB, should I literally just copy+paste or it feels like I should probably use a different method or tool to get the data from the older drive while ensuring integrity of it (checksums?)

Not entirely sure what I'll be doing with the old drive inside the Windows desktop, but I'm not planning to keep it in sync or anything afterwards.

The original post: /r/datahoarder by /u/BlueDragon9976 on 2024-12-10 00:04:23.

I have about 500GB of family photos and videos going back 20+ years and I need a better way to store them. We have 2 external HDDs at the moment, both of which are near failing. I thought maybe just a 1TB External SSD would suffice but they're not the cheapest things and I'd really like to have some confidence in longevity.

I was suggested 100gb BluRay Disks stored in a dark, dry, environment which could work, only I'd prefer digital storage.

What options do I have?

The original post: /r/datahoarder by /u/TemplarIRL on 2024-12-09 23:49:54.

Hello, new here. SORRY if this isn't the BEST sub for this... I am looking to expand my local storage at home. I just want to have a dedicated drive for my wife and I to keep files on, and it won't see that much use, unless we are storing something or pulling something to our respective computers (I will run it on my PC as a "server"). I think the biggest use would be streaming a few bird watching videos from YouTube for my cat when we are away at work.

I would like an opinion on the following budget setup: (YES it is highly unique.)

HUB: SABRENT 4-Bay USB Type-C External SSD Docking Station (DS-4BNE) - $100

STARTER Storage Device: Xiaomi Portable SSD 30TB External Hard Drive USB 3.1/Type-C Solid State Hard Disk High-Speed Storage Device for Laptops/Mac - $15

I was thinking of trying to set this up as a RAID device, though I have no prior knowledge with it, just what it "does". I believe I can do this with software after initializing the "portable SSDs".

The original post: /r/datahoarder by /u/p3apod1987 on 2024-12-09 23:32:42.

The original post: /r/datahoarder by /u/rahatable on 2024-12-09 22:03:24.

I'm a designer and I need a good cheap storage option like google drive to send files to my clients. Playbook gives 4tb to artists but files are extremely unorganised layout for clients. It's not good user experience.

The original post: /r/datahoarder by /u/Zorklis on 2024-12-09 22:23:11.

So I have Lithuanian TV program archival project and need more data from https://tvprograma.15min.lt/

Does anyone have any methods that I could use to scrape/collect all the date info for all the main channels, like say LNK - https://tvprograma.15min.lt/tv-programa/televizija/lnk/3/2018_04_26

into like a .txt?

Thank you :)

Happy holidays

The original post: /r/datahoarder by /u/manuelfantonix on 2024-12-09 22:15:56.

Hi, I have a 4 bay Yottamaster enclosure, the ps400ru3 model. I don't know why but it doesn't wanna go in sleep mode. Windows 11 or Linux ubuntu server doesn't make difference. After some time, it seems to go in sleep mode but after a couple of seconds it returns in the normal mode. I've installed just one 3.5 drive, no raid, just raw data.

Someone can help me? Thanks.

The original post: /r/datahoarder by /u/testertestermp on 2024-12-09 21:58:33.

Hello Guys,

Please rate and give suggestions on my folder structures.

Here is my old folder structure. I currently use this as a dump from various places and is in process to retire some old folders like Spy Videos (Event) and streamline things like License, Bill and Doctor Receipt in a single Documental Needs folder. I use AA Camera Group as my main dump for my Camera folder, and similar for Instagram, PC Screenshots, Snapchat, Unorganized (Misc. photos) of my phone/devices.

Picture 1

Here is my new folder structure that I am to follow:

Picture 2

I use things like Impact Collection, Randoms for things happening year around. Random Family is for random selfies and pics taken throughout the year at home or work for no special reason. Random Friends and Relatives is pretty self explanatory, and then comes events, which are categorized by year and month. Unorganized here is a kind of subset from Picture 1, where Unorganized in Picture 1 is global while here it is yearly organized, allowing me to go back and reflect and if possible create an event folder.

I would like to know if someone here have any thoughts/suggestions in improving this.

Additional facts (I'll add more from comments if needed):

1. This is a global drive that is shared between my spouse and family, if this information is of any use.

2. I am intentionally repeating the year name as it allows me to know where exactly I am just by looking at the picture's folder. Some use case require this

3. I am not putting dates in folder names as the events that can comprise of multiple days.

The original post: /r/datahoarder by /u/SpenZebra on 2024-12-09 21:36:41.

For starters, I want to store and backup my GoPro footage, photos, and some 3d projects. It's halfway to 1TB. I've read a bunch and learned that disks, tape, and sdds, if stored and operating properly can last quite some time. Any recommendations for M-DISCS? It has to be under $200.

Thank you very much!

The original post: /r/datahoarder by /u/identifiable_content on 2024-12-09 21:12:13.

99.9999% of people won't care about this. But I was annoyed I couldn't find exactly what I was looking for when I needed this, and it was deceptively more difficult to figure out than I expected.

But on a PC if you have a MSG file, you can just open it and download as a PDF. However, if you are on a Mac, this isn't an option as MSG is propritary to Microsoft.

So I made a little github repo that allows you to bulk process as many MSG files as you need.

Ultra simple, nothing special, but a niche tool that I personally struggled to find a free solution to. This is assuming you know minimally how to run python code. If you don't, ChatGPT is a great resource, just give it the url and ask what you need to do. Although, I can't imagine too many people in a non-tech space would need this haha

https://github.com/hyrumsdolan/MSG_to_PDF

--

This was just the most relevant sub that seemed to have more relaxed rules. I make no money on this, just know I would have been happy to stumble on this while I was searching and hoping this helps someone in the future.

The original post: /r/datahoarder by /u/Free-Art-9017 on 2024-12-09 20:46:10.

For years I used https://addons.mozilla.org/en-US/firefox/addon/easy-youtube-video-download/ to download HD files and in recent months Youtube has been kicking this guy's sweet add-on into the ground.

More recently I have been using https://addons.mozilla.org/en-US/firefox/addon/faststream/ and now this one no longer seems to function as of today.

I was able to come across this website https://cnvmp3.com/ but unfortunately it doesn't support hour long plus videos.

So my data hoarding is Critical Role and if you know anything about them they have VERY long videos. Anyone have an other options that work? Hopefully FastStream gets updated again (there was an update today, but didn't fix it) as Easy seems to be no longer be getting updates.

The original post: /r/datahoarder by /u/cabriofritolay on 2024-12-09 20:22:44.

TLDR: Trying to combine somewhere south of 20TB of data to a server and in that process automatically merge duplicates files.

I've been saving a lot of my video, photos etc. to portable hard drives through the years (like one still uses USB 1.0). I've recently purchased a server and essentially want to collect all of the data on those drive and put it in one place. However, I want to get rid of duplicate media. Is there any way you would recommend going about this?

The original post: /r/datahoarder by /u/Alarmed_Town_69 on 2024-12-09 19:28:35.

Hey everyone,

I recently bought a mini PC with a 2TB SSD, thinking it would be enough for my Jellyfin media server. However, I'm quickly realizing that I might need more storage space. I've been considering my options and I'm torn between getting a Direct Attached Storage (DAS) or a Network Attached Storage (NAS) solution.

From what I understand, a NAS offers more features like remote access, data redundancy, and better integration with network services. But, I'm wondering if a DAS could be a simpler and more cost-effective solution for my needs.

Here are a few points I'm considering:

Cost: DAS seems to be cheaper upfront compared to NAS. Is the price difference worth it?

Performance: Will a DAS provide the same performance for streaming media through Jellyfin?

Ease of Use: How easy is it to set up and manage a DAS compared to a NAS?

Future Proofing: Which option would be better in the long run as my media library grows?

I'd love to hear your experiences and recommendations. Is a DAS really that bad compared to a NAS for a home media server setup?

Thanks in advance!

The original post: /r/datahoarder by /u/KingRollos on 2024-12-09 19:18:45.

I'm aware that AWS glacier is the cheapest storage, although egress is expensive. The other factor that's putting me off is that AWS scans files on their servers, this means that unless I know what each is I'll need to take time to decrypt before I can use the file ☹️ What cheap options are there for cheap safe backup that won't scan my PRIVATE files

The original post: /r/datahoarder by /u/PhotosRLife on 2024-12-09 19:14:36.

There is a lot of information on the topic, but at the end of the day, I don't have a TON of data to store, and I'm wondering what's the best course of action to follow before I accumulate enough data that my setup becomes unmanageable. Here's my use-case:

I'm an amateur photographer, so my data consists of RAW image files, some video (not a lot, but it will likely increase over the years), as well as the processed jpegs and other misc. files. My setup right now is a scratchdisk SSD where I keep ~1y of images/video. This is my working disk where I work on the Raw files/video etc for fast access. I do my best to cull immediately after unloading the SD card, so this disk has about 700GB on it by the end of the year (seems like that's nothing compared to the amounts I see on the interwebs haha). All of the files are copied to two usb external 4TB HDDs. So these contain the data I accumulate over the years, and I replace them as needed when I reach capacity, which is relatively infrequently in my current usage. I then have another copy offsite that I backup to every now and then (albeit I should do this more often for a reliable 3-2-1 setup).

Now...my current drives are going to be at capacity soon, and while it's been a relatively pain-free life since I've started doing things this way, I'd like something a bit more robust. I'm going to start accumulating external drives as the years go on, and can foresee this becoming a pain. I'm witnessing that with my parents, who have a plethora of external drives accumulated over the last 20y that go from the chunky externally powered massive HDDs to the small passport-style drives. Those need different cable connections to function, become obsolete etc... I'd like to set myself up for a slightly simpler data hoarding future. What is the best way forward? NAS vs DAS, NAS seems a bit overkill? I don't "need" access to all of my images everywhere I go, although it could be nice. But is a NAS really needed for me, who for the foreseeable future only really needs ~8TB total? Will I regret it if I get a DAS instead, would a DAS end up being any cheaper realistically? Looking at different devices, it doesn't seem like there's that much of a difference. Another option could be sticking 2 internal HDDs in my desktop PC and continue with the system I have in place, accumulating internal HDDs instead of external ones? DIY NAS would be a fun project, but I can't say I have a ton of time to really learn the ins and outs. Scouring FB marketplace for an old PC to repurpose seems a bit risky, unless anyone here thinks it's really a relevant option?

Hoping to get some insight here, thank you!

The original post: /r/datahoarder by /u/Icy-Tennis-2581 on 2024-12-09 18:30:02.

Since Tiktok may get banned in the US, I wanna save my posts from my account all at once. I have a LOT of posts on there, and I want to save them all so I can still look back at the memories I made, but it would be extremely tedious to save them one, by one, by one. I need help finding a website of some sort that'l allow me to save all my posts in bulk. (without a paywall)

I tried to save them by downloading my account data from the app itself. The files for my videos work, but any files for my slideshow posts I can't open. I keep getting this

"Access Denied

You don't have permission to access "(file link to my slideshow post)" on this server"

I tried myfavett, but it never works because I keep getting this:

"Error @ Qze176: Uaq119{"statusCode":200,"mime":"application/json","contentLength":"0"}

Tiktok server is not returning the list of your followings today. It should be a temporary issue on TikTok's side. Please try on another day.

To confirm, open your own page https://www.tiktok.com/@(username) and click "following" to reveal your following list. If you can't get the actual list, it means tiktok is broken. If you discover any tricks that lead to your following list on tiktok.com, please tell myfaveTT developer."

Don't suggest any githubs because I'm too stupid to figure out code. Tried it before, and I cannot for the life of me get it to work. + I don't know of any safe installations because I don't want to risk getting a virus and messing up my computer. At this point I don't know what else do and I'm about to give up and just tediously download them one. by. one. by. one. but it'll take ffffffffffoooooooooooooorrrrrrrrrrrrreeeeeeeeeeeeeeevvvvvvvvvvvveeeeeeeeeerrrrrrrrrr

The original post: /r/datahoarder by /u/EwigNoob on 2024-12-09 18:20:36.

I've always accumulated a lot of different types of files. However, recently I've been learning A LOT of stuff that has given me a different focus. The problem is that I also discovered that 3 hard drives amounting to about 6TB of data are not in perfect health. With a mix of deleting unnecessary and redundant files, Handbrake, and some extra cloud storage, I should be able to absorb that without needing to rush the purchase of a new hard drive.

I've read the rules and will take technical questions to r/techsupport. Here, I'd like to ask other questions, of a more practical order in relation to my files, basically which ones to keep.

1. If I two versions of the same file, how do I decide which one to keep? Particularly when it comes to video? If image and sound are somewhat similar, should I choose based on bit rate? Should I always trust source files more? Should I always prefer the most compressed one?

2. Say I download two torrents of the same anime, one pack has more subtitles and the other is dual audio. Is it better to choose one? Which one? Or should I transfer the subtitles from one to the other? That would kill the torrent how long should I wait?

3. Any tips for compressing large amounts of files with Handbrake? If I plan to enhance/upscale the file, should I slip downloading it? I assume so, but would like to confirm. I usually use a H.265 10-bit setting with RF 26 or 27.

4)If I decide to get rid of files, PDF books, audio files, series, etc. Any advice on a place to dump them, in case one man's trash is another man's treasure?

The original post: /r/datahoarder by /u/Important_Pepper8832 on 2024-12-09 17:45:07.

Hey, so someone on here posted a while ago about being able to restore deleted soundcloud tracks from the waveform URL, and I was wondering if anyone still knows how to do this. Apparently the uid and waveform url is in track metadata. Link to OG post

The original post: /r/datahoarder by /u/sun_arcobaleno on 2024-12-09 17:26:26.

Has anybody 3D printed a 2.5" SSD Caddy for the Jonsbo N5, the one that goes into the 3.5" inch bays? Does anybody have the right 3D models for it?

I have no 3D printer and only basic skills in 3D modelling so I commissioned a 3D printing service to print this onefrom Thingiverse. Its for the N3 but I figured its got the same 3.5" inch bays and its dimensions could be the same. Sure enough it fits, but the tolerances for the SATA slot is a bit tight, see this image https://imgur.com/a/QD7eWH7

I have also tried printing a bunch of other SSD caddies, but one I could use with the vibration mounts given with the N5 case, like a normal 3.5 HDD would be.

https://www.thingiverse.com/thing:2672076/files

https://www.thingiverse.com/thing:29628/files

None has worked so far. I'm running out of options and commissioning a 3D printing service ain't cheap. I'm just shooting a shot that maybe someone has already tried but haven't uploaded or maybe I haven't just seen it. Maybe someone with resources could help and try? I dunno but any help would be appreciated.

The original post: /r/datahoarder by /u/svennirusl on 2024-12-09 17:23:05.

Is there any affordable way to make an SSD based MacMini media server? I've got a Synology NAS now, and it's quite slow.

I'm curious if there are any boxes that you can use to raid ssds together that make sense? I know it's not as reliable if its external, I don't know if that's absolute lunacy or just a sort of bad idea.