It's A Digital Disease!

The original post: /r/datahoarder by /u/-ThatGingerKid- on 2024-12-27 08:23:41.

I'm very confused, I've received about a bazillion different takes on what is permitted in Cloudflare's ToS.

So, if I have a media server and I want to make it accessible to the outside world via a web domain WITHOUT port dorwarding or a VPN, my understanding is that i can do this with a Zero Trust Tunnel or a reverse proxy service like Caddy with a service like Cloudflare.

However, it's also my understanding that Cloudflare doesn't like large amounts of non-html data served on free plans. There are many different takes on what is permitted since changes were made to ToS wording, and I don't know If this applies just to Zero Trust Tunnels or all of Cloudflare services, including of used with a reverse proxy.

What is allowed, and what is the best and safest way to serve media from a home server with a .com?

The original post: /r/datahoarder by /u/level3comm on 2024-12-27 08:18:46.

Hi Everyone! I recently encountered a failing nvme drive in my main PC and saved 99% of it using the amazing GNU ddrescue tool. (https://www.gnu.org/software/ddrescue/) Sadly there were one or two files in my NTFS partition that were not saveable according to ddrutility, another amazing tool that with NTFS partitions can tell you exactly which files are bad. (https://sourceforge.net/p/ddrutility/wiki/Home/)

After getting worried by this experience, I also used the tool to recover two older external HDDs that had been sitting on a self and they also had bad sectors but running the tool overnight with a lot of retries, I was able to save 99% of sectors with no NTFS file system errors.

Anyway, my question is that I'm now stuck with these raw disk images that are the size of the entire drives, as shown in the photo. However the NTFS partition that spans the drives is only 40% full in the case of the 4TiB partition with the rest being for the most part almost all 0s.

So, I would imagine these files would compress really well with even gzip, but the more reading I have done the more lost I am on how I can ever safely delete these raw DD images.

It seems like the safest thing to do would be to copy these files to a ZFS partition and run regular scrubs to ensure they are not corrupt, but that would waste 60% of the storage space tracking the 60% of the empty NTFS partition on the drive.

The creator of lzip has a very long article(https://www.nongnu.org/lzip/safety//_of//_the//_lzip//_format.html) on why it is the best tool for error detection and recovery for large data stream compression, and I believe the recommendation of the lzip creator would be for me to TAR all the drive images I have in the photo together and then use lzip to compress the TAR.

However when researching additional LZMA compression technologies that competed with lzip I ran into the concept of the PAR file as an alternative to a TAR for stream compression to get even more redundancy and error detection/recovery.

Some folks pointed out though, that for this type of long term storage, its much better to use a file archive compression tool rather than a stream compression tool and I was recommended WIN-RAR which actually incorporates PAR files into the archives. I don't mind paying for that software if its the best tool for this job, but the settings page for the PAR portion is quite complex... I don't really know how much redundancy I want or need, it would be nice if I new the probability of a bit flip occurring and what it might do if undetected, but I don't know what that would be given this setup.

So I guess my question is, does anyone have experience archiving large sparse compressible DD images in a safe way, I'm leaning towards just copying them to my ZFS Zraid pool and taking the space hit to ensure ensure the ZFS checksums will keep the files safe, but if someone more knowledgeable about lzip/WIN-RAR can recommend settings that have worked well maybe I can delete the originals and store the archives instead.

https://preview.redd.it/cp5v854woc9e1.png?width=1638&format=png&auto=webp&s=4c0398d9e1153524ac5afcbfc0ba0ec709a5e08c

The original post: /r/datahoarder by /u/_thermix on 2024-12-27 08:09:15.

Switched to a laptop and still have my old desktop lying around with all my media in it (curently accessing the media in the external hdd I used for backup and now the PC is the backup), been thinking of turning it into a NAS (for seeding torrents and making backups) but I'm kinda concerned about the GPU drawing too much power and adding to my bill (I know it's just a few bucks per month but still, I don't wanna be wasteful).

Should I unplug the GPU? It won't be plugged into a monitor or running any GUI programs, but it just being there would still draw power, right?

Edit: It's a gtx 1650, and my CPU has a integrated gpu

The original post: /r/datahoarder by /u/mrskymr on 2024-12-27 07:41:22.

The original post: /r/datahoarder by /u/gastralia1 on 2024-12-27 07:19:32.

Hey all. I like to back up a lot of stuff and save lots of files.

Right now i have 2 external drives and 1 USB.

The USB is very slow when it comes to transferring data. I do plan to purchase a 2TB external drive soon. Which one would you recommend? In terms of transfer speeds and cost. Should I just get 2 separate 1TB drives? I'm kind of new to the whole backing up a bunch of files and just data hoarding. so any tips and program that might be helpful would be nice to know. Thank you!

The original post: /r/datahoarder by /u/ExcuseNumerous on 2024-12-27 07:11:47.

So basically I used windows and am always low on space and performance issue, I want to shift to linux but don't have space for dual boot, also my windows have lot of my work.

So I am planning to slowly shoft to linux, and thought to run it on an external SSD(which is out of my budget at this moment) , then I thought to buy a sata SSD with an encloure and run VM on it to launch linux, will it work??

The original post: /r/datahoarder by /u/Temporary_Potato_254 on 2024-12-27 06:58:15.

The original post: /r/datahoarder by /u/Rezasaurus on 2024-12-27 06:48:45.

The original post: /r/datahoarder by /u/Proteus356 on 2024-12-27 05:43:40.

Got a storage server (Truenas) based on a P620 with 6HDD and looking for some more expansion, ideally 8-12 drives. Would prefer SAS if at all possible. I am looking for something storage specific. Quiet and compact that doesn’t look like crap (has to pass WAF as it will be in living room). So far, it looks like some of the NAS vendors make nice gear, but I don’t need or want the NAS part. I just want the case itself. I just need a dumb enclosure with SFF-8087 SAS ports to plug into an LSI SAS card. Something like the Areca 8-12 drive desktop enclosures (but JUST the enclosure without their controller) or the Terramaster T9-12, or the Synology or Qnap 8-12 drive enclosures would be great. Unfortunately everything I’ve found so far is just USB or thunderbolt (useless for Truenas). Oh..and I do NOT need hotswap. If there is something small and elegant, like an SFF PC with a way to fit 8 internal 3.5” drives and a pair of SFF8087 SAS expansion ports, that would be perfect.

So far, the closest I’ve found is a U-NAS ES-810 expansion (on their website, but doesn’t seem to be available anywhere) or the old QNAP TL-D800S (looks nice, but SATA, not SAS, and ridiculously overpriced.

Help!!

The original post: /r/datahoarder by /u/Familiar_Leather on 2024-12-27 04:39:20.

I pay $10.00 a month for Google One so I can use their 2 TB of cloud storage in order to back up important files from my computer. I want to be able to regularly back stuff up, but Google makes it such a pain in the ass to do! Files take hours, if not DAYS to upload sometimes, and I've tried everything I can think of. I've used Firefox. I've used the Google Drive desktop app. I've used Chrome. I've used Edge. My internet is very fast, so I don't understand what the problem is. Is Google throttling the upload speeds for a service I'm paying for? How can I fix this? I like the security that Google offers so I wouldn't switch unless I found a similarly priced and similarly secure, well known and trusted service to switch to.

My internet speeds, according to Speedtest . net are...

Download: 131.88 MBPS

Upload: 169.78 MBPS

(Mind you, I ran this test with Steam, Telegram, and Discord open, as well as streaming a movie and having about 12 tabs open for various websites on Firefox all on the same device, so it's probably actually higher than that if I don't have anything running.)

The original post: /r/datahoarder by /u/AngryMobster on 2024-12-27 04:09:33.

As per title, I have easy access to Seagate products in my region. but i'm majorly confused by the products such as Ironwolf/Firecuda/EXOS/etc. Which one should I go for i'm just looking to store and maybe watch videos on? A high read/write is also preferable.

I almost pulled the trigger on the EXOS but then there were multiple versions of them too like 7E10, 7E8, X16 and such.

The original post: /r/datahoarder by /u/rukawaxz on 2024-12-27 04:08:08.

The original post: /r/datahoarder by /u/SarthakSidhant on 2024-12-27 03:42:03.

edit: ignore this post please - i think i made a mistake or the seller made the msitake

my motherboard - x99 FD4 which is a dual CPU motherboard requires a 2x 30 PIN PSU, i tried finding it online but couldnt get any of them - could you suggest what wattage should i go for considering 8 2TB each - hard disks are connected? it runs a dual E5 2686V4

The original post: /r/datahoarder by /u/ContextMaterial7036 on 2024-12-27 03:15:49.

Title. What if I upgrade my 4TB drive to higher capacity one and copy current files over to it from the current one mapped to iDrive.

Will iDrive want to re-upload all of the files again since the modified timestamp or device changed?

If so, any way to avoid having to do that?

The original post: /r/datahoarder by /u/ThatFeel_IKnowIt on 2024-12-27 02:44:02.

I am massively confused by reading people's experiences with Windows Mirrored Volumes. There are 2 things that are confusing me. I will list them below. But first, I will also list my setup:

My setup: I have 2 internal HDDs in my PC with files on them. The disks are set up as dynamic disks with mirrored volumes. I was under the assumption that if one disk fails, the data will still be on the other. DISCLAIMER: I also have offsite backups. This is NOT my primary backup method. It's just a data protection method. Also, my OS is NOT on these HDDs. It's on a NVME SSD.

Confusion Point #1: I see people saying that if one disk drive dies, then you CANNOT access your data on the other mirrored drive? You have to have BOTH disks active? That makes no sense and defeats the purpose of mirrored volumes, and even goes against what Windows says on their website?

Confusion Point #2: People claim that if you were to reinstall windows or move the two mirrored disks to another PC, it cannot be accessed since you need to have the original disks on the original OS install to access them, plugged into the original SATA ports you used when making the mirrored volumes? This also makes no sense and I am like 99% sure I have indeed reformatted before and had no issues accessing the disks after?

Can anyone help explain these two confusion points to me?

Also, I realize that disk management mirrored volumes are technically "deprecated" however they still seemt to work? Storage spaces is newer but I've heard horror stories with that too..

The original post: /r/datahoarder by /u/MaximusRectum on 2024-12-27 02:30:44.

I just bought a Panasonic Pro-Line AG-1970 vcr and was wondering what else I need to transfer. I see a lot about FireWire, but is this the best way in 2024?

The original post: /r/datahoarder by /u/SnakeEdude on 2024-12-27 01:47:15.

Been using 6 external portable hdd to manage my local backups, and store media files, I want to move up to a DAS 4 or 5 bay to run WD or SG hdd, share your thoughts on some good DAS units? I've been watching some videos on YT and have some idea what I'm looking for.

The original post: /r/datahoarder by /u/O_G_P on 2024-12-27 01:30:42.

I'm on windows 11.

Thank you!

The original post: /r/datahoarder by /u/Effective-Tutor7325 on 2024-12-27 01:29:57.

Hello everyone,

I’m relatively new to the world of technology and am setting up my movie library to stream via Plex. Currently, I have 4 hard drives with a total capacity of approximately 40TB, of which I’m using around 15TB. I have all the Arrs and Plex running on my Raspberry Pi 4 (4GB).

I’m looking for advice on the following:

1. How can I periodically back up my library? Considering that one of the hard drives contains 15TB, I would need to back it up across at least two hard drives.
2. I’d like to back up my library every 2 months. What would be the best way to do this?
3. Would it make sense to invest in a NAS or something similar?

Any help would be greatly appreciated.

The original post: /r/datahoarder by /u/sbourwest on 2024-12-27 01:03:46.

Back last June I bought myself and my dad new computers. We upgraded from our 9 year old Windows 7 PCs to "shiny new Windows 11" pre-builts. We both are data hoarders and have several external hard-drives we use, mostly for storing media files. These are all USB 3.0 HDDs (Mostly Western Digital, some Seagate) and we both have noticed a considerable drop in transfer rate on our new PCs versus our old ones. Where it used to take less than a minute to transfer 1GB of data, now it takes several minutes on some of the older drives. They also seem to be very inconsistent with how often they stay spun up, sometimes they will remain active for hours, while others they will need to spin up after only 10 minutes inactivity. Keep in mind these are all just media drives, the only program that ever accesses them on it's own is our Emby streaming software, but only when we actively stream of course.

I even noticed my old internal secondary HDD I moved from my Windows 7 to my Windows 11 PC is way slower, I used it to store games, and while I do understand HDD is way slower than SSD, I used to play games fine on Windows 7, but now with a beefier gaming rig, games run slower and the only bottleneck is that HDD (which is connected internally via eSATA).

I'm very confused by why Windows 11 just really doesn't play nice with physical data storage solutions compared to older outdated versions? Any insight on this, or better yet, possible solutions I'm missing?

The original post: /r/datahoarder by /u/milkygirl21 on 2024-12-27 00:56:56.

https://www.reddit.com/r/DataHoarder/comments/zuukia/is_there_an_usb_type_c_powered_35_enclosure/

Someone asked 2 years ago and there was no answer. I wonder if there are any new products that can do it?

The original post: /r/datahoarder by /u/Rich_Mood2361 on 2024-12-27 00:48:20.

So my dad poured his last decade into his life's work - https://akomenepitas.in.rs/

I know! An assault on the senses. He was an unusual man, to say the least. But he was on his deathbed talking about the next "season" he had planned out for his site. It meant the world to him. The website will probably go offline at some point in the next few months. I don't have access to his emails, not sure where it's even hosted. I also know it has tons of pages that are not linked to anywhere on the site, I'm wondering if there's a way to find all of them and download the whole thing for archival purposes.

The original post: /r/datahoarder by /u/Dramatic_Squash_6877 on 2024-12-26 23:20:40.

Hello. I have a steam collection of games and i dont like the fact i technically dont own it. is there a software like with dvds that will rip it so i can have a copy of the game that works offline and doesnt have drm? Its only for personal use. I just want to own my stuff.

The original post: /r/datahoarder by /u/kaitlyn2004 on 2024-12-26 21:09:18.

Looking to replace my synology ds413j, which still functions okay as a backup device but is otherwise quite sluggish and limited hard drive support - basically at the max for that.

It’s 99% photos, otherwise documents. I use stuff like Dropbox and Google photos but I may shift towards truly using the NAS as my personal cloud.

I need approximately 8TB and accounting for some reasonable runway, let’s just say 12-14TB. Outside of possibly adding video in, I would expect 14TB to last me TONS of time.

Unlike over a decade ago, can easily get a single 20TB drive. So I figure I could just put those in RAID 1 and have more storage then I know what to do with.

So what reason would I choose a 4bay (or more) model? I’ve heard that if you actually have to rebuild the array, it can take suchhh a long time plus there can be failure during the rebuild. I’d already have another backup copy AND probably a cloud backup I could restore from if that was necessary…

So yeah, given I don’t need 20TB+ of available space, what am I losing out on if I only went with a 2bay?

The original post: /r/datahoarder by /u/deathpulse42 on 2024-12-26 18:05:26.

Context/Set-up:

HP Proliant DL380p Gen 8

HP P822 Smart Array Controller [in HBA/IT/passthrough mode]

Ubuntu 18.04 LTS [bare metal]

ZFS RAIDz2 [0.7.5-1ubuntu16.12] -- currently all 512e/SATA-3 drives

HP D2600 LFF Disk Enclosure [SAS-2 - 6Gbps | SATA-2 - 3Gbps]

TL;DR: Can I use 4kn SAS-3 drives in an 8th gen HP Proliant server running Ubuntu 18.04 LTS with an 8th gen HP RAID card in passthrough mode?

Considering a large storage upgrade to my current set-up. I have a D2600 disk enclosure with all SATA-3/512e drives1, and that's been working great for the last 4 years. No issues.

Another D2600 is on its way -- all that I need to decide on is the drives for a new ZFS array.

I've been doing some spreadsheet work with ebay prices, $/TB, and the like, and after some "best value vs. riskiness" number crunching, many of the best value candidates are refurbished/recertified enterprise [SAS-3 - 12Gbps]2 4kn drives (probably due to lower demand vs 512n/e in the used market because of [1] the SAS interface and [2] the exact uncertainty with legacy storage controller compatibility that I'm concerned about lol).

I have scoured many internet forums over the past week or so, and from what I can tell, HP didn't ever officially support 4kn on their 8th gen RAID cards. No problem, I figured -- I have mine in HBA/IT/passthrough mode for use with ZFS, so no hardware RAID support required! But I saw this post by TechMikeNY that states "4Kn drives – SAS only – could work in these [8th gen or earlier] servers in a non-RAID/pass-through configuration [exactly what I have]. Though the OS running on the server would need to support 4Kn formatted drives as well.  Due to the variables involved, we do not recommend it."

I haven't been able to find any stories/posts where someone has successfully used 4kn SAS drives with 8th gen HP RAID cards in HBA mode with Ubuntu 18.04 LTS. From what I've read about and have understood regarding the whole 512n/512e/4kn discussion, I think these 4kn SAS drives could work, especially since all of my HDDs show up as "512 bytes logical, 4096 bytes physical" anyway. **Also, the OpenZFS documentation describes a process by which one can use ZFS in Ubuntu 18.04 "on a drive which presents 4 KiB logical sectors (a '4Kn' drive)" as a the root filesystem, so I figure the OS support is probably there?

I wonder if anyone has any advice on the feasibility of this combination of parts. I don't necessarily need the SAS benefits, but if I can get them for a small premium (or even less in these cases), I figured I might as well get 2x speeds (albeit at SAS-2) and be able to do a more reliable dual-domain cabling configuration. If not, I can just go with the tried-and-true SATA-3 drives.

TBH, part of me just wants to buy 2-3 cheaper low-capacity 4kn SAS drives and just see if it works as a proof of concept...

Thanks for any help or advice you can provide. :)

Specifically, this is the model number I'd buy if I can prove that the 4kn SAS drives would work:

WD WUH721414AL4200 14TB SAS 12GBPS 4KN 3.5IN HC530 HDD

1 Yes, I realize I can only achieve [SATA-2 - 3 Gbps - 300 MB/s] speeds with this set-up, but it works perfectly fine for my use case -- a Plex server just doing sequential reads and writes of large media files for the most part.

2 Yes, I also realize I would be getting only SAS-2 speeds from these SAS-3 drives, but that's perfectly okay with me, especially since I can't find any older SAS-2 drives with the higher capacities that I'm aiming for. Plus, they are all much older drives in general, and I'd worry about reliability.