I won't comment on legality of anything, but if you encrypt your data it'll be impossible for anyone to tell what it is. Rclone has an option to do this.
this post was submitted on 14 Mar 2024
38 points (95.2% liked)
Selfhosted
39893 readers
462 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
founded 1 year ago
MODERATORS
Can I do incremental backups with rclone whilst encrypting it?
Idk about rclone but you can do that with Borg backup.
Borg is a great tool, you should check it out
But Borg does not work with object storage, it needs a borg process on the receiving side.
Another alternative then would be Restic. That's what I'm using for backups
Or rustic! It is compatible with restic but has some nice additions, for example the fact that supports a config files. It makes operations a bit easier IMHO (I am currently using both).
Thank you, I'll take a look
I've used restic before and it worked great with OVH's object storage. Moved away from cloud backups because of the cost though.
Yes.
Yes, that's how I do backups
Do you create one large binary blob or do you encrypt each file separately?
separately, that's how I can maintain partial backups. The rclone crypt stuff is good at keeping track on huge amounts of files quickly
Thanks, I'll take a look
I could be wrong, but I remember reading some companies using AI to analyze your data traffic. Even when encrypted, they may be able to tell that it must be video streaming of some sort. Many providers ban video streaming altogether, legal or not.
Sounds like OP is asking about file storage. Video streaming could be spotted using info leaked regarding traffic behavior. But uploading an encrypted file for storage shouldn't leak anything except the size.
Yes but they cannot determine it isn't audio of me singing in my own shower
I'd say encrypt before upload using rclone or similar, regardless of what provider you choose.
Oh absolutely, I was just wondering about the way I should do it: encrypt the entire thing into a binary blob, or each piece of media encrypted separately?
I like cryfs for this purpose:
Cryptomator is far more mature for this purpose with far better cross platform support: https://cryptomator.org/
encrypted separately like by album you can, even have rclone change the name and use hashes then it does not matter what you use long as they're keep your data safe. also it is best to keep a copy in the EU and US if you can.
This is very helpful! I would certainly be interested in something like that. How do you manage your encryption keys, and do you rotate them regularly? What does the process look like for you? The idea of changing metadata and hashing them is very appealing.
Use encryption if you are concerned.
Thanks, I'm looking into it
For a short period of time, nobody will care, especially with almost zero traffic.
But for only up to 3tb, you can easily fit that on a single drive.
Hey on that 3tb thing… can I pick and choose folders or parts of folders from the backups?
Or, say, I want to move stuff around in folders… is that possible after the backup has reached them and is only visible to me through their web interface?
Depends a lot on what backup software you use. Blackbase B2 ist just an S3-like object storage service. It's the underlying software stack of many different things, one of those can be backup software. They do have their own backup solution though. But in that case B2 is the wrong product for you to look at.
I wrote a WebDAV server you can use as the frontend for it that can encrypt everything you upload.
https://hub.docker.com/r/sciactive/nephele
There’s something I’m using that Backblaze doesn’t support, so I’m fixing that right now.
Edit: Ok, I fixed the issue and updated it on Docker. It now works with Backblaze. It’s a bit slow, but I’m not sure if that’s just my connection or Backblaze.
I've been using Backblaze B2 since 2020 for this exact purpose and I've had no issues. It's been reliable and cheap to run. I stream to my phone from their servers with no issues.
I'm very interested in your use-case: could you tell me how you achieved streaming from their servers to your mobile? What services do you use and how did you set that up?
Storj seems hard to beat regarding price.
Thanks for mentioning them, do they seem to be as reliable?
Never had a problem. According to their site, reliability seems great.
Just FYI, koofr has a lifetime deal with 1tb costing $120. At about $4 a month for storj, you're looking at a cost savings in just under 3 years. So if you intend to keep the storage, and assume koofr will still be there in 3 years, that's another good way to go.
I’ve got about 12 TB in Amazon Glacier. Granted, I’m using this as an offsite backup for my NAS, rather than regularly accessed for consumption.
That's very expensive to recall. Glacier download prices are extremely expensive
But isn't that the point? You pay a low fee for inconvenient access to storage in the hope you never need it. If you have a drive failure you'd likely want to restore it all. In which case the bulk restore isn't terrible in pricing and the other option is, losing your data.
I guess the question of whether this is a service for you is how often you expect a NAS (that likely has redundancy) to fail, be stolen, destroyed etc. I would expect it to be less often than once every 5 years. If the price to store 12TB for 5 years and then restore 12TB after 5 years is less than the storage on other providers, then that's a win, right? The bigger thing to consider is whether you're happy to wait for the data to become available. But for a backup of data you want back and can wait for it's probably still good value. Using the 12TB example.
Backblaze, simple cost. $6x12 = $72/month which over a 5-year period would be $4320. Depending on whether upload was fast enough to incur some fees on the number of operations during backup and restore might push that up a bit. But not by any noticeable amount, I think.
For amazon glacier I priced up (I think correctly, their pricing is overly complicated) two modes. Flexible access and deep archive. The latter is probably suitable for a NAS backup. Although of course you can only really add to it, and not easily remove/adjust files. So over time, your total stored would likely exceed the amount you actually want to keep. Some complex "diff" techniques could probably be utilised here to minimise this waste.
Deep archive
12288 put requests @ $0.05 = $614.40
Storage 12288GB per month = $12.17 x 60 = $729.91
12288 get requests @ $0.0004 = $4.92
12288GB retrieval @ $0.0025 / GB x 12288 = $30.72 (if bulk possible)
12288GB retrieval @ $0.02 / GB x 12288 = $245.76 (if bulk not possible)
Total: $1379.95 / $1594.99
Flexible
12288 put requests @ $0.03 = $368.64
Storage 12288GB per month = $44.24 x 60 = $2654.21
12288 get requests @ $0.0004 = $4.92
12288GB retrieval @ $0.01 / GB x 12288 = $122.88
Total: $3150.65
In my mind, if you just want to push large files you're storing on a high capacity NAS somewhere they can be restored on some rainy day sometime in the future, deep archive can work for you. I do wonder though, if they're storing this stuff offline on tape or something similar, how they bring back all your data at once. But, that seems to me to be their problem and not the user's.
Do let me know if I got any of the above wrong. This is just based on the tables on the S3 pricing site.
Aren't you charged data egress fees/S3 standard storage fees when they are preparing your bulk retrieval?
Not sure, this wasn't clear to me from their pricing page. There were 4 stars next to that item but the explanation for that didn't elaborate on bulk retrieve.
I assumed there was some minimum number of operations, or it had to be the entire backup restored to count as bulk.
Without egress fees, glacier pricing is amazing. But I need to talk to someone who has actually restored from glacier to be able to consider it.
I use it as a last resort backup for things that are worth the recall price if I lose them
Definitely not the cheapest option. I should probably look at migrating somewhere else one of these days. At least it was easy to set up for my Synology NAS.
What about compressing it into 7z password protected archives and uploading it to any good host like MEGA?
I don’t use b2, rather; I have a personal account. Backing up 3 computers and they’ve never said anything over years So. Yeah it’s fine
I have a personal account. Backing up 3 computers and they’ve never said anything over years
Until you need to use the backup and the process is like shit. And takes weeks to months.
Yeah, has anyone ever actually tried restoring from then? I only remember one disgruntled redditor posting about it, but that's about it.