don't mess around with imaging to a file on the zfs, then restoring it. simply clone nvme -> nvme using a usb nvme adapter then replace the internal with the clone.
this post was submitted on 22 May 2024
21 points (95.7% liked)
Selfhosted
40220 readers
1102 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
founded 1 year ago
MODERATORS
I would like to avoid buying additional hardware if possible.
usb nvme adapters are not expensive and it likely won't be the only time you need it. they are a handy accessory to have on hand if you have nvme storage.
I'll definitely take this route if the wifi slot will not support the m.2 drive. Thanks for the suggestion.
You can old nvme -> SATA -> new nvme with any old SATA drive you have lying around.
If you want an image, it doesn’t matter what the underlying file system is. You should be able to use a tool like Clonezilla and get a 1:1 copy. Depending how you’ve set up partitioning, you could also use sgdisk to set up the proper partitions and zfs send/recv for the new data portion of the drive and install a boot loader. That’s probably the way I’d go in this instance.
I get the part that the cloning software does not care for the underlying OS. My worry is the fact that I'll run the cloning software/command from a live USB which will not be able to detect the zfs mirror on my backup drive on its own and thus break the zfs mirror with bad consequences for the existing data. I could not find any commands to make the live USB OS discover and respect the existing zfs configuration.
It sounds like nobody actually understood what you want.
You have a non-ZFS boot drive, and a big ZFS pool, and you want to save an image of the boot drive to the pool, as a backup for the boot drive.
I guess you don't want to image the drive while booted off it, because that could produce an image that isn't fully self-consistent. So then the problem is getting at the pool from something other than the system you have.
I think what you need to do is find something else you can boot that supports ZFS. I think the Ubuntu live images will do it. If not, you can try something like re-installing the setup you have, but onto a USB drive.
Then you have to boot to that and zfs import your pool. ZFS is pretty smart so it should just auto-detect the pool structure and where it wants to be mounted, and you can mount it. Don't do a ZFS feature upgrade on the pool though, or the other system might not understand it. It's also possible your live kernel might not have a new enough ZFS to understand the features your pool uses, and you might need to find a newer one.
Then once the pool is mounted you should be able to dd your boot drive block device to a file on the pool.
If you can't get this to work, you can try using a non-ZFS-speaking live Linux and dding your image to somewhere on the network big enough to hold it, which you may or may not have, and then booting the system and copying back from there to the pool.
Phew! I almost believed I was asking for something beyond the scope of linux-fu. English not being my first language may be part of the reason but still I think I covered everything that was relevant.
Yes, that's exactly what I want and your post has given me the clarity I needed. M.2 wifi slots don't support disks so that option is definitely out. I'm going to boot with the latest Ubuntu live OS on a USB and attempt what you've outlined.
I don't have anything really critical on the zfs that is not backed up separately so I'm definitely going to attempt this and learn in the process.
Thank you for taking the time to respond!
Use an Ubuntu live USB, all recent versions of Ubuntu have ZFS drivers baked into the live environment. Then you should add your new SSD to the ZFS pool, and remove the old one from the ZFS pool. Your m.2 WiFi slot should be able to host the 2nd drive while you do this, but if not you can use an external USB housing for it, you'll just have to make sure that the ZFS pool knows its UUID so that it knows it's the same drive.
Why do I need to add my nvme to the zfs pool? That doesn't really make sense.
If the wifi slot does support m.2 drivers (I was just looking for some confirmation/document), it'll solve my problem. Thanks nonetheless.
You need to add the new drive to your existing pool because ZFS stores data across all drives by default, similar to a RAID0. Then you remove the old drive and ZFS will automatically copy the data off the failing drive onto the healthy one and allow you to remove the failing drive with no data loss.
The SSD is 256GB while the two HDDs are 4TB each. What kind of zfs config/array do you suggest I create from them?
That depends on your use case, I just did a simple zpool with no redundancy because I wanted maximum speed/capacity and all my data is backed up on an external HDD. If you need redundancy, I would look online for how to configure that and what the optimal setup is.
Actually, I assumed you just had the SSD, if you have more than 256gb of free space between those HDDs, you can go ahead and remove the SSD from your zpool right now (unless your bootloader is there, then you'll have to make an EFI system partition on one of the HDDs and install a bootloader first)
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:
| Fewer Letters | More Letters |
|---|---|
| SATA | Serial AT Attachment interface for mass storage |
| SSD | Solid State Drive mass storage |
| ZFS | Solaris/Linux filesystem focusing on data integrity |
[Thread #758 for this sub, first seen 22nd May 2024, 23:15] [FAQ] [Full list] [Contact] [Source code]
I think you want to see "zfs import" command, to get your pool back con the new OS.
Yes, the other answer also suggests this and I think this will do the trick. Thank you for your response.
You could use something like Veeam Endpoint (free), it will image while the OS is running so the underlying ZFS system shouldn't matter.