Advanced users • Re: RAID 1 (mirror) with two NVMe

I saw the following instructions on setting up SW raid, but not sure how this would work on the main boot drive

It won't. The firmware doesn't support booting from a RAID array.

You can put just the boot partition on SD but you'll likely need to modify the initramfs to have your root partition on RAID or on btrfs.

Couple of things to note:
1. RAID does not protect against all forms of data loss. Backups are still required.
2. Write performance will be halved as data will need to be sent over the PCIe bus twice.
3. You'd get more bandwidth per drive by putting one on each USB 3 port.

...but yes, it can.

A few tricks need to be applied, though:

1) the bootfs has to be RAID 1 with 0.90 metadata. This way, the firmware "sees" no difference compared with a non-RAID rootfs. Other partitions can use 1.2 metadata and RAID1 or RAID 10 (also possible with 2 disks, increasing read performance on USB disks).
2) after setting up the RAID 1 bootfs (on type fd RAID partitions), change the partition type for the bootfs back to c - this way the bootloader accepts it. Since the boot partition is strictly read-only during initramfs time, there is no danger for the data. The MD driver is unaffected by this, it auto-assembles the RAID partitions using the MD fingerprint regardless of the partition type. So all system updates (incl. firmware and initrd rebuilds) are automatically applied to both mirrors.
3) change the root=.. stanza in cmdline.txt to root=LABEL=RASPIMDROOT and label the root filesystem within RAID accordingly
4) adapt /etc/fstab in the MD Root partition; I use the MD UUIDs defined during MD creation, for example

ID=md-uuid-0af3926f:eefa7040:5ee071e6:c4c2d960 /boot/firmware vfat defaults 0 2
ID=md-uuid-0af3926f:eefa7040:5ee071e6:c4c2d961 / ext4 defaults,noatime 0 1

This way, I don't even need a customized mdadm.conf file; I usually just uncomment the MAILADDR line in mdadm.conf.

5) the initrd has to contain all necessary raid drivers. In recent versions of the RPI OS, the initramfs.conf MODULES= entry has changed from 'most' to 'dep' - making the initrd a few megabytes smaller, but it then can't boot into a MD RAID system anymore. Change it back to 'most', edit the initramfs-tools/modules file to add raid1 and perhaps raid10 and rebuild the initramfs.
6) add md-mod.start_dirty_degraded=1 to cmdline.txt - this way the system boots even when only one disk is available.

Meanwhile I install every new system this way using ansible, even if I start with only one disk. I always can add a mirror later if desired. With just one drive, the system comes up with "clean,degraded" arrays and operates just like a Non-RAID system.

Regards
Marcel

Statistics: Posted by abuelomg — Sun Nov 24, 2024 12:23 pm

---