Disk config with GRUB/EFI, RAID + LVM

[Andrew Ruthven]

Nice, good work.

A couple of suggestions...

I'd put swap on RAID1, if you have a disk go offline while the system is
running, you've lost anything that had been swapped out to the swap on the
failed disk.

I like to put a 100MB partition at the end of disks in a RAID pair to allow
for slightly differing size of a replacement disk. Although currently it
seems that any new disk we get is many many GB bigger than the old failed
one.

I've found that some servers allow the EFI partition to be RAID1, the
current servers we're using support that. So no need to copy files around.

Cheers,
Andrew

On Thu, 2023-11-30 at 17:20 +0100, Rémy Dernat wrote:

> Ok, FYI, I think I have a working configuration with :
> 
> ```
> # example of new config file for setup-storage
> #
> # <type> <mountpoint> <size>   <fs type> <mount options> <misc
> options>
> 
> disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1
> primary - 2G - -
> primary /boot/efi 1G vfat rw createopts="-F32"
> primary swap 2G swap sw,pri=1
> primary - 0- - -
> 
> #disk_config disk2 sameas:disk1
> disk_config disk2 disklabel:gpt fstabkey:uuid
> primary - 2G - -
> primary /boot/efi2 1G vfat rw createopts="-F32"
> primary swap 2G swap sw,pri=1
> primary - 0- - -
> 
> disk_config raid fstabkey:uuid
> #raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose
> --metadata=1.0" createopts="-L boot"
> raid1 /boot disk1.1,disk2.1 vfat rw,noatime mdcreateopts="--verbose
> --metadata=0.90"
> raid1 - disk1.4,disk2.4 - -
> 
> disk_config lvm fstabkey:uuid
> vg vg_system md1
> vg_system-root / 60G-200G ext4 rw,noatime
> vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev
> ##vg_system-scratch /scratch 30%- ext2 rw,noatime
> vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev
> ```
> 
> My previous message was crystal clear. md0 did not have any formatted
> filesystem on it. So I added `vfat` option on /boot.
> 
> Best regards,
> Rémy
> 
> 
> Le jeu. 30 nov. 2023 à 16:53, Rémy Dernat
<[remy.d1 at gmail.com](mailto:remy.d1 at gmail.com)> a écrit :
> 
> > 
> > What is strange here, is that md1 is found, as well as md0, but
> > contrary to md0, md1 have an UUID ...
> > 
> > However, UUID seems to be retrieve by blkid from /dev/disk/by-id,
> > where both md array have their own mdadm UUID, but only md1 have an
> > UUID from LVM (with pvdisplay / pvs ) ...
> > 
> > 
> > Le jeu. 30 nov. 2023 à 14:52, Rémy Dernat
<[remy.d1 at gmail.com](mailto:remy.d1 at gmail.com)> a écrit :
> > 
> > > 
> > > Hi Andrew,
> > > 
> > > Indeed, this removed the syntax issue. However, I have a new error
now:
> > > 
> > > ```
> > > /sbin/blkid -c /dev/null -s UUID -o value /dev/md0 had exit code 2
> > > Command had non-zero exit code
> > > ```
> > > Indeed, blkid is not able to found md0 UUID.
> > > 
> > > I tried to modify the disk_config file to avoid this error, but I am
> > > still stuck with it. Here is what look like my disk_config file now:
> > > 
> > > ```
> > > # example of new config file for setup-storage
> > > #
> > > # <type> <mountpoint> <size>   <fs type> <mount options> <misc
options>
> > > 
> > > disk_config disk1
> > > primary - 2G - -
> > > primary /boot/efi 1G vfat rw createopts="-F32"
> > > primary swap 2G swap sw,pri=1
> > > primary - 0- - -
> > > 
> > > #disk_config disk2 sameas:disk1
> > > disk_config disk2 disklabel:gpt fstabkey:uuid
> > > primary - 2G - -
> > > primary /boot/efi2 1G vfat rw createopts="-F32"
> > > primary swap 2G swap sw,pri=1
> > > primary - 0- - -
> > > 
> > > disk_config raid fstabkey:uuid
> > > raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose
> > > --metadata=0.90" createopts="-L boot"
> > > raid1 - disk1.4,disk2.4 - -
> > > 
> > > disk_config lvm fstabkey:uuid
> > > vg vg_system md1
> > > vg_system-root / 60G-200G ext4 rw,noatime
> > > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev
> > > ##vg_system-scratch /scratch 30%- ext2 rw,noatime
> > > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev
> > > ```
> > > 
> > > I also tried to change the raid metadata to 1.0, but that did not
> > > change anything. I wrote back ` fstabkey:uuid` behind disk_config raid
> > > because it does not fail with syntax error.
> > > 
> > > Thanks for help
> > > 
> > > Le jeu. 30 nov. 2023 à 11:55, Andrew Ruthven
<[andrew at etc.gen.nz](mailto:andrew at etc.gen.nz)> a écrit :
> > > 
> > > > 
> > > > Hey,
> > > > 
> > > > On Thu, 2023-11-30 at 11:14 +0100, Rémy Dernat wrote:
> > > > 
> > > > > Hi awesome Fai guys,
> > > > > 
> > > > > I have a weird syntax issue with the following disk_config:
> > > > > 
> > > > 
> > > > [snip]
> > > > 
> > > > > disk_config raid disklabel:gpt fstabkey:uuid bootable:1
> > > > 
> > > > 
> > > > This line is your problem, remove everything after the word raid. So
it
> > > > should just be:
> > > > 
> > > > disk_config raid
> > > > 
> > > > Cheers,
> > > > Andrew
> > > > --
> > > > Andrew Ruthven, Wellington, New Zealand
> > > > [andrew at etc.gen.nz](mailto:andrew at etc.gen.nz)         |
> > > > Catalyst Cloud:           | This space intentionally left blank
> > > >  [https://catalystcloud.nz](https://catalystcloud.nz) |
> > > > 
> > > 
> > 
>


```
-- 
Andrew Ruthven, Wellington, New Zealand
andrew at etc.gen.nz |
Catalyst Cloud: | This space intentionally left blank
 https://catalystcloud.nz |


```