Disk config with GRUB/EFI, RAID + LVM

Fri Dec 1 12:11:09 CET 2023

Hi Andrew,

Thanks for suggestions.

Now, my file "disk_config/GRUB_EFI_RAID1" is [*]:

```
# example of new config file for setup-storage
#
# <type> <mountpoint> <size>   <fs type> <mount options> <misc options>

disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1
primary - 2G - -
primary /boot/efi 1G vfat rw createopts="-F32"
primary - 2G - -
primary - 0- - -

#disk_config disk2 sameas:disk1
disk_config disk2 disklabel:gpt fstabkey:uuid
primary - 2G - -
primary /boot/efi2 1G vfat rw createopts="-F32"
primary - 2G - -
primary - 0- - -

disk_config raid fstabkey:uuid
raid1 /boot disk1.1,disk2.1 ext4 rw,noatime mdcreateopts="--verbose
--metadata=0.90"
raid1 swap disk1.3,disk2.3 swap sw
raid1 - disk1.4,disk2.4 - -

disk_config lvm fstabkey:uuid
vg vg_system md2
vg_system-root / 60G-200G ext4 rw,noatime
vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev
##vg_system-scratch /scratch 30%- ext2 rw,noatime
vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev
```

=> Even if the installation succeeded, grub does not seem to work.

Anyway, I am wondering how you set this in the disk_config file (syntax ?):

> I like to put a 100MB partition at the end of disks in a RAID pair to allow
> for slightly differing size of a replacement disk. Although currently it
> seems that any new disk we get is many many GB bigger than the old failed
> one.

Thanks again,

Best regards,
Rémy

[*] there was an error for /boot partition which was not formatted the
right way (vfat does not allow symbolic links). vfat should be used
for /boot/EFI only.

Le ven. 1 déc. 2023 à 05:43, Andrew Ruthven <andrew at etc.gen.nz> a écrit :
>
> Nice, good work.
>
> A couple of suggestions...
>
> I'd put swap on RAID1, if you have a disk go offline while the system is
> running, you've lost anything that had been swapped out to the swap on the
> failed disk.
>
> I like to put a 100MB partition at the end of disks in a RAID pair to allow
> for slightly differing size of a replacement disk. Although currently it
> seems that any new disk we get is many many GB bigger than the old failed
> one.
>
> I've found that some servers allow the EFI partition to be RAID1, the
> current servers we're using support that. So no need to copy files around.
>
> Cheers,
> Andrew
>
> On Thu, 2023-11-30 at 17:20 +0100, Rémy Dernat wrote:
>
> > Ok, FYI, I think I have a working configuration with :
> >
> > ```
> > # example of new config file for setup-storage
> > #
> > # <type> <mountpoint> <size>   <fs type> <mount options> <misc
> > options>
> >
> > disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1
> > primary - 2G - -
> > primary /boot/efi 1G vfat rw createopts="-F32"
> > primary swap 2G swap sw,pri=1
> > primary - 0- - -
> >
> > #disk_config disk2 sameas:disk1
> > disk_config disk2 disklabel:gpt fstabkey:uuid
> > primary - 2G - -
> > primary /boot/efi2 1G vfat rw createopts="-F32"
> > primary swap 2G swap sw,pri=1
> > primary - 0- - -
> >
> > disk_config raid fstabkey:uuid
> > #raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose
> > --metadata=1.0" createopts="-L boot"
> > raid1 /boot disk1.1,disk2.1 vfat rw,noatime mdcreateopts="--verbose
> > --metadata=0.90"
> > raid1 - disk1.4,disk2.4 - -
> >
> > disk_config lvm fstabkey:uuid
> > vg vg_system md1
> > vg_system-root / 60G-200G ext4 rw,noatime
> > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev
> > ##vg_system-scratch /scratch 30%- ext2 rw,noatime
> > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev
> > ```
> >
> > My previous message was crystal clear. md0 did not have any formatted
> > filesystem on it. So I added `vfat` option on /boot.
> >
> > Best regards,
> > Rémy
> >
> >
> > Le jeu. 30 nov. 2023 à 16:53, Rémy Dernat
> <[remy.d1 at gmail.com](mailto:remy.d1 at gmail.com)> a écrit :
> >
> > >
> > > What is strange here, is that md1 is found, as well as md0, but
> > > contrary to md0, md1 have an UUID ...
> > >
> > > However, UUID seems to be retrieve by blkid from /dev/disk/by-id,
> > > where both md array have their own mdadm UUID, but only md1 have an
> > > UUID from LVM (with pvdisplay / pvs ) ...
> > >
> > >
> > > Le jeu. 30 nov. 2023 à 14:52, Rémy Dernat
> <[remy.d1 at gmail.com](mailto:remy.d1 at gmail.com)> a écrit :
> > >
> > > >
> > > > Hi Andrew,
> > > >
> > > > Indeed, this removed the syntax issue. However, I have a new error
> now:
> > > >
> > > > ```
> > > > /sbin/blkid -c /dev/null -s UUID -o value /dev/md0 had exit code 2
> > > > Command had non-zero exit code
> > > > ```
> > > > Indeed, blkid is not able to found md0 UUID.
> > > >
> > > > I tried to modify the disk_config file to avoid this error, but I am
> > > > still stuck with it. Here is what look like my disk_config file now:
> > > >
> > > > ```
> > > > # example of new config file for setup-storage
> > > > #
> > > > # <type> <mountpoint> <size>   <fs type> <mount options> <misc
> options>
> > > >
> > > > disk_config disk1
> > > > primary - 2G - -
> > > > primary /boot/efi 1G vfat rw createopts="-F32"
> > > > primary swap 2G swap sw,pri=1
> > > > primary - 0- - -
> > > >
> > > > #disk_config disk2 sameas:disk1
> > > > disk_config disk2 disklabel:gpt fstabkey:uuid
> > > > primary - 2G - -
> > > > primary /boot/efi2 1G vfat rw createopts="-F32"
> > > > primary swap 2G swap sw,pri=1
> > > > primary - 0- - -
> > > >
> > > > disk_config raid fstabkey:uuid
> > > > raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose
> > > > --metadata=0.90" createopts="-L boot"
> > > > raid1 - disk1.4,disk2.4 - -
> > > >
> > > > disk_config lvm fstabkey:uuid
> > > > vg vg_system md1
> > > > vg_system-root / 60G-200G ext4 rw,noatime
> > > > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev
> > > > ##vg_system-scratch /scratch 30%- ext2 rw,noatime
> > > > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev
> > > > ```
> > > >
> > > > I also tried to change the raid metadata to 1.0, but that did not
> > > > change anything. I wrote back ` fstabkey:uuid` behind disk_config raid
> > > > because it does not fail with syntax error.
> > > >
> > > > Thanks for help
> > > >
> > > > Le jeu. 30 nov. 2023 à 11:55, Andrew Ruthven
> <[andrew at etc.gen.nz](mailto:andrew at etc.gen.nz)> a écrit :
> > > >
> > > > >
> > > > > Hey,
> > > > >
> > > > > On Thu, 2023-11-30 at 11:14 +0100, Rémy Dernat wrote:
> > > > >
> > > > > > Hi awesome Fai guys,
> > > > > >
> > > > > > I have a weird syntax issue with the following disk_config:
> > > > > >
> > > > >
> > > > > [snip]
> > > > >
> > > > > > disk_config raid disklabel:gpt fstabkey:uuid bootable:1
> > > > >
> > > > >
> > > > > This line is your problem, remove everything after the word raid. So
> it
> > > > > should just be:
> > > > >
> > > > > disk_config raid
> > > > >
> > > > > Cheers,
> > > > > Andrew
> > > > > --
> > > > > Andrew Ruthven, Wellington, New Zealand
> > > > > [andrew at etc.gen.nz](mailto:andrew at etc.gen.nz)         |
> > > > > Catalyst Cloud:           | This space intentionally left blank
> > > > >  [https://catalystcloud.nz](https://catalystcloud.nz) |
> > > > >
> > > >
> > >
> >
>
>
> ```
> --
> Andrew Ruthven, Wellington, New Zealand
> andrew at etc.gen.nz |
> Catalyst Cloud: | This space intentionally left blank
>  https://catalystcloud.nz |
>
>
> ```
>
>