Disk config with GRUB/EFI, RAID + LVM
Rémy Dernat
remy.d1 at gmail.com
Tue Dec 5 13:15:04 CET 2023
Hi,
After digging some time around this issue, I am finally able to boot
the RAID1 partition with EFI boot enabled on it (using this example:
https://fai-project.org/download/misc/disk_config/lvm-on-luks-on-raid).
Here is my final disk_config file:
```# example of new config file for setup-storage
#
# <type> <mountpoint> <size> <fs type> <mount options> <misc options>
disk_config disk1 disklabel:gpt bootable:1 fstabkey:uuid
primary - 2G - -
primary - 0- - -
disk_config disk2 sameas:disk1
disk_config raid
raid1 /boot/efi disk1.1,disk2.1 vfat rw,noatime
mdcreateopts="--verbose --metadata=0.90"
raid1 - disk1.2,disk2.2 - -
disk_config lvm
vg vg_system md1
vg_system-swap swap 16G swap sw
vg_system-root / 60G-200G ext4 rw,noatime
vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev
vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev
```
Best regards,
Rémy
Le ven. 1 déc. 2023 à 12:11, Rémy Dernat <remy.d1 at gmail.com> a écrit :
>
> Hi Andrew,
>
> Thanks for suggestions.
>
> Now, my file "disk_config/GRUB_EFI_RAID1" is [*]:
>
> ```
> # example of new config file for setup-storage
> #
> # <type> <mountpoint> <size> <fs type> <mount options> <misc options>
>
> disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1
> primary - 2G - -
> primary /boot/efi 1G vfat rw createopts="-F32"
> primary - 2G - -
> primary - 0- - -
>
> #disk_config disk2 sameas:disk1
> disk_config disk2 disklabel:gpt fstabkey:uuid
> primary - 2G - -
> primary /boot/efi2 1G vfat rw createopts="-F32"
> primary - 2G - -
> primary - 0- - -
>
> disk_config raid fstabkey:uuid
> raid1 /boot disk1.1,disk2.1 ext4 rw,noatime mdcreateopts="--verbose
> --metadata=0.90"
> raid1 swap disk1.3,disk2.3 swap sw
> raid1 - disk1.4,disk2.4 - -
>
> disk_config lvm fstabkey:uuid
> vg vg_system md2
> vg_system-root / 60G-200G ext4 rw,noatime
> vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev
> ##vg_system-scratch /scratch 30%- ext2 rw,noatime
> vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev
> ```
>
> => Even if the installation succeeded, grub does not seem to work.
>
> Anyway, I am wondering how you set this in the disk_config file (syntax ?):
>
> > I like to put a 100MB partition at the end of disks in a RAID pair to allow
> > for slightly differing size of a replacement disk. Although currently it
> > seems that any new disk we get is many many GB bigger than the old failed
> > one.
>
> Thanks again,
>
> Best regards,
> Rémy
>
>
>
> [*] there was an error for /boot partition which was not formatted the
> right way (vfat does not allow symbolic links). vfat should be used
> for /boot/EFI only.
>
> Le ven. 1 déc. 2023 à 05:43, Andrew Ruthven <andrew at etc.gen.nz> a écrit :
> >
> > Nice, good work.
> >
> > A couple of suggestions...
> >
> > I'd put swap on RAID1, if you have a disk go offline while the system is
> > running, you've lost anything that had been swapped out to the swap on the
> > failed disk.
> >
> > I like to put a 100MB partition at the end of disks in a RAID pair to allow
> > for slightly differing size of a replacement disk. Although currently it
> > seems that any new disk we get is many many GB bigger than the old failed
> > one.
> >
> > I've found that some servers allow the EFI partition to be RAID1, the
> > current servers we're using support that. So no need to copy files around.
> >
> > Cheers,
> > Andrew
> >
> > On Thu, 2023-11-30 at 17:20 +0100, Rémy Dernat wrote:
> >
> > > Ok, FYI, I think I have a working configuration with :
> > >
> > > ```
> > > # example of new config file for setup-storage
> > > #
> > > # <type> <mountpoint> <size> <fs type> <mount options> <misc
> > > options>
> > >
> > > disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1
> > > primary - 2G - -
> > > primary /boot/efi 1G vfat rw createopts="-F32"
> > > primary swap 2G swap sw,pri=1
> > > primary - 0- - -
> > >
> > > #disk_config disk2 sameas:disk1
> > > disk_config disk2 disklabel:gpt fstabkey:uuid
> > > primary - 2G - -
> > > primary /boot/efi2 1G vfat rw createopts="-F32"
> > > primary swap 2G swap sw,pri=1
> > > primary - 0- - -
> > >
> > > disk_config raid fstabkey:uuid
> > > #raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose
> > > --metadata=1.0" createopts="-L boot"
> > > raid1 /boot disk1.1,disk2.1 vfat rw,noatime mdcreateopts="--verbose
> > > --metadata=0.90"
> > > raid1 - disk1.4,disk2.4 - -
> > >
> > > disk_config lvm fstabkey:uuid
> > > vg vg_system md1
> > > vg_system-root / 60G-200G ext4 rw,noatime
> > > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev
> > > ##vg_system-scratch /scratch 30%- ext2 rw,noatime
> > > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev
> > > ```
> > >
> > > My previous message was crystal clear. md0 did not have any formatted
> > > filesystem on it. So I added `vfat` option on /boot.
> > >
> > > Best regards,
> > > Rémy
> > >
> > >
> > > Le jeu. 30 nov. 2023 à 16:53, Rémy Dernat
> > <[remy.d1 at gmail.com](mailto:remy.d1 at gmail.com)> a écrit :
> > >
> > > >
> > > > What is strange here, is that md1 is found, as well as md0, but
> > > > contrary to md0, md1 have an UUID ...
> > > >
> > > > However, UUID seems to be retrieve by blkid from /dev/disk/by-id,
> > > > where both md array have their own mdadm UUID, but only md1 have an
> > > > UUID from LVM (with pvdisplay / pvs ) ...
> > > >
> > > >
> > > > Le jeu. 30 nov. 2023 à 14:52, Rémy Dernat
> > <[remy.d1 at gmail.com](mailto:remy.d1 at gmail.com)> a écrit :
> > > >
> > > > >
> > > > > Hi Andrew,
> > > > >
> > > > > Indeed, this removed the syntax issue. However, I have a new error
> > now:
> > > > >
> > > > > ```
> > > > > /sbin/blkid -c /dev/null -s UUID -o value /dev/md0 had exit code 2
> > > > > Command had non-zero exit code
> > > > > ```
> > > > > Indeed, blkid is not able to found md0 UUID.
> > > > >
> > > > > I tried to modify the disk_config file to avoid this error, but I am
> > > > > still stuck with it. Here is what look like my disk_config file now:
> > > > >
> > > > > ```
> > > > > # example of new config file for setup-storage
> > > > > #
> > > > > # <type> <mountpoint> <size> <fs type> <mount options> <misc
> > options>
> > > > >
> > > > > disk_config disk1
> > > > > primary - 2G - -
> > > > > primary /boot/efi 1G vfat rw createopts="-F32"
> > > > > primary swap 2G swap sw,pri=1
> > > > > primary - 0- - -
> > > > >
> > > > > #disk_config disk2 sameas:disk1
> > > > > disk_config disk2 disklabel:gpt fstabkey:uuid
> > > > > primary - 2G - -
> > > > > primary /boot/efi2 1G vfat rw createopts="-F32"
> > > > > primary swap 2G swap sw,pri=1
> > > > > primary - 0- - -
> > > > >
> > > > > disk_config raid fstabkey:uuid
> > > > > raid1 /boot disk1.1,disk2.1 - - mdcreateopts="--verbose
> > > > > --metadata=0.90" createopts="-L boot"
> > > > > raid1 - disk1.4,disk2.4 - -
> > > > >
> > > > > disk_config lvm fstabkey:uuid
> > > > > vg vg_system md1
> > > > > vg_system-root / 60G-200G ext4 rw,noatime
> > > > > vg_system-tmp /tmp 20G-50G ext2 rw,noatime,nosuid,nodev
> > > > > ##vg_system-scratch /scratch 30%- ext2 rw,noatime
> > > > > vg_system-home /home 30%- ext4 rw,noatime,nosuid,nodev
> > > > > ```
> > > > >
> > > > > I also tried to change the raid metadata to 1.0, but that did not
> > > > > change anything. I wrote back ` fstabkey:uuid` behind disk_config raid
> > > > > because it does not fail with syntax error.
> > > > >
> > > > > Thanks for help
> > > > >
> > > > > Le jeu. 30 nov. 2023 à 11:55, Andrew Ruthven
> > <[andrew at etc.gen.nz](mailto:andrew at etc.gen.nz)> a écrit :
> > > > >
> > > > > >
> > > > > > Hey,
> > > > > >
> > > > > > On Thu, 2023-11-30 at 11:14 +0100, Rémy Dernat wrote:
> > > > > >
> > > > > > > Hi awesome Fai guys,
> > > > > > >
> > > > > > > I have a weird syntax issue with the following disk_config:
> > > > > > >
> > > > > >
> > > > > > [snip]
> > > > > >
> > > > > > > disk_config raid disklabel:gpt fstabkey:uuid bootable:1
> > > > > >
> > > > > >
> > > > > > This line is your problem, remove everything after the word raid. So
> > it
> > > > > > should just be:
> > > > > >
> > > > > > disk_config raid
> > > > > >
> > > > > > Cheers,
> > > > > > Andrew
> > > > > > --
> > > > > > Andrew Ruthven, Wellington, New Zealand
> > > > > > [andrew at etc.gen.nz](mailto:andrew at etc.gen.nz) |
> > > > > > Catalyst Cloud: | This space intentionally left blank
> > > > > > [https://catalystcloud.nz](https://catalystcloud.nz) |
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> > ```
> > --
> > Andrew Ruthven, Wellington, New Zealand
> > andrew at etc.gen.nz |
> > Catalyst Cloud: | This space intentionally left blank
> > https://catalystcloud.nz |
> >
> >
> > ```
> >
> >
More information about the linux-fai
mailing list