Install ZFS root with FAI?

[Rémy Dernat]

Hi,

I am not doing this, but I think it should not be to difficult. I have a
similar script running as a post-install step; maybe you can use something
like this as a script :

#!/bin/bash

check_zpool()
{
    zfs_value=`/sbin/zpool list|grep -ic "zfs_bigvol"`
}

check_zpool
if [ $zfs_value -eq 0 ];then
    ## Create a zfs zpool raid0 for every disks locally except /dev/sda
    /sbin/modprobe zfs
    ls -l /dev/disk/by-id/|egrep -v "sr|sda|part" |awk '/scsi/ {print $9}'
> /tmp/disks-by-id.txt
    while read disk; do
        /sbin/parted -s /dev/disk/by-id/$disk mklabel GPT;
    done < /tmp/disks-by-id.txt
    all_disks_inline=`tr '\n' ' ' < /tmp/disks-by-id.txt`
    # sometimes it fails to create the zpool
    for i in {1..15}; do
        check_zpool
        if [ $zfs_value -eq 0 ];then
            /sbin/zpool create -o ashift=12 -m /media/bigvol zfs_bigvol
$all_disks_inline
            sleep 5
            /sbin/zfs set compression=lz4 zfs_bigvol
            sleep 1
            /sbin/zfs mount -a
        else
            break
        fi
    done
    rm -f /tmp/disks-by-id.txt
fi


This scripts creates a RAID0 volume using parted and disks id. Note that if
you have many many disks, RAID0 is obvisouly not recommanded (perhaps you
would switch to radiz3) and you may want to use ZFS with disks by path.

Another example using apt (this will create 3 zpool with 20 disks in each
of zpool):

apt-get install zfsutils-linux parted

ls -l /dev/disk/by-id/|egrep -v "sr|sda|part" |awk '/scsi/ {print $9}'
|head -20 > /root/pool1.txt
ls -l /dev/disk/by-id/|egrep -v "sr|sda|part" |awk '/scsi/ {print $9}' |
awk 'NR>20&&NR<=40' > pool2.txt
ls -l /dev/disk/by-id/|egrep -v "sr|sda|part" |awk '/scsi/ {print $9}' |
awk 'NR>40&&NR<=60' > pool3.txt


mkdir -p /export/pool{1,2,3}
pool1_disks=`tr '\n' ' ' < pool1.txt`
pool2_disks=`tr '\n' ' ' < pool2.txt`
pool3_disks=`tr '\n' ' ' < pool3.txt`
apt-get install parted
while read disk; do /sbin/parted -s /dev/disk/by-id/$disk mklabel GPT; done
< pool1.txt
while read disk; do /sbin/parted -s /dev/disk/by-id/$disk mklabel GPT; done
< pool2.txt
while read disk; do /sbin/parted -s /dev/disk/by-id/$disk mklabel GPT; done
< pool3.txt

zpool create pool1 -o ashift=12 -m /export/pool1 raidz3 $pool1_disks
zpool create pool2 -o ashift=12 -m /export/pool2 raidz3 $pool2_disks
zpool create pool3 -o ashift=12 -m /export/pool3 raidz3 $pool3_disks


4 kernel modules are necessary (generally, you will only see spl and zfs) :
zfs.ko, spl.ko, znvpair.ko, zcommon.ko

If you need to build ZFS from source, you will need git (or download the
tar.gz version with wget) :

apt-get install zlib1g-dev uuid-dev libattr1-dev libblkid-dev
libselinux-dev libudev-dev libssl-dev parted lsscsi wget ksh gdebi
apt-get install build-essential autoconf libtool gawk alien fakeroot
linux-headers-$(uname -r)

mkdir -p /opt/zfs-source
cd $_
git clone https://github.com/zfsonlinux/spl
git clone https://github.com/zfsonlinux/zfs

cd spl
git checkout master
sh autogen.sh
./configure
make -s -j$(nproc)
make install

cd ../zfs
git checkout master
sh autogen.sh
./configure
make -s -j$(nproc)
make install

echo "/lib64" > /etc/ld.so.conf.d/zfs.conf
echo "/usr/local/lib" >> /etc/ld.so.conf.d/zfs.conf
ldconfig
updatedb
locate spl.ko |xargs -I{} ls -l {}
# overwritting old kernel modules with more recent ones
cp /opt/zfs-source/spl/module/spl/spl.ko /lib/modules/$(uname -r)
/extra/spl/spl/spl.ko
cp /opt/zfs-source/spl/module/spl/spl.ko /lib/modules/$(uname -r)
/kernel/zfs/spl/spl.ko
locate zfs.ko |xargs -I{} ls -l {}
cp /opt/zfs-source/zfs/module/zfs/zfs.ko /lib/modules/$(uname -r)
/kernel/zfs/zfs/zfs.ko
locate znvpair.ko |xargs -I{} ls -l {}
cp /opt/zfs-source/zfs/module/nvpair/znvpair.ko /lib/modules/$(uname -r)
/kernel/zfs/nvpair/znvpair.ko
locate zcommon.ko |xargs -I{} ls -l {}
cp /opt/zfs-source/zfs/module/zcommon/zcommon.ko /lib/modules/$(uname -r)
/kernel/zfs/zcommon/zcommon.ko

modprobe spl
modprobe znvpair
modprobe zcommon
modprobe zfs


So you can imagine in the `package_config` a class with all the necessary
packages and in `scripts` the same class which is running other commands. I
don't think there is anything specific in FAI to use ZFS with the FAI
`disk_config`.


Kind regards,
Rémy

Le ven. 5 oct. 2018 à 09:06, Steffen Grunewald <steffen.grunewald at aei.mpg.de>
a écrit :

> Good morning,
>
> once again I have browsed multiple years of the list archive, without
> finding
> anything useful, but it's Firday and I may have gone blind over the past
> week.
>
> What I'm looking for is FAI installation details to set up a root
> filesystem
> on ZFS (two mirrored disks, possibly one spare) on a number of new
> machines.
> (I know that FAI isn't mandatory for this, but never underestimate the
> documentation effect of having the proper classes.)
>
> - Which additions to the nfsroot are required / recommended?
> - Which packages must be added to the machine package configuration?
> - Which hooks should I use to create the root filesystem?
>
> There must be recipes already existing - I'm perhaps just unable to find
> them
> (I have found the zfsonlinux wiki pages, but they don't know about FAI; and
> the only place where ZFS is mentioned on fai-project.org is the roadmap.)
>
> Any pointer is appreciated.
>
> Thanks,
>  Steffen
>
> --
> Steffen Grunewald, Cluster Administrator
> Max Planck Institute for Gravitational Physics (Albert Einstein Institute)
> Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany
> ~~~
> Fon: +49-331-567 7274
> Mail: steffen.grunewald(at)aei.mpg.de
> ~~~
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.uni-koeln.de/pipermail/linux-fai/attachments/20181005/f78cd62c/attachment.html>