FAI install with huge JBODs present - disk/by-id?

[Steffen Grunewald]

Hi Thomas,

On Tue, 2024-12-17 at 10:43:57 +0100, Thomas Lange wrote:
> I wonder if you have a grub or a FAI problem?

Until yesterday I had none of both...
FAI worked perfectly installing my plan, but with the 120 disks now
partitioned (something I didn't hand over to FAI, due to a lack of
planning for that part) it's grub that fails.

So yes, I have a grub problem that needs to be worked around, by
modifiying plans.

> If it's a grub problem, I suggest to try Super Grub2 Disk which is a
> very sophisticated grub menu detecting all sorts of configurations. If
> this also fails, grub itself can't handle your environment (maybe
> because of bug in an edge case).

Not so sure about "edge case" (as 120 disks should be more or less
normal in a datacenter setup_.

> Or is it only hard to get the right device name inside FAI for your
> disk_config?

Let me ask the question the other way round.
Most of the examples only use disk names from /dev (as in sda etc.).
Can I use the full-fledged path, e.g.
  /dev/disk/by-path/pci-0000:06:00.0-ata-..0
in all places? This seems to work for the disk_config line itself,
but for mdraid definitions?!

> In FAI 6.2 we have some new helper subroutines which are alsi used to
> setting the variable $disklist which is the used by
> setup-storage.

I had tried to rebuild my old and trusted nfsroot and failed multiple
times. The new one would not start the main fai routine and instead
open a vt before almost immediately rebooting - a behaviour I currently
cannot afford as time is running out and I had intended to use the
holiday season to copy 200 TB of data to the new storage.

Therefore I'm with 6.0.5 - which did its job for a year now.

>     disks_by_id()

looks the most promising from your feature list...

> Maybe those functions helps you to select those disk that you like to
> be listed in $disklist.

Hm, perhaps I could tweak $disklist and pass a "short list" of two disks
to the installer, then use "disk1" and "disk2"? That's be in a partition
hook, right? (Write to variables.log?)

> BTW, which FAI version are you using? Is there a grub config file involved
> that comes from FAI (only if you boot FAI from ISO or USB)?

Currently it looks like GRUB doesn't find the grub.cfg at all, having to
parse 200+ existing partitions with "insmod zfs" enabled.
An older, somewhat similar setup with 60 disks boots without zfs enabled,
so the disk count is not the main problem, it's apparently the combination
of zfs and many disks that eats memory.

Thanks,
 Steffen

-- 
Steffen Grunewald, Cluster Administrator
Max Planck Institute for Gravitational Physics (Albert Einstein Institute)
Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany
~~~
Fon: +49-331-567 7274
Mail: steffen.grunewald(at)aei.mpg.de
~~~