/tmp read-only? FAI 5.5, Stretch, NFSv3

[Steffen Grunewald]

Hello,

after running several sysinfo FAI_ACTIONs with jessie setups (and "aufs"
in the append line), I decided the time has come to switch to Stretch.

I upgraded fai-* to 5.5 from the uni-koeln Stretch repository, copied
/etc/fai to /etc/fai-stretch, added a few packages to NFSROOT, and then
built the NFS-root following the docs (BTW, page 29 has a small "c"
instead of capital "C" in the fai-make-nfsroot example line).

A few "error" lines showed up in the log:
root at t-pring:/etc/fai-stretch# grep -iC3 error: nfsroot.log 
Setting up dracut (045+132-1) ...
dracut: Generating /boot/initrd.img-4.9.0-5-amd64
/usr/lib/dracut/modules.d/45url-lib/module-setup.sh: line 33: warning: command substitution: ignored null byte in input
dracut-install: ERROR: installing '/etc/ssl/certs/ca-certificates.crt'
dracut: FAILED: /usr/lib/dracut/dracut-install -D /var/tmp/dracut.Mk2W0u/initramfs /etc/ssl/certs/ca-certificates.crt
/usr/lib/dracut/modules.d/45url-lib/module-setup.sh: line 33: warning: command substitution: ignored null byte in input
dracut-install: ERROR: installing '/etc/ssl/certs/ca-certificates.crt'
dracut: FAILED: /usr/lib/dracut/dracut-install -D /var/tmp/dracut.Mk2W0u/initramfs /etc/ssl/certs/ca-certificates.crt
/usr/lib/dracut/modules.d/45url-lib/module-setup.sh: line 33: warning: command substitution: ignored null byte in input
dracut-install: ERROR: installing '/etc/ssl/certs/ca-certificates.crt'
dracut: FAILED: /usr/lib/dracut/dracut-install -D /var/tmp/dracut.Mk2W0u/initramfs /etc/ssl/certs/ca-certificates.crt
Setting up openssl (1.1.0f-3+deb9u1) ...
Setting up threeware-control (10.2-1) ...
--
Setting up grub-pc (2.02~beta3-5) ...

Creating config file /etc/default/grub with new version
grub-probe: error: cannot find a device for / (is /dev mounted?).
grub-probe: error: cannot find a device for /boot (is /dev mounted?).
grub-probe: error: cannot find a device for /boot/grub (is /dev mounted?).
Setting up libisccfg140:amd64 (1:9.10.3.dfsg.P4-12.3+deb9u4) ...
Setting up emacs25-nox (25.1+1-4+deb9u1) ...
update-alternatives: using /usr/bin/emacs25-nox to provide /usr/bin/emacs (emacs) in auto mode

- they seem to be sufficiently benign though.

With a pxelinux.cfg file:

root at t-pring:/srv/fai/tftp/pxelinux.cfg# cat 0A966401 
# generated by fai-chboot for host mds-eth0 with IP 10.150.100.1
default fai-generated

label fai-generated
kernel vulcan-stretch/vmlinuz
append initrd=vulcan-stretch/initrd.img ip=dhcp  root=10.150.100.198:/srv/fai/nfsroots/vulcan-stretch:vers=3   FAI_FLAGS=verbose,sshd,createvt FAI_CONFIG_SRC=nfs://10.150.100.198/srv/fai/config/vulcan-stretch FAI_ACTION=sysinfo 

... I get the infamous /tmp read-only error *although* all mounts are NFSv3:

root at t-pring:~# ssh -oUserKnownHostsFile=/dev/null root at mds-eth0
The authenticity of host 'mds-eth0 (10.150.100.1)' can't be established.
ECDSA key fingerprint is SHA256:dRQY3FUCjC5bCTiiXYEJxNdDVE9v2/ihKy3zc4JSkpk.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'mds-eth0,10.150.100.1' (ECDSA) to the list of known hosts.
root at mds-eth0's password:
Linux mds-eth0 4.9.0-5-amd64 #1 SMP Debian 4.9.65-3+deb9u2 (2018-01-04) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
root at mds-eth0:~# df
Filesystem                                      1K-blocks      Used Available Use% Mounted on
devtmpfs                                         65946268         0  65946268   0% /dev
tmpfs                                            65963204      1612  65961592   1% /run
10.150.100.198:/srv/fai/nfsroots/vulcan-stretch 622391552 497678848 124712704  80% /
10.150.100.198:/srv/fai/config/vulcan-stretch   622391552 497678848 124712704  80% /var/lib/fai/config
root at mds-eth0:~# mount
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,noexec,size=65946268k,nr_inodes=16486567,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,mode=755)
10.150.100.198:/srv/fai/nfsroots/vulcan-stretch on / type nfs (ro,relatime,vers=3,rsize=262144,wsize=262144,namlen=255,hard,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.150.100.198,mountvers=3,mountport=47820,mountproto=udp,local_lock=all,addr=10.150.100.198)
10.150.100.198:/srv/fai/config/vulcan-stretch on /var/lib/fai/config type nfs (ro,noatime,vers=3,rsize=262144,wsize=262144,namlen=255,acregmin=1800,acregmax=1800,acdirmin=1800,acdirmax=1800,hard,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.150.100.198,mountvers=3,mountport=47820,mountproto=udp,local_lock=all,addr=10.150.100.198)

(Of course I followed the suggestions, to "catch" NFSv4 with a fake export.)
Is this a new issue with recent kernels, did I miss a necessary step somewhere?
Is there anything I can do to debug this?

Scratching my head... S