reporting a documentation bug and couple of questions

[Thomas Neumann]

This mail is a bit long because a cfengine rant is included for free.

a) documentation bug

I found a documentation bug in
http://fai-project.org/fai-guide/ar01s07.html . (What is the proper way to
report this bug?)

Chapter 7.2. states that the tasks are listed in order of execution,
however task "tests" is executed after task finish.
- fai client 3.4.2
- ubuntu package from launchpad ppa
- according to /usr/lib/fai/subroutines .

I noticed this by accident, because I wanted to find out which tasks are
called for a softupdate.

b) feature request

Make the layout tabular and provide an indication which "fai type" uses
what task. Something like:

,--- install
|,-- dirinstall
||,- softupdate
----------------------------------------------
x-- partition   Calls [...]
x-- mountdisks  Mounts [...]
-x- extrbase    Extracts a [...]
xxx mirror      If a local [...]
xxx debconf     Calls fai-debconf(8) [...]
xx- prepareapt  Set up resolv.conf [...]


c) question

I know task tests isn't really used at the moment. I want to use it in my
config space. Is there any specific reason it is not included in
'softupdates' or is it just forgotten / not implemented (yet)?

I want to provide "functionality" based tests. Class FUNC_PROXY should
configure a proxy server and "tests" should try to fetch a file by using a
proxy on port xyz if this class is set. This way I can test not only if my
configuration completed without error, but if it's actually doing what I
intended it to do. (error in config file, forgot to restart daemon, daemon
fails to start because of resource conflict, etc.)

c) code refactor suggestion

task dirinstall has some hardcoded logic which prepares the install
directory. Wouldn't it be more appropriate to move this into a dirinstall
specific 'mountdisks' task? (Or whatever is a proper name. I think this
hardcoded logic is rather ugly. Especially in conjunction with a possible
b), because that mounting does not show up as a task and may lead to
surprises.)



All in all I'd like to say, that I'm an even bigger fan of fai after my
third attempt to write a decent cfengine3 config. It is (was) so
unbelievable hard just to install packages[1] without user-intervention or
update some /proc/sys settings on the fly[2] it's just not funny anymore.
cfengine may have it's uses, but fai is definitely more to my liking.

The promise theory sounds really nice but with the amount of time invested
in cf3 I could have easily implemented promise theory in a custom perl
library. Maybe I'm just stupid and don't grok cf3 properly. Maybe it's
just too complicated.

I know what I achieved with fai already and I happily come back to it. At
least I can bend it to my will as I see fit. (I LOVE THE HOOKS! THANK YOU
FOR IMPLEMENTING THEM!)




<rant mode, please ignore if not interested in cfengine>

[1] Sure, there is a generic install bundle in the Open Body Promise
Library. But it's not much more then a glorified "apt-get/aptitude
install"  which definitely does not run unattended out of the box. (Try to
install postfix/exim.)
 - need to manually set DEBIAN_FRONTEND to disable whiplash
 - need to manually set --force-confdef
 - need to manually set --force-confold
Forgot any of these? Thought so.

I have to correct myself. The current (3.1.) COPBL now configures all the
required settings. COPBL 3.0 was a simple "--assume-yes install".

[2] I tried to set /proc/sys/kernel/hostname to a new value. This updates
the hostname "on-the-fly". Sometimes it's useful to change the hostname
without a reboot.

vars:
  "hostnamefile" => string "/etc/hostname";
commands:
  "cat $(hostnamefile) > /proc/sys/kernel/hostname";

error message:

 -> Executing '/bin/cat /etc/hostname > /proc/sys/kernel/hostname'
...(timeout=-678,owner=-1,group=-1)
 !! Finished command related to promiser "/bin/cat /etc/hostname >
/proc/sys/kernel/hostname" -- an error occurred (returned 1)
Q: ".../bin/cat /etc/h": unconfigured
Q: ".../bin/cat /etc/h": /bin/cat: >: No such file or directory
Q: ".../bin/cat /etc/h": unconfigured

I think cfengine complains because it tries to be too smart and interpret
the output redirection as a filename. I tried to tackle this differently
by editing /p/s/k/hostname as a file, but then cfengine complains it can't
create a temporary file in /proc/... - which is not suprising at all.

I'd rather not be forced to start kludging around cf3's cleverness.
(Create a scratch file containing "kernel.hostname=value", issue "sysctl
-p <file>", remove scratch file.)

failover / redundancy

I haven't been able to figure out how to specify 2 different policy hosts
as a global (site-specific) variable and let cfengine decide which one to
pick. Either I'm too stupid to write the correct config or cfengine
expects exactly one policy host.

file editing and promise theory

I think it's badly broken. What is stuff like this doing in a tool that
praises itself by touting the promise theory? (Specify the outcome and the
tool will apply the necessary steps to reach that goal.)

bundle edit_line append_to_line_end(start,end)
#
# Lines starting with "$(start)" and not ending with "$(end)"
# will get appended with "$(end)", whitespaces will be left unmodified.
# For example, append_to_line_end("kernel", "vga=791") would replace
# "kernel /boot/vmlinuz root=/dev/sda7"
# with
# "kernel /boot/vmlinuz root=/dev/sda7 resume=/dev/sda9 vga=791"
#
# WARNING: Be careful not to have multiple promises matching the same line,
#          which would result in the line growing indefinetively.

This is a perversion. It's a promise that something is at the end of the
line. (I promise the line ends in foo.) What would be more useful (and
more in the spirit of the promise theory) is a bundle that would check if
some value is currently set and add it to the config string if it's not
found. But there is no such bundle in the default library. You have to
write this yourself.

Do you think this is an esoteric feature?  Then think about configuring
secondary groups in /etc/group, editing debian/ubuntu's
/boot/grub/menu.lst (defoptions=[...]), allowed programs in /etc/sudoers,
...

variables

If a variable is undefined (typo?), then cfengine will happily insert the
variable's definition instead of its name. Fun! Be prepared to find
$(g.forgot-to-set-this) in your config files!

templates

It's been a bit, since I used them, so I could be wrong. Citing from
memory (refreshed by the documentation) it seems like it's a three way
process.

 1. fetch template and save as file
 2. edit file, replace template variables
 3. copy _modified_ template to final location

One could skip the third step, but that would result in a config file
change with every iteration of cfengine. If there's a
process-restart-trigger on file change then this will trigger every time.
IMHO this process could be improved by

 1. fetch template and save as file
 2. load file into memory, replace template variables, compare edited
template and target file, replace target file if changed

Sounds complicated? Well, edit_lines already works this way. Load old
file, edit in memory, compare if changes occured, save if changed.

output

By default cf3 outputs nothing and will happily continue if errors are
encountered. One could make cf3 --inform about changes or create --verbose
output. A simple 'I'm currently working on foo' (which is probably
sufficient in most cases) has to be handcoded via reports. [So... if you
forgot to update the COBPL to 3.1 or using an unmodified 3.0 version, then
cfengine/apt may decide to wait (until timeout) for user input without any
noticeable prompt. That's really nice, isn't it?]

dependencies

Exist. On paper.

bundle a declares to provide handle_a
bundle b declares to depend on handle_a

However this has no effect except some kind of documentation in cfengine
nova (enterprise verson). Why does this have no real effect even in
enterprise? This could be really useful. If the promise belonging to
handle_a can not be kept (a.k.a: can not be fixed) then please do not
attempt to repair promises belonging to bundle b since the result may be
undefined or even desastrous. (Do not try to add/remove packages if the
promise controlling the repository configuration files failed.)



May main gripe with cf3 is not that it's hard to learn or conceptually
very different from day-to-day administration. My main gripe is that it's
so frustratingly hard to automate simple day-to-day tasks in a safe manner
- especially because you have to describe common tasks in a totally
different syntax.

The only difference between

sed --in-place=".cfsaved" "s/^do_symlinks = .*$/do_symlinks = no/"
/etc/kernel-img.conf

and a cf-promise is (in my eyes) that sed will try to fix that 'promise'
every time. Which can be easily changed by wrapping that statement in a
grep. Sure - automatic expansion of a list of values into a loop IS nice.
But I don't think the attached cost is justified.