pool generation

[Niall Young]

On Tue, 26 Nov 2002, Bruce  Edge wrote:

> Here's my pool generation snippet, it's from another script which isn't finished yet, but the pool generation works OK. It uses apt-get to copy over all the packages it needs into a single dir, then generates Packages.gz.
> It uses the package list from the prev thing I sent you.
> It's a modified version of Mathew Palmer's. It generates the necessary config files on the fly, so they don't need to exist.

> > playing with apt-move but I'm going to write my own repository builder
> > as woody's apt-move doesn't yet support pools.  It'll do
> > until the next
> > release, ideally we should use apt-move but I don't like the thought

Here's mine.  It'd be great if your "-l" patch to install_packages could
also generate all dependencies of those packages - make sure we have
*everything* we need in the pool.  You could parse
`dpkg-awk Package:^$package$ -- Depends` and append them to $packlist,
keep generating dependencies until you reach the end.  Otherwise this
build_repository script is pretty useless, we'll be missing a heap of
dependencies and installs will break.  I could have done it here but
your patch seems a better place.

Thomas, an `apt-get upgrade` fails in NFSROOT as it's looking for
Packages and not Packages.gz - bit strange that it doesn't do both but I
haven't had time to look into it further.

Niall Young                                    Chime Communications Pty Ltd
niall at chime.net.au                            Level 6, 263 Adelaide Terrace
Ph: 08 9213 1330 / 0408 192 797               Perth, Western Australia 6000

     "there's a lot of movement in my trousers at the moment"
                                     -- Dennis Kristofich, Sep 2002

#!/bin/sh

# build_repository
#
# Niall Young <niall at chime.net.au> Thu Nov 28 16:31:49 WST 2002
#   for FAI  (Fully Automatic Installation)
#   (c) 2000-2002 by Thomas Lange, lange at informatik.uni-koeln.de
#   Universitaet zu Koeln
#
# This script uses the "-l" patch to install_packages from
# Bruce Edge <bedge at troikanetworks.com> to retrieve a list of
# required packages for all FAI classes defined; then downloads
# the latest version of each package using apt-get; moves them
# into a pool/ compliant filesystem and generates index files
#
# generate_pool() populate_pool() and generate_indices() should be
# replaced with calls to apt-move >= 4.2.8 when this reaches stable

. /etc/fai/fai.conf

VERSION=0.1
FAI_LIBDIR=/usr/lib/fai # this should really be defined in fai.conf
APT_ARCHIVE=/var/cache/apt/archives
PACKAGE_ROOT=fai/packages
ARCH=`/usr/bin/dpkg --print-architecture`
SECTION=fai

CLASSES="`/bin/ls $FAI_CONFIGDIR/package_config | tr '\n' ' '`"
PACKAGES=`$FAI_LIBDIR/sbin/install_packages -l 2>/dev/null`

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
download_packages() {

    echo " - - - - - Downloading all required packages"
    /usr/bin/apt-get clean
    /usr/bin/apt-get --download-only --reinstall -yu install $PACKAGES
    echo

}

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
generate_pool() {

    echo " - - - - - Generating apt compliant package pool in $NFSROOT/$PACKAGE_ROOT/pool/$SECTION"
    /bin/mkdir -vp $NFSROOT/$PACKAGE_ROOT/pool/$SECTION
    for prefix in 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z; do
	/bin/mkdir $NFSROOT/$PACKAGE_ROOT/pool/$SECTION/$prefix
	/bin/mkdir $NFSROOT/$PACKAGE_ROOT/pool/$SECTION/lib$prefix
    done
    /bin/mkdir -vp $NFSROOT/$PACKAGE_ROOT/dists/$debdist/$SECTION/binary-$ARCH/
    echo

}

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
populate_pool() {

    echo " - - - - - Populating the pool with downloaded packages"
    files=$APT_ARCHIVE/*_$ARCH.deb
    for file in $files; do
	prefix=`echo $file | cut -c 1,2,3`
	if [ $prefix == "lib" ]; then
	    prefix=`echo $file | cut -c 1,2,3,4`
	else
	    prefix=`echo $file | cut -c 1`
	fi
	if [ -e $file ]; then
	    /bin/mv -v $file $NFSROOT/$PACKAGE_ROOT/pool/$SECTION/$prefix/
	fi
    done
    echo

}
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
generate_indices() {

    /usr/bin/apt-ftparchive packages $NFSROOT/$PACKAGE_ROOT/pool > $NFSROOT/$PACKAGE_ROOT/dists/$debdist/$SECTION/binary-$ARCH/Packages
    /bin/echo "Archive: $SECTION" > $NFSROOT/$PACKAGE_ROOT/dists/$debdist/$SECTION/binary-$ARCH/Release
    /bin/echo "Version: $VERSION" >> $NFSROOT/$PACKAGE_ROOT/dists/$debdist/$SECTION/binary-$ARCH/Release
    /bin/echo "Component: $SECTION" >> $NFSROOT/$PACKAGE_ROOT/dists/$debdist/$SECTION/binary-$ARCH/Release
    /bin/echo "Origin: `hostname`:$0" >> $NFSROOT/$PACKAGE_ROOT/dists/$debdist/$SECTION/binary-$ARCH/Release
    /bin/echo "Label: FAI" >> $NFSROOT/$PACKAGE_ROOT/dists/$debdist/$SECTION/binary-$ARCH/Release
    /bin/echo "Architecture: $ARCH" >> $NFSROOT/$PACKAGE_ROOT/dists/$debdist/$SECTION/binary-$ARCH/Release

}

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# mainline

echo
echo build_repository v$VERSION
echo
echo "	FAI CLASSES = $CLASSES"
echo "	PACKAGES    = $PACKAGES"
echo

if [ ! $PACKAGES == "" ]; then

    if [ ! -d $NFSROOT/$PACKAGE_ROOT ]; then
	generate_pool
    fi

    download_packages
    populate_pool
    generate_indices

    echo " - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -"
    echo Done.  You may now use $NFSROOT/packages as an apt-get source with:
    echo
    echo "/bin/echo \"deb file: /$PACKAGE_ROOT $debdist $SECTION\" > $NFSROOT/etc/apt/sources.list"
    exit 0;
else
    echo There are no packages to install?  Are you sure your classes are defined properly?
    exit 1;
fi