Debian GNU/Linux DiskLess Install ================================== Structure read-only root file-system over NFS overlaid by local tmpfs+aufs Structure of directories at diskless-server providing root file-system over NFS to DiskLess stations /srv - diskless - debian-squeeze - tools - SystemRescueCd - 1.x - 2.0 - homes - tftp - boot initrd.img-2.6.32-5-amd64 initrd.img-diskless-amd64 -> initrd.img-2.6.32-5-amd64 vmlinuz-2.6.32-5-amd64 vmlinuz-diskless-amd64 -> vmlinuz-2.6.32-5-amd64 - pxelinux.cfg default -> pxelinux-menu graphics.conf pxelinux-menu service.menu - sysresccd - sysresccd2 memdisk menu.c32 pxelinux.0 The PXElinux configuration in /srv/tftp/pxelinux.cfg directory can be specified for some address ranges i.e. for 172.16.149.0/24 convert three first numbers into hexadecimal AC1095 -> pxelinux-menu pxelinux-menu specification ---------------------------------------------------------- default graphics.conf pxelinux-menu service.menu root@installtest:/srv/tftp/pxelinux.cfg# cat pxelinux-menu default menu.c32 prompt 0 menu PXE menu for DiskLess boot menu INCLUDE pxelinux.cfg/graphics.conf MENU AUTOBOOT Starting Local System in # seconds MENU TITLE InstallFest DiskLess Boot Menu label bootlocal # Boot from local disk menu label ^Local boot menu default localboot 0 timeout 300 TOTALTIMEOUT 3000 ---------------------------------------------------------- label linux # Start Debian Squeeze DiskLess # add kernel option break=premount to debug initramfs menu label ^DiskLess Debian GNU/Linux KERNEL boot/vmlinuz-diskless-amd64 boot=nfs root=/dev/nfs nfsroot=147.32.30.169:/srv/diskless/debian-squeeze,ro,tcp ro APPEND initrd=boot/initrd.img-diskless-amd64 ---------------------------------------------------------- Add what should be exported to which machines into /etc/exports ---------------------------------------------------------- /srv/homes 172.16.149.0/24(rw,sync,no_root_squash,no_subtree_check) /srv/diskless/debian-squeeze 172.16.149.0/24(ro,async,no_root_squash,no_subtree_check) /srv/diskless/tools 172.16.149.0/24(ro,async,no_root_squash,no_subtree_check) ---------------------------------------------------------- debootstrap --keyring=/usr/share/keyrings/debian-archive-keyring.gpg \ --arch=amd64 --include=debian-keyring,etckeeper \ squeeze /srv/diskless/debian-squeeze/ ftp://ftp.cz.debian.org/debian/ nano /srv/diskless/debian-squeeze/usr/sbin/policy-rc.d ------------------------------------ #!/bin/sh [ "installtest.sh.cvut.cz" = "`hostname`" ] && exit 101 exit 0 ------------------------------------ chroot /srv/diskless/debian-squeeze cd /etc nano apt/sources.list --------------- #deb http://ftp.us.debian.org/debian squeeze main deb http://ftp.cz.debian.org/debian squeeze main contrib non-free deb-src http://ftp.cz.debian.org/debian squeeze main contrib non-free --------------- git add apt/sources.list git status git commit -m "my: Select Czech mirror for sources.list" nano etckeeper/etckeeper.conf ---------------------------- AVOID_DAILY_AUTOCOMMITS=1 AVOID_COMMIT_BEFORE_INSTALL=1 ------------------------------ git commit -m "my: configure etckeeper to block package manager if there are uncommited changes in etc" git log aptitude update --------------------------------------------------------- Remark about obtaining packages list from other machine aptitude search '?installed?not(?automatic)' \ | sed -n -e 's/^i[ \t]\+\([^ \t]*\)[ \t].*$/\1/p' >packages Remove grub and grub2 for now, makes problems during updates when no device map can be scanned -------------------------------------------------------- aptitude install `cat packages` -------------------------------------------------------- Most important packages linux-image-2.6 aufs-tools nfs-common ntp ntpdate openssl rcconf -------------------------------------------------------- The policy-rc.d is important it demonstrates as Setting up network-manager (0.8.1-6) ... invoke-rc.d: policy-rc.d denied execution of force-reload. Disabling interfaces configured with plain DHCP in /etc/network/interfaces so that NetworkManager can take them over Auto interfaces found: invoke-rc.d: policy-rc.d denied execution of start. If not defined the server system/kernel state/configuration would be screwed to death --------------------------------------------------------- if problem is encountered cd /etc git status git add . git commit -m "partial packages install recovered" ----------------------------------------------------- typical problem with Java packages ----------------------------------------------------- Setting up openjdk-6-jre-headless (6b18-1.8.3-2) ... the java command requires a mounted proc fs (/proc). dpkg: error processing openjdk-6-jre-headless (--configure): subprocess installed post-installation script returned error exit status 1 ----------------------------------------------------- Fixed by mount /proc aptitude install ant umount /proc The clients have problems when proc is left mounted in the image. ---------------------------------------------------- Run dpkg-reconfigure locales git status git add . git commit -m "my: dpkg-reconfigure locales" --------------------------------------------------- Frame Core Changes for Overlay There is necessary more changes, provided as patch git am /patches/my-initramfs-and-init.d-scripts-for-diskless.patch Possible conflicts resolution discussed later Most important changes Add aufs and or unionfs into initramfs-tools/modules initramfs scripts initramfs-tools/scripts/nfs-bottom/diskless_setup initramfs-tools/scripts/nfs-bottom/root_overlay Services startup init.d scripts init.d/diskless-linux init.d/findswap --------------------------------------------------- update-initramfs -u -k 2.6.32-5-amd64 update-initramfs -u -k 2.6.32-5-xen-amd64 --------------------------------------------------- Copy vmlinuz-* and initrd.img-* to /srv/diskless/debian-squeeze/boot to /srv/tftp/boot --------------------------------------------------- zcat initrd.img-2.6.32-5-amd64 >initrd-to-check.cpio /srv/diskless/debian-squeeze/boot/initrd-to-check.cpio#ucpio/scripts/nfs-bottom ORDER diskless_setup root_overlay initramfs-tools/scripts/nfs-bottom/root_overlay mkdir /tmp/unirw mount -n -t tmpfs none /tmp/unirw mount -n -t aufs -o dirs=/tmp/unirw=rw:${rootmnt}=nfsro unionfs ${rootmnt} mkdir -p ${rootmnt}/overlay/unirw mount -n -o move /tmp/unirw ${rootmnt}/overlay/unirw chmod 755 ${rootmnt} # Disable rw access for non-root users mkdir -p ${rootmnt}/tmp mkdir -p ${rootmnt}/var/lock initramfs-tools/scripts/nfs-bottom/diskless_setup rm -f ${rootmnt}/etc/hosts cat < ${rootmnt}/etc/hosts 127.0.0.1 localhost ${IPV4ADDR} ${HOSTNAME}.${DNSDOMAIN} ${HOSTNAME} ${nfsroot%%:*} diskless-server EOF Look for maybe_break XXX in /srv/diskless/debian-squeeze/boot/initrd-to-check.cpio#ucpio/init Then run rcconf to activate findswap and diskless-linux scripts git add . git commit -m "my: run rcconf to activate findswap and diskless-linux scripts" Setup root password passwd git add shadow Use rcconf to disable avahi-daemon and network-manager git add -A git status git commit -m "my: avahi and networkmanager disabled at rcconf level" nano fstab ------------------------------------------------------ none /tmp tmpfs defaults 0 0 none /var/lock tmpfs defaults 0 0 # /etc/hosts contain the correct IP address for diskless-server (set in ramdisk scripts) diskless-server:/srv/homes /home nfs defaults,tcp 0 0 ------------------------------------------------------ git add . git commit -m "my: Added /srv/homes and RAM based temporaries to fstab" nano network/interfaces ------------------------------------------------------ auto lo iface lo inet loopback ------------------------------------------------------ git add . git commit -m "my: Added local host / lo interface" This is critical even for mount over NFS to work. Check this if you encounter problems. dpkg-reconfigure tzdata git add timezone git commit -m "my: Changed timezone to Europe/Prague by dpkg-reconfigure tzdata" git add localtime git commit -m "my: Localtime after run of dpkg-reconfigure tzdata" If the computer hardware should be shared with Windows boot then use crummy setup to keep RTC date in local time instead of UTC nano default/rcS UTC=no git add default/rcS git commit -m "my: keep hwclock in local (non UTC) time for compatibility with Windows" The asynchronous NFS mounts inwokation after network manager or udev events does not work with interface configured during boot in initramfs nano default/rcS Add option ASYNCMOUNTNFS=no git add default/rcS git commit -m "my: additional fstab defined NFS mounts requires ASYNCMOUNTNFS=no for diskless" create directory /home-local for temporrary/local users git am /patches/my-added-local-guest-account-and-guestXX-accounts.patch git am --resolved git am --skip git am --abort git am /patches/0008-My-Added-automatic-creation-of-home-directories.patch git am /patches/0009-My-Group-permissions.patch git am /patches/0005-My-Added-CVUT-FELK-ceritificated-needed-for-LDAP-a.patch ------------------------------------------------------ Debugging initramfs problems The boot option break=mountroot allows test NFS mount infrastructure step by step modprobe nfs modprobe af_packet ipconfig -t 180 -c dhcp -d eth0 ping -c 4 `echo $nfsroot | sed -n -e 's/^\(.*\):\/.*$/\1/p'` mkdir /test-root-mnt nfsmount -o nolock -o ro -o tcp `echo $nfsroot | sed -n -e 's/^\([^,]*\)\(,.*\|\)$/\1/p'` /test-root-mnt ------------------------------------------------------ Links to our labs setup info for DCE FEL CVUT.cz students http://support.dce.felk.cvut.cz/mediawiki/index.php/GNU/Linux_v_laborato%C5%99%C3%ADch A4M35OSP Open-source programming - this subject is one of more courses which has been prepared and provided by our group http://rtime.felk.cvut.cz/osp/cviceni/ Playing with BusyBox and initramfs http://rtime.felk.cvut.cz/osp/cviceni/2/ ------------------------------------------------------ Similar projects in the world http://en.wikipedia.org/wiki/Diskless_Remote_Boot_in_Linux http://www.drbl.org/ http://clonezilla.org/ ------------------------------------------------------ Log author Pavel Pisa pisa@cmp.felk.cvut.cz http;//cmp.felk.cvut.cz/~pisa