The Labs.Com OS Lab FreeBSD FreeBSD Diskless

The Labs - Design & Functionality For The Net

FreeBSD Diskless Clients

A small HOW TO for "FreeBSD Diskless Clients with Etherboot". This document is work-in-progress, best check back once in a while for updates and better solutions. Feedback and hints appreciated.

1. Preparation
2. Pre Read
3. Etherboot ROM
4. Server Disk Layout
5. DHCP Server
6. DISKLESS Kernel
7. NFS Server
8. Local Disk as Swap & Scratch Disk
9. Screenshot
10. XFree86/X11
11. Etherboot & PXE Hybrid
12. Todo's
13. Links

I first started with a vacant old Pentium-100, but I quickly realized it took me too much time (even with a KVM or sitting next by to the machine) to go into BIOS to change boot sequence etc - and I used VMware direct, configured a diskless virtual machine with 32MB RAM. And made a CD-ISO image and used it as CD within VMware (easy to attach and detach). Without VMware I would have spent/wasted a lot of time, that way the setup I had working within 4-5 hours.

Finally I tested the entire setup also on the real machine again.

FreeBSD Diskless 2. Pre Read

First read: Handbook: Diskless and man diskless. It saves you a lot of time if you read it because most things are answered.

FreeBSD Diskless 3. Etherboot ROM

Via Port

Install etherboot (there is no 'make install'):  cd /usr/ports/net/etherboot/; make  You may edit work/*/src/Config:  # If you prefer the old style rotating bar progress display   CFLAGS32+=  -DBAR_PROGRESS  and redo make; after that you have the ROMs in  cd work/*/src  create the .dsk image (check which network card you have and replace NICID with the corresponding id)  cat bin/boot1a.bin bin32/{NICID}.rom > /root/floppy.dsk

Via ROM- O- MATIC

You can also visit ROM-O-MATIC.net and download the prepared .zdsk file, do a "Configure" before requesting the image: select [X] BAR_PROGRESS select [X] IMAGE_MULTIBOOT select [X] IMAGE_FREEBSD deselect [ ] DOWNLOAD_PROTO_TFTP select [X] DOWNLOAD_PROTO_NFS select [X] DEFAULT_PROTO_NFS

Write image to floppy disk (for testing)

cat /root/floppy.dsk > /dev/fd0

or

dd if=/root/floppy.dsk of=/dev/fd0

To create bootable CD ISO:

dd if=floppy.dsk of=floppy.dsk.img conv=sync bs=1440k

mkdir data

cp floppy.dsk.img data/

mkisofs -b floppy.dsk.img -c boot.cat -o cd.iso data/

The ROM-O-MATIC allows multiple formats (.zdsk, .zlilo etc) which is quite handy, and also provides more features and drivers than the etherboot port.

To boot etherboot via disk see below "Using Local Disk"

FreeBSD Diskless 4. Server Disk Layout

Edit /usr/share/examples/diskless/clone_root to change /diskless_root to /usr/diskless_root or best to a dedicated partition, as usually the / (root) partition in a default FreeBSD isn't big enough to handle all the coming extensions in diskless_root (e.g. dedicated /etc/ for particular clients residing there)

su

mkdir /usr/diskless_root

cd /usr/share/examples/diskless/; sh clone_root all

which does all the important work, and works with /etc/rc.diskless[12] together.

As next begin with the fine-tuning, rename the etc.cpio.gz so it's not used but etc/ below

cd /usr/diskless_root/conf/

mv etc.cpio.gz etc.cpio.gz.old

edit defaults for all diskless clients (rather don't change the /usr/diskless_root/conf/base/* but make all changes in /usr/diskless_root/conf/default/* files instead)

cd /usr/diskless_root/conf/

mkdir -p default/etc/

cd default/etc

Edit fstab

proc           /proc   procfs  rw 0 0

192.168.0.1:/usr       /usr     nfs ro 0 0

192.168.0.1:/usr/home  /usr/home nfs   rw 0 0

at least, and edit/create rc.conf with

local_startup=""

to avoid /usr/local/etc/rc.d/ (of /usr) not be executed on clients, (server likely runs dhcpd and is started there, you don't want each client run amother dhcpd)

FreeBSD Diskless 5. DHCP Server

Install /usr/ports/net/isc-dhcp* and edit /usr/local/etc/isc-dhcpd.conf like this:

subnet 192.168.0.0 netmask 255.255.255.0 {

range 192.168.0.160 192.168.0.200;

option routers 192.168.0.1;

option domain-name-servers 192.168.0.1;

option domain-name "local";

option broadcast-address 192.168.0.255;

# --- for diskless machines:

use-host-decl-names on;

option root-path "192.168.0.1:/usr/diskless_root/";

filename "/kernel.diskless";

next-server 192.168.0.1;

}

192.168.0.1 is the IP of the server (change it to fit your environment)

When you make changes in dhcpd.conf,

/usr/local/etc/rc.d/isc-dhcpd.sh restart

reloads the config file (it actually stops and restarts the daemon)

FreeBSD Diskless 6. DISKLESS Kernel

cd /sys/i386/conf/

cp GENERIC DISKLESS

and add in DISKLESS those lines:

options MFS

options BOOTP

options BOOTP_NFSROOT

options BOOTP_COMPAT

eventually

options         BOOTP_WIRED_TO=fxp0

to force boot happening through a particular ethernet card

then

config DISKLESS

cd ../../compile/DISKLESS

make depend && make

cp kernel /kernel.diskless

FreeBSD Diskless 7. NFS Server

Make sure your nfs-server exports all required dirs, /etc/exports:

# -- to load /kernel.diskless

/ -maproot=root:0 -ro -alldirs -network 192.168.0 -mask 255.255.255.0

# -- diskless /usr readonly (/usr/home/ r/w mounted)

/usr -maproot=root:0 -alldirs -network 192.168.0 -mask 255.255.255.0

FreeBSD Diskless 8. Local Disk as Swap & Scratch Disk

Sure it's not "diskless" anymore, but useful to use the local disk for swapping and scratch-space (for temporary non backup'ed stuff).

To make to use local swapspace, and rest of the disk as scratch-space, do this:

mkdir -p /usr/diskless_root/mnt/scratch

and on the client (once it's running):

/stand/sysinstall

In the Menu: Configure -> FDisk and (Disk)Label

Make two slices (in one partition):

1. swap space (2x physical RAM size of client)
2. scratch disk (mount /mnt/scratch)

then a dedicated /etc/fstab for that client needs to be created, this is archieved by

1. in /usr/local/etc/dhcpd.conf make a dedicated profile:

   host sherlock {

   hardware ethernet 00:00:00:00:00; # edit your MAC address

   fixed-adress {fixedIP};           # edit your static IP

   }

2. in /usr/diskless_root/conf/

   mkdir -p {fixedIP}/etc/

   cd {fixedIP}/etc

   and create a fstab for this client, like this:

   /dev/ad0s1b    none           swap   sw   0  0

   /dev/ad0s1e    /mnt/scratch   ufs   rw 1 1

   proc        /proc    procfs   rw    0  0

   192.168.0.1:/usr    /usr     nfs ro 0 0

   192.168.0.1:/usr/home    /usr/home     nfs rw 0 0

Booting Etherboot on the Disk

You can put the floppy.dsk direct on the disk (and overwrite the partition table, in order words, no slices, no partitions, just 16K or 32K boot-rom on your huge disk - not really what we want). On the client (the local disk being /dev/ad0):  dd if=floppy.dsk of=/dev/ad0  I have not been able to use multiple slices (e.g. two): /dev/ad0s1 /dev/ad0s2a (scratch-disk) /dev/ad0s2b (swap-space) and installed the FreeBSD bootloader, then I put floppy.dsk to /dev/ad0s1, but etherboot fails with:  F1     FreeBSD      Default: F1      .0000.003FBoot err  If you have any solution to boot Etherboot on the HD with a partition table (multiple slices) please let me know (About Us).

FreeBSD Diskless 9. Screenshot

Below a screenshot of a working setup:

Booting VMWare-4.0 machine (diskless) with Etherboot to boot FreeBSD-4.9-RELEASE

Here the complete dmesg of it.

FreeBSD Diskless 10. XFree86/X11

The tricky thing is you have a common /etc/X11/XF86Config for all your clients, if you have all the same brands of videocards and monitors, then it's easy - but in real life it is not the case. Best define each MAC to an IP (via DHCP configuration file), and then define a dedicated XF86Config file for each client, e.g. diskless_root/conf/IP/etc/X11/XF86Config.

We will provide an auto-configuration for XFree86 soon so each client has its dedicated XF86Config configured at run-time (videocard & monitor probing ala KNOPPIX).

If you need different keymaps under X11, then following steps are required as XFree86 doesn't conform UNIX disk-layout fully yet (it writes into /usr/X11R6/ at run-time and not into /var/X11R6/ or so):

cd /usr/X11R6/lib/X11/xkb

mv compiled compiled.dist && ln -s /var/tmp/xkb compiled

mkdir -p /var/tmp/xkb

Edit your /usr/diskless_root/conf/default/etc/rc.local and make sure /var/tmp/xkb is created.

FreeBSD Diskless 11. Etherboot & PXE Hybrid

Some mainboard support PXE on their LOM (LAN on motherboard), also some Intel NIC do support PXE natively (without additional ROM), such as Intel Pro 1000/MT (Gigabit).

Enable tftp, enable in /etc/inetd.conf following line:

tftp    dgram   udp     wait    root    /usr/libexec/tftpd      tftpd -s /tftpboot

restart inetd, and then

• mkdir /tftpboot/ and
• cp /boot/pxeboot /tftpboot/ (or /usr/obj/usr/src/sys/boot/i386/pxeldr/pxeboot), and also
• cp /kernel.DISKLESS /tftpboot/

As next add following lines in your dhcpd.conf:

class "pxe-diskless" {

match if substring (option vendor-class-identifier, 0, 3) = "PXE";

filename "/pxeboot";

}

class "etherboot-diskless" {

match if substring (option vendor-class-identifier, 0, 9) = "Etherboot";

option vendor-encapsulated-options 3c:09:45:74:68:65:72:62:6f:6f:74:ff;

}

Just before the subnet declaration. At the first DHCP request from a PXE Loader it requests the /pxeboot via TFTP, if it's an Etherboot bootstrap the other declaration done already apply.

FreeBSD Diskless 12. Todo's

• /usr/diskless_root/base/etc/{passwd,master.passwd,pwd.db,spwd.db} needs to be updated once in a while, e.g. put it into your root crontab to copy them every minute or so.
• We are developing currently a webmin-module to handle diskless clients under FreeBSD.
• ...

FreeBSD Diskless 13. Links

Etherboot.org Etherboot

Netboot PDF paper, very useful

PXE Netboot PXE approach for installing FreeBSD (many hints can be used to run diskless as well)

otherwise google arround with "diskless freebsd".

                                                                                                                                   

Jail Tools

FreeBSD