In terms of the basic ideas of FreeBSD jails, FreeBSD Handbook Chapter 15 Jails explais them quite well, including a "raw" way of building jails. However, a tool called ezjail makes it quite easy, especially multi-jail management.

Preparation

First install ezjail from FreeBSD ports. Edit ezjail’s conf file /usr/local/ezjail.conf according to its comments, and here is the copy works for me:

ezjail_jaildir=/usr/jails     # just more convenient if it is not under /usr/local
ezjail_jailtemplate=${ezjail_jaildir}/newjail
ezjail_jailbase=${ezjail_jaildir}/basejail
ezjail_sourcetree=/usr/src

# In case you want to provide a copy of ports tree in base jail, set this to a cvsroot near you
ezjail_portscvsroot=anoncvs@anoncvs1.FreeBSD.org:/home/ncvs

# This is where the install sub command defaults to fetch its packages from
ezjail_ftphost=ftp.freebsd.org

ezjail_default_execute="/usr/bin/login -f root"
ezjail_default_flavour=""
ezjail_archivedir=`pwd -P`
# base jail will provide a soft link from /usr/bin/perl to /usr/local/bin/perl
# to accomodate all scripts using '#!/usr/bin/perl'...
ezjail_uglyperlhack="YES"

# Default options for newly created jails
# Note: Be VERY careful about disabling ezjail_mount_enable. Mounting
# basejail via nullfs depends on this. You will have to find other
# ways to provide your jail with essential system files
ezjail_mount_enable="YES"
ezjail_devfs_enable="YES"
ezjail_devfs_ruleset="devfsrules_jail"
ezjail_procfs_enable="YES"
ezjail_fdescfs_enable="YES"

Actually I only change the value of ezjail_portscvsroot entry, since the default one does not work for FreeBSD 8.0 which is the release currently running in my box. Besides that, default settings should be fine.

Second, grab FreeBSD source via CVSup (refer to this), and build the kernel:

cd /usr/src
sudo make buildworld

That may take a while depending on how fast your computer is.

It is also a good time to configure the networking. Apart from local loopback lo, assume you have two network interfaces fxp0 and fxp1, and fxp0 is configured to have external IP address 192.168.10.10/16 while fxp1 has internal IP address 192.168.1.100/24. Assume the jail will be using the latter IP address 192.168.1.100. The related /etc/rc.conf can look like:

hostname="myhost.example.com"
ifconfig_fxp0="192.168.10.10/16"
defaultrouter="192.168.1.1"
ifconfig_fxp1="192.168.1.100/24"
pf_enable="YES"
pflog_enable="YES"
sshd_enable="YES"
syslogd_enable="YES"
ezjail_enable="YES"

Of course, another way of assigning jail IP address is to create an alias of fxp0 or fxp1.

The PF conf file /etc/pf.conf can be something like

ext_if="fxp0"    # The external interface
int_if="fxp1"    # The internal interface
external_addr="192.168.10.10"
internal_net="192.168.1.0/24"

nat on $ext_if from $internal_net to any -> ($ext_if)
rdr on $ext_if proto tcp from any to $external_addr/16 port 80 -> 192.168.1.100 port 80  # in case running a webserver in jail

# Make sure we don't block any traffic
pass in all
pass out all
Note
 Whenever you change the PF conf file, remember to flush rules by sudo pfctl -F all -f /etc/pf.conf. Also, sudo pfctl -vnf /etc/pf.conf can be used to check PF errors if rules are complicated (-v means verbose, -n means not being actually loaded), and sudo pfctl -s nat to check whether NAT rules are loaded.

Also, configure sshd and syslogd (or any other daemons) to listen on specific IP address which should not coincide with jail’s IP addresses. By default, they listen at whole-range IP addresses including the addresses jails will be using, which will prompt warnings when you create jails later. If you do nothing about it, it will prevent jails from starting.

So edit /etc/ssh/sshd_configure, modify/add:

ListenAddress 127.0.0.1
ListenAddress 192.168.10.10

and open /etc/rc.conf and add

syslogd_program="/usr/sbin/syslogd"
syslogd_flags="-s -b 127.0.0.1"

At last restart sshd and syslogd

sudo /etc/rc.d/sshd restart
sudo /etc/rc.d/syslogd restart

Creating Jails

Ezjail really makes this job easy. First, create base jail and a copy of ports tree:

sudo ezjail-admin update -i -p

It will take some time. But before doing that, make sure you have build the kernel in /usr/src as described in "Preparation".

Jail Flavor

Jail flavor is a jail template which is to be copied to newly created jails, plus an initialization script to be run at the same time. It can be quite handy if you need to create multiple jails, since you don’t have to recompile all ports packages which you can not live with (like vim, etc).

To create a jail flavor is simply copying files (well, maybe a little more). All jail flavors are located in $JAILROOT/flavours (in this case /usr/jails/flavours). Create your own flavor:

cd /usr/jails/flavours
sudo mkdir generic
sudo mkdir -p generic/usr/local
sudo mkdir -p generic/var/db
sudo mkdir etc
sudo cp -R /usr/local generic/usr
sudo cp -R /var/db/pkg generic/var/db/
sudo cp /etc/localtime generic/etc/
sudo cp /etc/resolv.conf generic/etc/
sudo cp /etc/motd generic/etc/
sudo cp /etc/shells generic/etc/
sudo cp /etc/syslog.conf generic/etc/
sudo cp /etc/make.conf generic/etc/
Note
 I used to have some problems after copying all /usr/local/* and /var/db/pkg to one jail flavor: somehow it just refused me to log into the jail console. I have no idea what happened. But otherwise it seemed just fine. Copying /etc/* is okay too. But I do have to go through the process to recompile packages I normally use in each jail.

If you want, you can also copy /etc/periodic.conf to /usr/jails/flavours/etc/. Refer to some resources online (e.g., here) and periodic.conf(5). Also, you can edit jail’s rc.conf for fine tuning.

sendmail_enable="NO"
rpcbind_enable="NO"   # disable RPC daemon
kern_securelevel_enable="YES"
kern_securelevel="1"
# and more .....
Note
ezjail.flavour
Notice that you can also edit /usr/jails/flavours/generic/ezjail.flavour, which is essentially a shell script, to automate initialization jobs during jail creation, such as adding groups and users, creating files, etc.

Also, be sure to add these lines to jail’s make.conf:

WRKDIRPREFIX=           /var/ports
DISTDIR=                /var/ports/distfiles
PACKAGES=               /var/ports/packages

The reason is jail’s /usr/ports' is read-only. If you just cd into the package’s ports directory and run sudo make install clean, you will probably get errors like "/usr/ports/xxxx is read-only".

Creating A Jail

Now it is time to create a jail:

sudo ezjail-admin create -f generic webserver 192.168.1.100

It means you create a jail named webserver with (local) IP address 192.168.1.100 with jail flavor "generic". Then you need to assign a fully qualified domain name (FQDN) to the jail by editing an entry in its conf file at /usr/local/etc/ezjail/webserver:

export jail_webserver_hostname="webserver.example.com"

Now, start the jail:

sudo /usr/local/etc/rc.d/ezjail.sh start

Managing Jails

To see the jail status:

sudo ezjail-admin list

After you make sure jail is running, start the console of jail:

sudo ezjail-admin console webserver

You will log in as jail root by default. Now it looks like a freshly installed FreeBSD system. Since you have already configure PF outside jail, jail’s outgoing traffic will be routed outside seen as myhost.example.com. However, it is important to configure jail’s DNS otherwise you will have networking trouble. I just add google’s public DNS into /etc/resolv.conf

nameserver 8.8.8.8
nameserver 8.8.4.4
Note
 You can also do it when you create the jail flavor.

To update jail’s copy of ports tree:

sudo ezjail-admin update -P

Since jail is segregated from the host system, the its ports tree updates will not affect the ports tree of the host system.

To stop jails, run

sudo ezjail-admin stop

To restart, run

sudo ezjail-admin restart

For more information, consult ezjail-admin(1).

Install Apache In Jail

Simple. First install apache from ports (/usr/ports/www/apache22), add apache22_enable="YES" to jail’s /etc/rc.conf, configure /usr/local/etc/apache22/httpd.conf, and run

/usr/local/etc/rc.d/apache22 start

or

/usr/local/sbin/apachectl start

If it gives errors, first check if kernel module accf_http is loaded outside the jail (jail does not have the privilege to load kernel module), and then check apache error log file (typically at /var/log/httpd-error.log. The error may be from apache’s various modules.

Removing Jails

If you use ezjail, it is easy:

sudo ezjail-admin delete -w webserver

If you install jails following FreeBSD Handbook Chapter 15, first stop the jail

sudo /etc/rc.d/jail stop jail1

and remove the corresponding jail records in /etc/rc.conf/. After that, just delete the whole jail directory.

sudo chflags -R noschg $JAIL/webserver
sudo rm -rf $JAIL/webserver

Reference

blog comments powered by Disqus