Skip to main content.

The ultimate OpenBSD router

2013-11-13

Live demo in BSD Now Episode 011 | Originally written by TJ for bsdnow.tv | Last updated: 2014/08/19

"Friends don't let friends use consumer networking equipment."

This is a saying that many sysadmins and BSD/UNIX fans have probably heard. It's really easy to go to a store and pick up a cheap little plastic router, but you might regret it later. They're proprietary, have security issues and offer very little flexibility. Often times, these routers are severely limited in functionality because they're aimed at consumers who aren't very tech-savvy. We're going to show you how to build your own, and take back control of your network! Note that this will only be a wired router, not a WiFi access point. That may come at a later time.


Hardware

This is a list of hardware I'll be using (although nothing in this tutorial is specific to it).

Buy whatever hardware you want, just make sure the network cards are supported beforehand. The Soekris board I chose uses Intel NICs that are known to have good BSD support.


Background

I should probably define exactly what a router is, since everyone has different requirements. For mine, I want it to basically be a drop-in replacement for those cheap consumer ones you get. I've got three computers that need to share my cable internet connection. One of them is a FreeBSD server that I'd like to be able to SSH into remotely, but I don't want any other inbound connections to pass through. That means my router will be doing the following things:

  • Performing Network Address Translation
  • Giving my server "meimei" and my laptop "suigintou" static IP addresses
  • Handing out local IP addresses via DHCP to everyone else
  • Doing local DNS caching for the LAN and encrypting all outgoing DNS lookups
  • Allowing incoming SSH connections to my server
  • Allowing incoming SSH connections to the router (for maintenance, can be disabled later)
  • Using only IPv4
  • Automatically emailing me when there's an update available
  • Not being a pain in the ass to set up.. for the reader at least

This ultimate router will be running nothing more than OpenBSD. Everything I'm using is included in the base system unless otherwise noted. Pretty cool! The same setup can be done with FreeBSD, but you'll need to use the older pf syntax and a DHCP server from ports. If you're using FreeBSD, the files will also be in a different location in some cases. This guide is written purely for OpenBSD. On their website, they have a write-up on setting up a home gateway. It will provide a foundation for this tutorial's pf config file.

I'm going to assume you're capable of installing the OS on your machine and have a working internet connection. If you want a fully-encrypted installation, see our tutorial for that. If you have a serial cable, install it that way. Combining FDE and serial requires some additional steps though. If you don't like serial, install the OS on the USB drive from another computer, then put it in the Soekris box. You can also install over PXE if that's your thing. I used the whole device as one partition and installed to that. I let DHCP on the modem configure my first network interface and left the rest of them untouched.

The hardware I chose has four NICs. They show up in the OS as em0, em1, em2 and em3. I'm going to be using the first one as the external interface and the other three as the internal interfaces for the LAN. If you need more, I'd recommend getting a system with more NICs or picking up a cheap gigabit switch. The Soekris I'm using also has a PCIe x1 slot that can be used for expansion.


Filesystem and Network

So, let's get a shell and start working. You can use the serial console, SSH into the router from another system or just do this all on another computer booted from the USB drive (swapping it into the Soekris when done). First of all, since I'm using a flash drive for the OS, I want to minimize the number of writes to it. I'll append the "noatime" flag to the mount point and enable soft updates. You may also want to consider using mfs for /var and /tmp.

# vi /etc/fstab

Assuming my root device is "sd0" (yours might not be) my fstab will look like this:

/dev/sd0a / ffs rw,noatime,softdep 1 1

You'll need to reboot for this to take effect, but let's get everything in place before we do that. Next, let's add a few network settings so we can bridge your internal interfaces with the virtual vether interface.

# echo 'dhcp -inet6' > /etc/hostname.em0
# echo 'up -inet6' > /etc/hostname.em1
# echo 'up -inet6' > /etc/hostname.em2
# echo 'up -inet6' > /etc/hostname.em3
# echo 'inet 192.168.1.1 255.255.255.0 192.168.1.255 -inet6' > /etc/hostname.vether0
# vi /etc/hostname.bridge0

Add the following:

add vether0
add em1
add em2
add em3
blocknonip vether0
blocknonip em1
blocknonip em2
blocknonip em3
up -inet6

We need to enable IP forwarding and adjust a couple other values to increase performance.

# vi /etc/sysctl.conf

Add the following:

net.inet.ip.forwarding=1
kern.bufcachepercent=50
net.inet.ip.ifq.maxlen=1024
net.inet.tcp.mssdflt=1440
kern.securelevel=2
  • net.inet.ip.forwarding lets OpenBSD pass traffic through the interfaces when needed (required for NAT).
  • kern.bufcachepercent tells the kernel how much of the RAM it can use for cache.
  • net.inet.ip.ifq.maxlen should be 256 times the number of NICs you have - 4 in this case.
  • net.inet.tcp.mssdflt should match the "max-mss" value in our firewall config. A value of 1440 is a good general rule for most networks, but you can adjust it to be higher or lower depending on your needs, or disable it entirely.
  • kern.securelevel locks the securelevel to the highest setting and prevents changes to the firewall rules. This is optional, but recommended.

DHCP and NTP

Next we tell the DHCP server to start on boot, but we'll wait until we have a valid config file in place before we actually start it. Let's also configure OpenNTPD so that our router's time will always be correct.

# echo 'dhcpd_flags="vether0"' >> /etc/rc.conf.local

We'll tell the NTP daemon to start up on boot. If you plan on running an NTP server on your router, even if only for your LAN, we have a tutorial for that. We'll go ahead and sync the clock now too.

# echo 'ntpd_flags=""' >> /etc/rc.conf.local
# ntpd -s

Now we configure the DHCP server:

# mv /etc/dhcpd.conf /etc/dhcpd.conf.orig
# vi /etc/dhcpd.conf

This is what I use:

option domain-name-servers 192.168.1.1;
subnet 192.168.1.0 netmask 255.255.255.0 {
    option routers 192.168.1.1;
    range 192.168.1.4 192.168.1.254;
    host meimei {
        fixed-address 192.168.1.2;
        hardware ethernet 00:00:00:00:00:00;
        }
    host suigintou {
        fixed-address 192.168.1.3;
        hardware ethernet 11:11:11:11:11:11;
        }
}

You can specify any IP address range you want to use and any DNS servers you want to use. By default, I want all clients to query the local DNS resolver that we'll set up in just a minute. This will speed up repeated lookups and is handy to have. Use the MAC addresses of your computers if you want static IPs.


The Firewall

The centerpiece of this entire guide is the file /etc/pf.conf. That's where all the firewall rules will be going, and can be customized in many different ways. If you'd like to really learn pf, I recommend our tutorial.

# mv /etc/pf.conf /etc/pf.conf.orig
# vi /etc/pf.conf

As mentioned before, there are lots of resources online for learning the pf syntax. For my purposes, I ended up with something like this:

int_if="{ vether0 em1 em2 em3 }"
broken="224.0.0.22 127.0.0.0/8 192.168.0.0/16 172.16.0.0/12 \
         10.0.0.0/8 169.254.0.0/16 192.0.2.0/24 \
         192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24, \
         169.254.0.0/16 0.0.0.0/8 240.0.0.0/4 255.255.255.255/32"
set block-policy drop
set loginterface egress
set skip on lo0
match in all scrub (no-df random-id max-mss 1440)
match out on egress inet from !(egress:network) to any nat-to (egress:0)
antispoof quick for (egress)
block in quick on egress from { $broken no-route urpf-failed } to any
block in quick inet6 all
block out quick inet6 all
block out quick log on egress proto { tcp udp } from any to any port 53
block out quick log on egress from any to { no-route $broken }
block in all
pass out quick inet keep state
pass in on egress inet proto tcp to (egress) port 222 rdr-to 192.168.1.2
pass in on egress inet proto tcp from any to (egress) port 2222 flags S/SA synproxy state
pass in on $int_if inet
pass in on $int_if proto { tcp udp } from any to ! 192.168.1.1 port 53 rdr-to 192.168.1.1

My router will be running SSH on port 2222 and the server will be running SSH on port 222, both open to the internet. Check /etc/ssh/sshd_config for more options. Running sshd on an alternative port, while not really providing any additional security, will cut down the rogue login attempts. You definitely don't want your USB flash drive being worn out from /var/log filling up with Chinese IPs. Modify the "ListenAddress" line of sshd_config if you only want the router's sshd listening for connections on the LAN. See the pf.conf man page linked above if you want to learn more about each of the rules in more detail.


DNS

Setting up a local DNS caching server is very easy. We'll be using BIND, which is included in the OpenBSD base system as well. In the future, it will probably be replaced by unbound. The default configuration for named is mostly fine, but let's change a few little things. We'll be combining it with DNSCrypt so all our DNS lookups will be encrypted.

# echo 'named_flags=""' >> /etc/rc.conf.local
# vi /var/named/etc/named.conf

Here's a diff to show what to modify:

--- named.conf  2014-02-14 00:39:04.000000000 -0500
+++ named.conf  2014-02-14 00:39:44.000000000 -0500
@@ -10,15 +10,15 @@
 // the IPv6 localhost address.
 //
 acl clients {
-   localnets;
-   ::1;
+   127.0.0.1; 192.168.1.0/24;
 };

 options {
    version ""; // remove this to allow version queries

-   listen-on    { any; };
-   listen-on-v6 { any; };
+   forward only ;
+   forwarders {127.0.0.1 port 40 ;} ;
+   listen-on    { 127.0.0.1; 192.168.1.1; };

    empty-zones-enable yes;

Then we'll install dnscrypt-proxy. Unfortunately, it's not part of the base system, so we'll need to install it from ports or packages.

# export PKG_PATH=http://ftp.openbsd.org/pub/OpenBSD/`uname -r`/packages/`uname -m`/
# pkg_add dnscrypt-proxy
# echo '/usr/local/sbin/dnscrypt-proxy -a 127.0.0.1:40 -u _dnscrypt-proxy -l /dev/null -d' >> /etc/rc.local
# echo 'nameserver 127.0.0.1' > /etc/resolv.conf

You can edit /etc/dhclient.conf so it doesn't overwrite your local nameserver, or you can use a more forceful approach:

# chflags schg /etc/resolv.conf

The port will use OpenDNS by default. Check the included documentation if you'd like to use a different server. We probably don't need a sound server running on a router, so let's disable it.

# echo 'sndiod_flags=NO' >> /etc/rc.conf.local

Finally, reboot to make sure everything works.

# reboot

At this point you should be able to plug in some computers to the other ethernet ports and everything will work. They'll be assigned IP addresses and granted access to the internet, while being protected by the firewall. If that's all you want, you can stop reading right here. However, there are some other cool things you can do...


Email

It's possible to configure the router to send you nightly emails using nothing but the OpenBSD base system and an email account. I'm using a throwaway gmail account for this example, but you can obviously use any mail server. This first step is only required for OpenBSD versions before 5.6, where sendmail was still the default.

# mv /etc/mailer.conf /etc/mailer.conf.orig
# vi /etc/mailer.conf

Add the following:

sendmail     /usr/sbin/smtpctl
send-mail    /usr/sbin/smtpctl
mailq        /usr/sbin/smtpctl
makemap      /usr/libexec/smtpd/makemap
newaliases   /usr/libexec/smtpd/makemap

Make sure your system's hostname is in present in /etc/hosts:

127.0.0.1       localhost bsdnow.tv

Next we add the email account you'll be sending the mail from.

# newaliases
# echo 'gmail youruser@gmail.com:yourpassword' > /etc/mail/secrets
# chmod 640 /etc/mail/secrets
# chown root:_smtpd /etc/mail/secrets
# makemap /etc/mail/secrets

Move the default OpenSMTPD configuration to a backup file and create a new one.

# mv /etc/mail/smtpd.conf /etc/mail/smtpd.conf.orig
# vi /etc/mail/smtpd.conf

Add the following, changing the server to whatever you used:

listen on lo0
table aliases db:/etc/mail/aliases.db
table secrets db:/etc/mail/secrets.db
accept for local alias <aliases> deliver to mbox
accept for any relay via tls+auth://gmail@smtp.gmail.com:587 auth <secrets>

Finally, disable sendmail and enable OpenSMTPD by default. This is only needed if you are running a version before 5.6.

# /etc/rc.d/sendmail stop
# echo 'sendmail_flags=NO' >> /etc/rc.conf.local
# echo 'smtpd_flags=""' >> /etc/rc.conf.local
# /etc/rc.d/smtpd start

Now you should be able to take the output of any command and send it with your email account. We can test it by doing something like this:

# echo 'Woah, my router can send emails! Nice tutorials as always dude!' | mail -v feedback@bsdnow.tv

While this is very basic to most admins, the ability to send email from the router is really cool. You can pipe any command or script's output to an email, send it off and then check what's going on in the morning. It can be used for firewall logs, automatically checking for updates and patches, really anything you can think of! It's a phenomenal tool that can be used very creatively. You could even install GPG and encrypt the emails before sending them, especially if you're not a fan of gmail's privacy history.


Automatic Security Notifications

Here's a little script to check for errata (security or "reliability" updates to the OS) every night that I wrote. Before we make it, I'm going to get a checksum of the current errata page for my version of the OS. I don't claim to be a sed expert, but this works for my needs. If you want to create a new unprivileged user to run the script, that might be a good idea. The automatic source code patching will only work as root, unless you change the permissions of /usr/src. If you don't want the automatic patching, everything else can be adjusted to run as a normal user. Obviously make sure you actually have the system source code in /usr/src or that part won't work at all. If you want something less intrusive that simply alerts you that there are patches, comment out the relevant lines. You'll have to manually check the errata and patch it yourself in that case. Since version 5.5, the project also emails errata patches via the announce list. It's recommended to subscribe to it. OpenBSD once included the lynx web browser in the base system, but now you'll have to install it yourself.

# ftp -V4o - http://www.openbsd.org/errata`uname -r | cut -c 1,3`.html | sha256 > /root/checksum
# ftp -V4o /usr/local/bin/erratacheck http://www.bsdnow.tv/patches/erratachecker.txt

The script contains the following (plus comments and more info):

#!/bin/sh
# OpenBSD errata alert script from www.bsdnow.tv
# For src only, but xenocara can be easily added.

sleep `jot -r 1 0 3600` 

UNM=`uname -r`
VER=`uname -r | cut -c 1,3`
CVS=`uname -r | sed 's/\./_/'`
ERR=/root/errata
CHK=/root/checksum
URL=http://www.openbsd.org/errata$VER.html
SHA=sha256
WATASHI='me@myemail.com'
LASTKNOWN=`cat $CHK` 
FETCH=`ftp -V4o - $URL | $SHA`

if [ "$LASTKNOWN" == "$FETCH" ]; then
    exit
else
    echo "Check $URL for rebuilding instructions." > $ERR
    echo "Files affected:" >> $ERR
    cd /usr/src
        if sysctl kern.version | grep -q stable; then
            cvs -q up -rOPENBSD_$CVS -Pd 2>&1 | tee -a $ERR
        else
        if sysctl kern.version | grep -q current; then
            echo "this script is not for -current."
            exit 1
        else
        if sysctl kern.version | grep -q beta; then
            echo "this script is not for -beta."
            exit 1
        else
        ftp -V4 http://ftp.openbsd.org/pub/OpenBSD/patches/$UNM.tar.gz
        tar xzf $UNM.tar.gz
        rm $UNM.tar.gz
        mv $UNM/*/*sig .
        rm -r $UNM
        for f in *sig
            do
            signify -Vep /etc/signify/openbsd-`uname -r | sed 's/\.//'`-base.pub \
            -x $f -m - | (cd /usr/src && patch -p0 -sN)
            rm $f
            done
        fi
        fi
    fi
    lynx -dump -nolist $URL | \
    sed -e '1,23d' >> $ERR
    cat $ERR | \
    mail -s "New OpenBSD patches for `hostname`" $WATASHI
    rm $ERR
    echo $FETCH > $CHK
fi

Be sure to replace the email variable with your address. Then just put /usr/local/bin/erratacheck in the crontab and make it executable. The script will run at 7:30AM every day. You can change it to any time you want.

# chmod +x /usr/local/bin/erratacheck
# crontab -e

Add a line similar to this one:

30    7    *       *       *       /bin/sh /usr/local/bin/erratacheck

Power Saving

You may also want to enable apmd to save power if your hardware supports it. It will scale the CPU down during idle times and turn it up when the load reaches a certain point. Check the man page for a few different options. I'll be using the "cool running performance adjustment" mode.

# echo 'apmd_flags="-C"' >> /etc/rc.conf.local
# /etc/rc.d/apmd start

You can check what level (with 0 being the lowest, 100 being the highest) the CPU is running at with:

# sysctl hw.setperf

Or force it to use the lowest power setting:

# sysctl hw.setperf=0

Try different levels and apmd settings to find the balance you're most comfortable with. Always running it on the lowest setting might limit the data throughput too much, but it will really depend on what hardware you're using.


Conclusion

In closing, I think that OpenBSD makes for a great public-facing firewall. It's really handy to have all the tools needed in the base system. Some other ports that you might consider installing:

If you're not comfortable setting things up yourself via the commandline, pfSense is a good option. FreeBSD, NetBSD or DragonFlyBSD would all make excellent routers and firewalls as well. Whatever OS you decide to use, remember to keep it patched and up to date. Otherwise, you end up in a similar situation as you were when you had the consumer crap. OpenBSD does a new release every six months. Consider supporting their work by purchasing cool stuff. Now start replacing routers.

Latest News

BSDCan 2014

2014-04-30

We just wrapped up episode 35 after having some horrible audio issues. Sorry about the quality being lower than usual, we did the best we could given the circumstances. Next week we've got a normal episode, but the following week Allan and Kris will be at BSDCan. That week will...

AsiaBSDCon 2014

2014-03-05

Both Allan and Kris will be going to AsiaBSDCon this year, so episode 28 will be shorter than usual. We'll be back the following week with a huge episode. Hopefully they can get some interviews there!...

Christmas & New Year

2013-12-19

Episode 16 was just uploaded, and that's the last one we'll be doing live for this year. Episode 17 will be on Christmas, and feature a prerecorded interview with Scott Long about his BSD magic over at Netflix. Thanks for watching everyone! We look forward to more BSD Now in...

Welcome iXsystems

2013-12-11

As you may have noticed in Episode 015, BSD Now has gotten our first sponsor! We're very happy to welcome iXsystems to the BSD Now team. In case you aren't familiar with them, they have quite a long history with FreeBSD. Their current CTO is in fact Jordan Hubbard, one of...


Episode 052: Reverse Takeover

2014-08-27

Direct Download: Video | HD Video | MP3 Audio | OGG Audio | Torrent This episode was brought to you by Headlines FreeBSD foundation August update The foundation has published a new PDF detailing some of their recent activities It includes project development updates, the 10.1-RELEASE schedule and some of its new features There is also a...

Episode 051: Engineering Nginx

2014-08-20

Direct Download: Video | HD Video | MP3 Audio | OGG Audio | Torrent This episode was brought to you by Headlines Password gropers take spamtrap bait Our friend Peter Hansteen, who keeps his eyes glued to his log files, has a new blog post He seems to have discovered another new weird phenomenon in his...

Episode 050: VPN, My Dear Watson

2014-08-13

Direct Download: Video | HD Video | MP3 Audio | OGG Audio | Torrent This episode was brought to you by Headlines MeetBSD 2014 is approaching The MeetBSD conference is coming up, and will be held on November 1st and 2nd in San Jose, California MeetBSD has an "unconference" format, which means there will be both...

Episode 049: The PC-BSD Tour

2014-08-06

Direct Download: Video | HD Video | MP3 Audio | OGG Audio | Torrent This episode was brought to you by Headlines FreeBSD foundation semi-annual newsletter The FreeBSD foundation published their semi-annual newsletter, complete with a letter from the president of the foundation "In fact after reading [the president's] letter, I was motivated to come up...