Skip to main content.

Building OpenBSD binary packages in bulk

2014-04-16

Live demo in BSD Now Episode 033 | Originally written by TJ for bsdnow.tv | Last updated: 2015/05/01

NOTE: the author/maintainer of the tutorial(s) is no longer with the show, so the information below may be outdated or incorrect.

In an earlier tutorial, we showed you how to build FreeBSD binary packages en masse to create your own repository. This is basically the OpenBSD version of that, using a tool called dpb - Distributed Ports Build. It's a lot easier to pronounce! As was the case for the poudriere tutorial, it's recommended to do this on a system with a fairly powerful CPU and decent amount of RAM. Let's start out by getting a fresh ports tree.

# cd /usr
# cvs -qd anoncvs@anoncvs.usa.openbsd.org:/cvs get -rOPENBSD_`uname -r | tr . _` -P ports

Replace the mirror with one that's close to you. OpenBSD only provides port security updates for the latest -stable and -current, so to update your ports tree to -stable when a fix is committed, run:

# cd /usr/ports
# cvs -q up -rOPENBSD_`uname -r | tr . _` -Pd

Next I'm going to make a symlink in /usr/local/bin to the actual dpb script. This isn't required, but it saves me from having to edit my $PATH or type out the full location every time.

# ln -s /usr/ports/infrastructure/bin/dpb /usr/local/bin/dpb

I want to sign all the packages we'll be making, so we need to create a key pair and tell dpb to automatically sign all the packages it builds.

# signify -G -n -s /etc/signify/obsd-pkg.sec -p /etc/signify/obsd-pkg.pub
# echo 'SIGNING_PARAMETERS=-s signify -s /etc/signify/obsd-pkg.sec' >> /etc/mk.conf

You'll need to copy the "obsd-pkg.pub" file to your client machines' /etc/signify directory. Next, create a list of ports you want to be installed in a location that you can remember. The ports should be listed with one category/name on each line. For example:

# vi /root/pkg-list

editors/vim
net/irssi
games/sl

By default, OpenBSD's /etc/login.conf is too strict for dpb. Let's increase some of the max values so we can build our packages in parallel.

--- /etc/login.conf    Tue Feb 25 05:21:46 2014
+++ /etc/login.conf    Mon Mar 10 10:35:35 2014
@@ -43,11 +43,11 @@
 default:\
     :path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin /usr/local/bin /usr/local/sbin:\
     :umask=022:\
-    :datasize-max=512M:\
-    :datasize-cur=512M:\
-    :maxproc-max=256:\
-    :maxproc-cur=128:\
-    :openfiles-cur=512:\
+    :datasize-max=4G:\
+    :datasize-cur=4G:\
+    :maxproc-max=512:\
+    :maxproc-cur=256:\
+    :openfiles-cur=1024:\
     :stacksize-cur=4M:\
     :localcipher=blowfish,6:\
     :ypcipher=old:\
@@ -72,10 +72,10 @@
 # Staff have fewer restrictions and can login even when nologins are set.
 #
 staff:\
-    :datasize-cur=512M:\
+    :datasize-cur=4G:\
     :datasize-max=infinity:\
     :maxproc-max=512:\
-    :maxproc-cur=128:\
+    :maxproc-cur=256:\
     :ignorenologin:\
     :requirehome@:\
     :tc=default:

You will need to log out and back in for that to take effect. Before we start the build, let's make a symlink for our packages directory. By doing so, we can still run the webserver in a chroot.

# rm -r /usr/ports/packages
# mkdir /var/www/htdocs/packages
# chown root:daemon /var/www/htdocs/packages
# ln -s /var/www/htdocs/packages /usr/ports/packages

Finally, we can start the compiling now.

# dpb -c -P /root/pkg-list

If you see build failures or crashes, try increasing the login.conf limits even further. While it's building, you can see the status of the build using this legend (from the man page):

     I=   number of built packages that can be installed.

     B=   number of built packages, not yet known to be installable, because
          of run depends that still need to be built.

     Q=   number of packages in the queue, e.g., stuff that can be built now,
          assuming we have a free slot.

     T=   number of packages to build, where dependencies are not yet
          resolved.

     F=   number of distfiles to fetch, when -f is used.

     !=   number of ignored packages.  Details in engine.log.

     L=   list of packages that cannot currently be built because of locks.

     E=   list of packages in error, that cannot currently be built.

     H=   list of packages that haven't shown up yet, usually due to nfs, but
          watch out for revision bumps.

Once it finishes, you'll be able to see a list of packages that were built:

# ls /usr/ports/packages/`uname -m`/all

./                           gnupg-1.4.13p2.tgz           rsync-3.0.9p3-iconv.tgz
../                          groff-1.22.2p1.tgz           sl-3.03p0.tgz
adsuck-2.5.0p0.tgz           iperf-2.0.5p3.tgz            vim-7.3.850-no_x11.tgz
arpwatch-2.1a15p4.tgz        lrzsz-0.12.20p0.tgz          vnstat-1.11p3.tgz
dnscrypt-proxy-1.2.0.tgz     pftop-0.7p11.tgz

You can set up httpd to serve this directory to your other systems. If you're only doing this for a single system, you can skip this part.

# echo 'httpd_flags=""' >> /etc/rc.conf.local
# vi /etc/httpd.conf

Add the following:

server "default" {
        listen on egress port 80
        root "/htdocs"
        directory auto index
}

Replace "egress" with an internal IP address if you only want it open to the LAN. Start up the webserver:

# /etc/rc.d/httpd start

Remember to set the $PKG_PATH variable on the client systems, like so:

# export PKG_PATH=http://your-webserver/packages/`uname -m`/all/
# pkg_add vim

That's it. If you don't want to open a webserver, pkg_add can also use an "scp://" URL scheme if you have SSH setup on the build box.

Latest News

New announcement

2017-05-25

We understand that Michael Dexter, Brad Davis, and George Rosamond think there should be more real news....

Two Year Anniversary

2015-08-08

We're quickly approaching our two-year anniversary, which will be on episode 105. To celebrate, we've created a unique t-shirt design, available for purchase until the end of August. Shirts will be shipped out around September 1st. Most of the proceeds will support the show, and specifically allow us to buy...

New discussion segment

2015-01-17

We're thinking about adding a new segment to the show where we discuss a topic that the listeners suggest. It's meant to be informative like a tutorial, but more of a "free discussion" format. If you have any subjects you want us to explore, or even just a good name...

How did you get into BSD?

2014-11-26

We've got a fun idea for the holidays this year: just like we ask during the interviews, we want to hear how all the viewers and listeners first got into BSD. Email us your story, either written or a video version, and we'll read and play some of them for...


Episode 259: Long Live Unix

2018-08-15

Direct Download:MP3 AudioVideo This episode was brought to you by Picking the contest winner 1) Vincent 2) Bostjan 3) Andrew 4) Klaus-Hendrik 5) Will 6) Toby 7) Johnny 8) David 9) manfrom 10) Niclas 11) Gary 12) Eddy 13) Bruce 14) Lizz 15) Jim Random number generator Headlines The Strange Birth and Long Life of Unix They say that when one door closes on you, another opens. People generally...

Episode 258: OS Foundations

2018-08-08

Direct Download:MP3 AudioVideo This episode was brought to you by Headlines FreeBSD Foundation Update, July 2018 MESSAGE FROM THE EXECUTIVE DIRECTOR We’re in the middle of summer here, in Boulder, CO. While the days are typically hot, they can also be quite unpredictable. Thanks to the Rocky Mountains, waking up to 50-degree (~10...

Episode 257: Great NetBSD 8

2018-08-01

Direct Download:MP3 AudioVideo This episode was brought to you by Headlines NetBSD v8.0 Released The NetBSD Project is pleased to announce NetBSD 8.0, the sixteenth major release of the NetBSD operating system. This release brings stability improvements, hundreds of bug fixes, and many new features. Some highlights of the...

Episode 2^8: Because Computers

2018-07-25

Direct Download:MP3 AudioVideo This episode was brought to you by Win Celebrate our 256th episode with us. You can win a Mogics Power Bagel (not sponsored). To enter, go find the 4 episodes we did in December of 2017. In the opening, find the 4 letters in the bookshelf behind me. They...