Skip to main content.

Building OpenBSD binary packages in bulk

2014-04-16

Live demo in BSD Now Episode 033 | Originally written by TJ for bsdnow.tv | Last updated: 2015/05/01

NOTE: the author/maintainer of the tutorial(s) is no longer with the show, so the information below may be outdated or incorrect.

In an earlier tutorial, we showed you how to build FreeBSD binary packages en masse to create your own repository. This is basically the OpenBSD version of that, using a tool called dpb - Distributed Ports Build. It's a lot easier to pronounce! As was the case for the poudriere tutorial, it's recommended to do this on a system with a fairly powerful CPU and decent amount of RAM. Let's start out by getting a fresh ports tree.

# cd /usr
# cvs -qd anoncvs@anoncvs.usa.openbsd.org:/cvs get -rOPENBSD_`uname -r | tr . _` -P ports

Replace the mirror with one that's close to you. OpenBSD only provides port security updates for the latest -stable and -current, so to update your ports tree to -stable when a fix is committed, run:

# cd /usr/ports
# cvs -q up -rOPENBSD_`uname -r | tr . _` -Pd

Next I'm going to make a symlink in /usr/local/bin to the actual dpb script. This isn't required, but it saves me from having to edit my $PATH or type out the full location every time.

# ln -s /usr/ports/infrastructure/bin/dpb /usr/local/bin/dpb

I want to sign all the packages we'll be making, so we need to create a key pair and tell dpb to automatically sign all the packages it builds.

# signify -G -n -s /etc/signify/obsd-pkg.sec -p /etc/signify/obsd-pkg.pub
# echo 'SIGNING_PARAMETERS=-s signify -s /etc/signify/obsd-pkg.sec' >> /etc/mk.conf

You'll need to copy the "obsd-pkg.pub" file to your client machines' /etc/signify directory. Next, create a list of ports you want to be installed in a location that you can remember. The ports should be listed with one category/name on each line. For example:

# vi /root/pkg-list

editors/vim
net/irssi
games/sl

By default, OpenBSD's /etc/login.conf is too strict for dpb. Let's increase some of the max values so we can build our packages in parallel.

--- /etc/login.conf    Tue Feb 25 05:21:46 2014
+++ /etc/login.conf    Mon Mar 10 10:35:35 2014
@@ -43,11 +43,11 @@
 default:\
     :path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin /usr/local/bin /usr/local/sbin:\
     :umask=022:\
-    :datasize-max=512M:\
-    :datasize-cur=512M:\
-    :maxproc-max=256:\
-    :maxproc-cur=128:\
-    :openfiles-cur=512:\
+    :datasize-max=4G:\
+    :datasize-cur=4G:\
+    :maxproc-max=512:\
+    :maxproc-cur=256:\
+    :openfiles-cur=1024:\
     :stacksize-cur=4M:\
     :localcipher=blowfish,6:\
     :ypcipher=old:\
@@ -72,10 +72,10 @@
 # Staff have fewer restrictions and can login even when nologins are set.
 #
 staff:\
-    :datasize-cur=512M:\
+    :datasize-cur=4G:\
     :datasize-max=infinity:\
     :maxproc-max=512:\
-    :maxproc-cur=128:\
+    :maxproc-cur=256:\
     :ignorenologin:\
     :requirehome@:\
     :tc=default:

You will need to log out and back in for that to take effect. Before we start the build, let's make a symlink for our packages directory. By doing so, we can still run the webserver in a chroot.

# rm -r /usr/ports/packages
# mkdir /var/www/htdocs/packages
# chown root:daemon /var/www/htdocs/packages
# ln -s /var/www/htdocs/packages /usr/ports/packages

Finally, we can start the compiling now.

# dpb -c -P /root/pkg-list

If you see build failures or crashes, try increasing the login.conf limits even further. While it's building, you can see the status of the build using this legend (from the man page):

     I=   number of built packages that can be installed.

     B=   number of built packages, not yet known to be installable, because
          of run depends that still need to be built.

     Q=   number of packages in the queue, e.g., stuff that can be built now,
          assuming we have a free slot.

     T=   number of packages to build, where dependencies are not yet
          resolved.

     F=   number of distfiles to fetch, when -f is used.

     !=   number of ignored packages.  Details in engine.log.

     L=   list of packages that cannot currently be built because of locks.

     E=   list of packages in error, that cannot currently be built.

     H=   list of packages that haven't shown up yet, usually due to nfs, but
          watch out for revision bumps.

Once it finishes, you'll be able to see a list of packages that were built:

# ls /usr/ports/packages/`uname -m`/all

./                           gnupg-1.4.13p2.tgz           rsync-3.0.9p3-iconv.tgz
../                          groff-1.22.2p1.tgz           sl-3.03p0.tgz
adsuck-2.5.0p0.tgz           iperf-2.0.5p3.tgz            vim-7.3.850-no_x11.tgz
arpwatch-2.1a15p4.tgz        lrzsz-0.12.20p0.tgz          vnstat-1.11p3.tgz
dnscrypt-proxy-1.2.0.tgz     pftop-0.7p11.tgz

You can set up httpd to serve this directory to your other systems. If you're only doing this for a single system, you can skip this part.

# echo 'httpd_flags=""' >> /etc/rc.conf.local
# vi /etc/httpd.conf

Add the following:

server "default" {
        listen on egress port 80
        root "/htdocs"
        directory auto index
}

Replace "egress" with an internal IP address if you only want it open to the LAN. Start up the webserver:

# /etc/rc.d/httpd start

Remember to set the $PKG_PATH variable on the client systems, like so:

# export PKG_PATH=http://your-webserver/packages/`uname -m`/all/
# pkg_add vim

That's it. If you don't want to open a webserver, pkg_add can also use an "scp://" URL scheme if you have SSH setup on the build box.

Latest News

Two Year Anniversary

2015-08-08

We're quickly approaching our two-year anniversary, which will be on episode 105. To celebrate, we've created a unique t-shirt design, available for purchase until the end of August. Shirts will be shipped out around September 1st. Most of the proceeds will support the show, and specifically allow us to buy...

New discussion segment

2015-01-17

We're thinking about adding a new segment to the show where we discuss a topic that the listeners suggest. It's meant to be informative like a tutorial, but more of a "free discussion" format. If you have any subjects you want us to explore, or even just a good name...

How did you get into BSD?

2014-11-26

We've got a fun idea for the holidays this year: just like we ask during the interviews, we want to hear how all the viewers and listeners first got into BSD. Email us your story, either written or a video version, and we'll read and play some of them for...

EuroBSDCon 2014

2014-09-18

As you might expect, both Allan and Kris will be at EuroBSDCon this year. They'll be busy hunting down various BSD developers and forcing them to do interviews, but don't hesitate to say hi if you're a listener!...


Episode 186: The Fast And the Firewall: Tokyo Drift

2017-03-22

Direct Download:VideoHD VideoMP3 AudioOGG AudioTorrent This episode was brought to you by Headlines AsiaBSDcon Reports and Reviews AsiaBSDcon schedule Schedule and slides from the 4th bhyvecon Michael Dexter’s trip report on the iXsystems blog NetBSD AsiaBSDcon booth report TrueOS Community Guidelines are here! TrueOS has published its new Community Guidelines The TrueOS Project has existed for over ten years. Until...

Episode 185: Exit Interview

2017-03-16

Direct Download:VideoHD VideoMP3 AudioOGG AudioTorrent This episode was brought to you by Interview – Kris Moore – kris@trueos.org / @pcbsdKrisTrueOS founder, FreeNAS developer, BSD Now co-hostBenedict Reuschling – bcr@freebsd.org / @bsdbcrFreeBSD commiter & FreeBSD Foundation Vice President, BSD Now co-host Send questions, comments, show ideas/topics, or stories you want mentioned on the show...

Episode 184: Tokyo Dreaming

2017-03-08

Direct Download:VideoHD VideoMP3 AudioOGG AudioTorrent This episode was brought to you by Headlines OpenBSD A2k17 hackathon reports a2k17 hackathon report: Patrick Wildt on the arm64 port a2k17 hackathon report: Antoine Jacoutot on syspatch, rc.d improvements and more a2k17 hackathon report: Martin Pieuchot on NET_LOCK and much more a2k17 hackathon report: Kenneth Westerback on...

Episode 183: Getting Steamy Here

2017-03-01

Direct Download:VideoHD VideoMP3 AudioOGG AudioTorrent This episode was brought to you by Headlines playonbsd with TrueOS: It’s Getting Steamy in Here and I’ve Had Too Much Wine We’ve done a couple of tutorials in the past on using Steam and Wine with PC-BSD, but now with the addition of playonbsd to the...