NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

Virtualization showdown – FreeBSD’s bhyve vs. Linux’s KVM

The Birth of Standard Error

News Roundup

Investigating why Steam started picking a random font

Curious Case of Maintaining Sufficient Free Space with ZFS

Call for testing on updated Apple M1/M2 bootloader code

FreeBSD on my workstation

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• Brad - Initial Setup
• Joseph - openbsd and postgresql

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv ***

NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

Using FreeBSD’s pkg-audit

The 20 year old bug that went to Mars

It's rare that you come across a bug so subtle that it can last for two decades. But, that's exactly what has happened with the Lempel-Ziv-Oberhumer (LZO) algorithm. Initially written in 1994, Markus Oberhumer designed a sophisticated and extremely efficient compression algorithm so elegant and well architected that it outperforms zlib and bzip by four or five times their decompression speed.

I was impressed to find out that his LZO algorithm has gone to the planet Mars on NASA devices multiple times! Most recently, LZO has touched down on the red planet within the Mars Curiosity Rover, which just celebrated its first martian anniversary on Tuesday.

In the past few years, LZO has gained traction in file systems as well. LZO can be used in the Linux kernel within btrfs, squashfs, jffs2, and ubifs. A recent variant of the algorithm, LZ4, is used for compression in ZFS for Solaris, Illumos, and FreeBSD.

With its popularity increasing, Lempel-Ziv-Oberhumer has been rewritten by many engineering firms for both closed and open systems. These rewrites, however, have always been based on Oberhumer's core open source implementation. As a result, they all inherited a subtle integer overflow. Even LZ4 has the same exact bug, but changed very slightly.

Because the LZO algorithm is considered a library function, each specific implementation must be evaluated for risk, regardless of whether the algorithm used has been patched. Why? We are talking about code that has existed in the wild for two decades. The scope of this algorithm touches everything from embedded microcontrollers on the Mars Rover, mainframe operating systems, modern day desktops, and mobile phones. Engineers that have used LZO must evaluate the use case to identify whether or not the implementation is vulnerable, and in what format.

News Roundup

FreeBSD on Slimbook -- 14 months of updates

LLDB FreeBSD kernel core dump support

Steam on OpenBSD

Beastie Bits

• [OpenSSH Agent Restriction](http://undeadly.org/cgi?action=article;sid=20211220061017)
• [OpenBSD’s Clang upgraded to version 13](http://undeadly.org/cgi?action=article;sid=20211220060327)
• [Cool, but obscure X11 tools](http://cyber.dabamos.de/unix/x11/)

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

  ---

Headlines

What has to happen with Unix virtual memory when you have no swap space

Recently, Artem S. Tashkinov wrote on the Linux kernel mailing list about a Linux problem under memory pressure (via, and threaded here). The specific reproduction instructions involved having low RAM, turning off swap space, and then putting the system under load, and when that happened (emphasis mine):

Once you hit a situation when opening a new tab requires more RAM than is currently available, the system will stall hard. You will barely be able to move the mouse pointer. Your disk LED will be flashing incessantly (I'm not entirely sure why). [...]

I'm afraid I have bad news for the people snickering at Linux here; if you're running without swap space, you can probably get any Unix to behave this way under memory pressure. If you can't on your particular Unix, I'd actually say that your Unix is probably not letting you get full use out of your RAM.

To simplify a bit, we can divide pages of user memory up into anonymous pages and file-backed pages. File-backed pages are what they sound like; they come from some specific file on the filesystem that they can be written out to (if they're dirty) or read back in from. Anonymous pages are not backed by a file, so the only place they can be written out to and read back in from is swap space. Anonymous pages mostly come from dynamic memory allocations and from modifying the program's global variables and data; file backed pages come mostly from mapping files into memory with mmap() and also, crucially, from the code and read-only data of the program.

• See link for the rest of the article

Dsynth details on Dragonfly

First, history: DragonFly has had binaries of dports available for download for quite some time. These were originally built using poudriere, and then using the synth tool put together by John Marino. Synth worked both to build all software in dports, and as a way to test DragonFly’s SMP capability under extreme load.

Matthew Dillon is working on a new version, called dsynth. It is available now but not yet part of the build. He’s been working quickly on it and there’s plenty more commits than what I have linked here. It’s already led to finding more high-load fixes.

• dsynth

DSynth is basically synth written in C, from scratch. It is designed to give us a bulk builder in base and be friendly to porting and jails down the line (for now its uses chroot's).

The original synth was written by John R. Marino and its basic flow was used in writing this program, but as it was written in ada no code was directly copied.

• The intent is to make dsynth compatible with synth's configuration files and directory structure.

• This is a work in progress and not yet ready for prime-time. Pushing so we can get some more eyeballs. Most of the directives do not yet work (everything, and build works, and 'cleanup' can be used to clean up any dangling mounts).

• dsynth code

News Roundup

Instant Workstation

Some considerable time ago I wrote up instructions on how to set up a FreeBSD machine with the latest KDE Plasma Desktop. Those instructions, while fairly short (set up X, install the KDE meta-port, .. and that’s it) are a bit fiddly.

So – prompted slightly by a Twitter exchange recently – I’ve started a mini-sub-project to script the installation of a desktop environment and the bits needed to support it. To give it at least a modicum of UI, dialog(1) is used to ask for an environment to install and a display manager.

The tricky bits – pointed out to me after I started – are hardware support, although a best-effort is better than having nothing, I think.

In any case, in a VBox host it’s now down to running a single script and picking Plasma and SDDM to get a usable system for me. Other combinations have not been tested, nor has system-hardware-setup. I’ll probably maintain it for a while and if I have time and energy it’ll be tried with nVidia (those work quite well on FreeBSD) and AMD (not so much, in my experience) graphics cards when I shuffle some machines around.

• Here is the script in my GitHub repository with notes-for-myself.

New Servers, new Tech

Following up on an earlier post, the new servers for DragonFly are in place. The old 40-core machine used for bulk build, monster, is being retired. The power efficiency of the new machines is startling. Incidentally, this is where donations go – infrastructure.

• New servers in the colo, monster is being retired

We have three new servers in the colo now that will be taking most/all bulk package building duties from monster and the two blades (muscles and pkgbox64) that previously did the work. Monster will be retired. The new servers are a dual-socket Xeon (sting) and two 3900X based systems (thor and loki) which all together burn only around half the wattage that monster burned (500W vs 1000W) and 3 times the performance. That's at least a 6:1 improvement in performance efficiency.

With SSD prices down significantly the new machines have all-SSDs. These new machines allow us to build dports binary packages for release, master, and staged at the same time and reduces the full-on bulk build times for getting all three done down from 2 weeks to 2 days. It will allow us to more promptly synchronize updates to ports with dports and get binary packages up sooner.

Monster, our venerable 48-core quad-socket opteron is being retired. This was a wonderful dev machine for working on DragonFly's SMP algorithms over the last 6+ years precisely because its inter-core and inter-socket latencies were quite high. If a SMP algorithm wasn't spot-on, you could feel it. Over the years DragonFly's performance on monster in doing things like bulk builds increased radically as the SMP algorithms got better and the cores became more and more localized. This kept monster relevant far longer than I thought it would be.

But we are at a point now where improvements in efficiency are just too good to ignore. Monster's quad-socket opteron (4 x 12 core 6168's) pulls 1000W under full load while a single Ryzen 3900X (12 core / 24 thread) in a server configuration pulls only 150W, and is slightly faster on the same workload to boot.

I would like to thank everyone's generous donations over the last few years! We burned a few thousand on the new machines (as well as the major SSD upgrades we did to the blades) and made very good use of the money, particularly this year as prices for all major components (RAM, SSDs, CPUs, Mobos, etc) have dropped significantly.

Experimenting with streaming setups on NetBSD

Ever since OBS was successfully ported to NetBSD, I’ve been trying it out, seeing what works and what doesn’t. I’ve only just gotten started, and there’ll definitely be a lot of tweaking going forward.

Capturing a specific application’s windows seems to work okay. Capturing an entire display works, too. I actually haven’t tried streaming to Twitch or YouTube yet, but in a previous experiment a few weeks ago, I was able to run a FFmpeg command line and that could stream to Twitch mostly OK.

My laptop combined with my external monitor allows me to have a dual-monitor setup wherein the smaller laptop screen can be my “broadcasting station” while the bigger screen is where all the action takes place. I can make OBS visible on all Xfce workspaces, but keep it tucked away on that display only. Altogether, the setup should let me use the big screen for the fun stuff but I can still monitor everything in the small screen.

NetBSD Made Progress Thanks To GSoC In Its March Towards Steam Support

Ultimately the goal is to get Valve's Steam client running on NetBSD using their Linux compatibility layer while the focus the past few months with Google Summer of Code 2019 were supporting the necessary DRM ioctls for allowing Linux software running on NetBSD to be able to tap accelerated graphics support.

Student developer Surya P spent the summer working on compat_netbsd32 DRM interfaces to allow Direct Rendering Manager using applications running under their Linux compatibility layer.

These interfaces have been tested and working as well as updating the "suse131" packages in NetBSD to make use of those interfaces. So the necessary interfaces are now in place for Linux software running on NetBSD to be able to use accelerated graphics though Steam itself isn't yet running on NetBSD with this layer.

Those curious about this DRM ioctl GSoC project can learn more from the NetBSD blog. NetBSD has also been seeing work this summer on Wayland support and better Wine support to ultimately make this BSD a better desktop operating system and potentially a comparable gaming platform to Linux.

Beastie Bits

• FreeBSD in Wellington?
• FreeBSD on GFE
• Clarification
• Distrotest.net now with BSDs
• Lecture: Anykernels meet fuzzing NetBSD
• Sun Microsystems business plan from 1982 [pdf]

Feedback/Questions

• Alan - Questions
• Rodriguez - Feedback and a question
• Jeff - OpenZFS follow-up, FreeBSD Adventures

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

This episode was brought to you by

Headlines

More BSD conference videos

• We mentioned it a few times, but the "New Directions in Operating Systems" conference was held in November in the UK
• The presentations videos are now online, with a few BSD-related talks of interest
• Antti Kantee, Rump kernels and why / how we got here
• Franco Fichtner, An introduction to userland networking
• Robert Watson, New ideas about old OS security
• Lots of other interesting, but non-BSD-related, talks were also presented, so check the full list if you're interested in operating systems in general
• The 2014 AsiaBSDCon videos are also slowly being uploaded (better late than never)
• Kirk McKusick, An Overview of Security in the FreeBSD Kernel
• Matthew Ahrens, OpenZFS ensures the continued excellence of ZFS
• Eric Allman, Bambi Meets Godzilla: They Elope - Open Source Meets the Commercial World
• Scott Long, Modifying the FreeBSD kernel Netflix streaming servers
• Dru Lavigne, ZFS for the Masses
• Kris Moore, Snapshots, Replication, and Boot Environments
• David Chisnall, The Future of LLVM in the FreeBSD Toolchain
• Luba Tang, Bold, fast optimizing linker for BSD
• John Hixson, Introduction to FreeNAS development
• Zbigniew Bodek, Transparent Superpages for FreeBSD on ARM
• Michael Dexter, Visualizing Unix: Graphing bhyve, ZFS and PF with Graphite
• Peter Grehan, Nested Paging in Bhyve
• Martin Matuška, Deploying FreeBSD systems with Foreman and mfsBSD
• James Brown, Analysys of BSD Associate Exam Results
• Mindaugas Rasiukevicius, NPF - progress and perspective
• Luigi Rizzo, Netmap as a core networking technology
• Michael W. Lucas, Sudo: You're Doing it Wrong (not from a BSD conference, but still good)
• They should make for some great material to watch during the holidays ***

OpenBSD vs FreeBSD security features

• From the author of both the OpenBSD and FreeBSD secure gateway articles we've featured in the past comes a new entry about security
• The article goes through a list of all the security features enabled (and disabled) by default in both FreeBSD and OpenBSD
• It covers a wide range of topics, including: memory protection, randomization, encryption, privilege separation, Capsicum, securelevels, MAC, Jails and chroots, network stack hardening, firewall features and much more
• This is definitely one of the most in-depth and complete articles we've seen in a while - the author seems to have done his homework
• If you're looking to secure any sort of BSD box, this post has some very detailed explanations of different exploit mitigation techniques - be sure to read the whole thing
• There are also some good comments on DaemonForums and lobste.rs that you may want to read ***

The password? You changed it, right?

• Peter Hansteen has a new blog post up, detailing some weird SSH bruteforcing he's seen recently
• He apparently reads his auth logs when he gets bored at an airport
• This new bruteforcing attempt seems to be targetting D-Link devices, as evidenced by the three usernames the bots try to use
• More than 700 IPs have tried to get into Peter's BSD boxes using these names in combination with weak passwords
• Lots more details, including the lists of passwords and IPs, can be found in the full article
• If you're using a BSD router, things like this can be easily prevented with PF or fail2ban (and you probably don't have a "d-link" user anyway) ***

Get started with FreeBSD, an intro for Linux users

• Another new BSD article on a mainstream technology news site - seems we're getting popular
• This article is written for Linux users who may be considering switching over to BSD and wondering what it's all about
• It details installing FreeBSD 9.3 and getting a basic system setup, while touching on ports and packages, and explaining some terminology along the way
• "Among the legions of Linux users and admins, there seems to be a sort of passive curiosity about FreeBSD and other BSDs. Like commuters on a packed train, they gaze out at a less crowded, vaguely mysterious train heading in a slightly different direction and wonder what traveling on that train might be like" **

Interview - Michael W. Lucas - mwlucas@michaelwlucas.com / @mwlauthor

FreeBSD Mastery: Storage Essentials

News Roundup

OpenSMTPD status update

• The OpenSMTPD guys, particularly Gilles, have posted an update on what they've been up to lately
• As of 5.6, it's become the default MTA in OpenBSD, and sendmail will be totally gone in 5.7
• Email is a much more tricky protocol than you might imagine, and the post goes through some of the weirdness and problems they've had to deal with
• There's also another post that goes into detail on their upcoming filtering API - a feature many have requested
• The API is still being developed, but you can test it out now if you know what you're doing - full details in the article
• OpenSMTPD also has portable versions in FreeBSD ports and NetBSD pkgsrc, so check it out ***

OpenCrypto changes in FreeBSD

• A little while back, we talked to John-Mark Gurney about updating FreeBSD's OpenCrypto framework, specifically for IPSEC
• Some of that work has just landed in the -CURRENT branch, and the commit has a bit of details
• The ICM and GCM modes of AES were added, and both include support for AESNI
• There's a new port - "nist-kat" - that can be used to test the new modes of operation
• Some things were fixed in the process as well, including an issue that would leak timing info and result in the ability to forge messages
• Code was also borrowed from both OpenBSD and NetBSD to make this possible ***

First thoughts on OpenBSD's httpd

• Here we have a blog post from a user of OpenBSD's new homegrown web server that made its debut in 5.6
• The author loves that it has proper privilege separation, a very simple config syntax and that it always runs in a chroot
• He also mentions dynamic content hosting with FastCGI, and provides an example of how to set it up
• Be sure to check our interview with Reyk about the new httpd if you're curious on how it got started
• Also, if you're running the version that came with 5.6, there's a huge patch you can apply to get a lot of the features and fixes from -current without waiting for 5.7 ***

Steam on PCBSD

• One of the most common questions people who want to use BSD as a desktop ask us is "can I run games?" or "can I use steam?"
• Steam through the Linux emulation layer (in FreeBSD) may be possible soon, but it's already possible to use it with WINE
• This video shows how to get Steam set up on PCBSD using the Windows version
• There are also some instructions in the video description to look over
• A second video details getting streaming set up ***

Feedback/Questions

• Charlie writes in
• Sean writes in
• Predrag writes in ***