NOTES

This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

OpenBSD is a Cozy Operating System

Lichee Console 4A - RISC-V mini laptop

News Roundup

Lessons learned with XZ vulnerability

Techies vs spies: the xz backdoor debate

Not Not Porting 9front to Power64

One less Un*xy option for 32-bit PowerPC

Beastie Bits

• 20 years since...
• Jails PDFs
• NixOS BSD
• rigg - run indie games on OpenBSD
• pkgsrc 2024Q1
• PackMule
• AcephalOS - A new FreeBSD image build tool

Tarsnap

This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

• Join us and other BSD Fans in our BSD Now Telegram channel

NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

Using FreeBSD’s pkg-audit

The 20 year old bug that went to Mars

It's rare that you come across a bug so subtle that it can last for two decades. But, that's exactly what has happened with the Lempel-Ziv-Oberhumer (LZO) algorithm. Initially written in 1994, Markus Oberhumer designed a sophisticated and extremely efficient compression algorithm so elegant and well architected that it outperforms zlib and bzip by four or five times their decompression speed.

I was impressed to find out that his LZO algorithm has gone to the planet Mars on NASA devices multiple times! Most recently, LZO has touched down on the red planet within the Mars Curiosity Rover, which just celebrated its first martian anniversary on Tuesday.

In the past few years, LZO has gained traction in file systems as well. LZO can be used in the Linux kernel within btrfs, squashfs, jffs2, and ubifs. A recent variant of the algorithm, LZ4, is used for compression in ZFS for Solaris, Illumos, and FreeBSD.

With its popularity increasing, Lempel-Ziv-Oberhumer has been rewritten by many engineering firms for both closed and open systems. These rewrites, however, have always been based on Oberhumer's core open source implementation. As a result, they all inherited a subtle integer overflow. Even LZ4 has the same exact bug, but changed very slightly.

Because the LZO algorithm is considered a library function, each specific implementation must be evaluated for risk, regardless of whether the algorithm used has been patched. Why? We are talking about code that has existed in the wild for two decades. The scope of this algorithm touches everything from embedded microcontrollers on the Mars Rover, mainframe operating systems, modern day desktops, and mobile phones. Engineers that have used LZO must evaluate the use case to identify whether or not the implementation is vulnerable, and in what format.

News Roundup

FreeBSD on Slimbook -- 14 months of updates

LLDB FreeBSD kernel core dump support

Steam on OpenBSD

Beastie Bits

• [OpenSSH Agent Restriction](http://undeadly.org/cgi?action=article;sid=20211220061017)
• [OpenBSD’s Clang upgraded to version 13](http://undeadly.org/cgi?action=article;sid=20211220060327)
• [Cool, but obscure X11 tools](http://cyber.dabamos.de/unix/x11/)

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

  ---

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

I got the J language working on OpenBSD

Rubenerd: Comparing FreeBSD GELI and OpenZFS encrypted pools with keys

News Roundup

What is FreeBSD, actually? Think again.

OpenBSD's pledge and unveil from Python

Beastie Bits

• [Hibernate time reduced](http://undeadly.org/cgi?action=article;sid=20210831050932)
• [(open)rsync gains include/exclude support](http://undeadly.org/cgi?action=article;sid=20210830081715)
• [Producer JT's latest ancient find that he needs help with](https://twitter.com/q5sys/status/1440105555754848257)
• [Doas comes to MidnightBSD](https://github.com/slicer69/doas)
• [FreeBSD SSH Hardening](https://gist.github.com/koobs/e01cf8869484a095605404cd0051eb11)
• [OpenBSD 6.8 and you](https://home.nuug.no/~peter/openbsd_and_you/#1)
• [By default, scp(1) now uses SFTP protocol](https://undeadly.org/cgi?action=article;sid=20210910074941)
• [FreeBSD 11.4 end-of-life](https://lists.freebsd.org/pipermail/freebsd-announce/2021-September/002060.html)
• [sched_ule(4): Improve long-term load balancer](https://cgit.freebsd.org/src/commit/?id=e745d729be60a47b49eb19c02a6864a747fb2744)

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

  ---

FreeBSD and custom firmware on the Google Pixelbook

• FreeBSD and custom firmware on the Google Pixelbook

Back in 2015, I jumped on the ThinkPad bandwagon by getting an X240 to run FreeBSD on. Unlike most people in the ThinkPad crowd, I actually liked the clickpad and didn\u2019t use the trackpoint much. But this summer I\u2019ve decided that it was time for something newer. I wanted something..

• lighter and thinner (ha, turns out this is actually important, I got tired of carrying a T H I C C laptop - Apple was right all along);
• with a 3:2 display (why is Lenovo making these Serious Work\u2122 laptops 16:9 in the first place?? 16:9 is awful in below-13-inch sizes especially);
• with a HiDPI display (and ideally with a good size for exact 2x scaling instead of fractional);
• with USB-C ports;
• without a dGPU, especially without an NVIDIA GPU;
• assembled with screws and not glue (I don\u2019t necessarily need expansion and stuff in a laptop all that much, but being able to replace the battery without dealing with a glued chassis is good);
• supported by FreeBSD of course (\u201csome development required\u201d is okay but I\u2019m not going to write big drivers);
• how about something with open source firmware, that would be fun.

I was considering a ThinkPad X1 Carbon from an old generation - the one from the same year as the X230 is corebootable, so that\u2019s fun. But going back in processor generations just doesn\u2019t feel great. I want something more efficient, not less!

And then I discovered the Pixelbook. Other than the big huge large bezels around the screen, I liked everything about it. Thin aluminum design, a 3:2 HiDPI screen, rubber palm rests (why isn\u2019t every laptop ever doing that?!), the \u201cconvertibleness\u201d (flip the screen around to turn it into.. something rather big for a tablet, but it is useful actually), a Wacom touchscreen that supports a pen, mostly reasonable hardware (Intel Wi-Fi), and that famous coreboot support (Chromebooks\u2019 stock firmware is coreboot + depthcharge).

So here it is, my new laptop, a Google Pixelbook.

• Conclusion

Pixelbook, FreeBSD, coreboot, EDK2 good.

Seriously, I have no big words to say, other than just recommending this laptop to FOSS enthusiasts :)

Porting NetBSD to the AMD x86-64: a case study in OS portability

• Abstract

NetBSD is known as a very portable operating system, currently running on 44 different architectures (12 different types of CPU). This paper takes a look at what has been done to make it portable, and how this has decreased the amount of effort needed to port NetBSD to a new architecture. The new AMD x86-64 architecture, of which the specifications were published at the end of 2000, with hardware to follow in 2002, is used as an example.

• Portability

Supporting multiple platforms was a primary goal of the NetBSD project from the start. As NetBSD was ported to more and more platforms, the NetBSD kernel code was adapted to become more portable along the way.

• General

Generally, code is shared between ports as much as possible. In NetBSD, it should always be considered if the code can be assumed to be useful on other architectures, present or future. If so, it is machine-independent and put it in an appropriate place in the source tree. When writing code that is intended to be machine-independent, and it contains conditional preprocessor statements depending on the architecture, then the code is likely wrong, or an extra abstraction layer is needed to get rid of these statements.

• Types

Assumptions about the size of any type are not made. Assumptions made about type sizes on 32-bit platforms were a large problem when 64-bit platforms came around. Most of the problems of this kind had to be dealt with when NetBSD was ported to the DEC Alpha in 1994. A variation on this problem had to be dealt with with the UltraSPARC (sparc64) port in 1998, which is 64-bit, but big endian (vs. the little-endianness of the Alpha). When interacting with datastructures of a fixed size, such as on-disk metadata for filesystems, or datastructures directly interpreted by device hardware, explicitly sized types are used, such as uint32_t, int8_t, etc.

• Conclusions and future work

The port of NetBSD to AMD's x86-64 architecture was done in six weeks, which confirms NetBSD's reputation as being a very portable operating system. One week was spent setting up the cross-toolchain and reading the x86-64 specifications, three weeks were spent writing the kernel code, one week was spent writing the userspace code, and one week testing and debugging it all. No problems were observed in any of the machine-independent parts of the kernel during test runs; all (simulated) device drivers, file systems, etc, worked without modification.

News Roundup

ZFS performance really does degrade as you approach quota limits

Every so often (currently monthly), there is an "OpenZFS leadership meeting". What this really means is 'lead developers from the various ZFS implementations get together to talk about things'. Announcements and meeting notes from these meetings get sent out to various mailing lists, including the ZFS on Linux ones.

• In the September meeting notes, I read a very interesting (to me) agenda item:
  • Relax quota semantics for improved performance (Allan Jude)
  • Problem: As you approach quotas, ZFS performance degrades.
  • Proposal: Can we have a property like quota-policy=strict or loose, where we can optionally allow ZFS to run over the quota as long as performance is not decreased.

This is very interesting to me because of two reasons. First, in the past we have definitely seen significant problems on our OmniOS machines, both when an entire pool hits a quota limit and when a single filesystem hits a refquota limit. It's nice to know that this wasn't just our imagination and that there is a real issue here. Even better, it might someday be improved (and perhaps in a way that we can use at least some of the time).

Second, any number of people here run very close to and sometimes at the quota limits of both filesystems and pools, fundamentally because people aren't willing to buy more space. We have in the past assumed that this was relatively harmless and would only make people run out of space. If this is a known issue that causes serious performance degradation, well, I don't know if there's anything we can do, but at least we're going to have to think about it and maybe push harder at people. The first step will have to be learning the details of what's going on at the ZFS level to cause the slowdown. (It's apparently similar to what happens when the pool is almost full, but I don't know the specifics of that either.)

With that said, we don't seem to have seen clear adverse effects on our Linux fileservers, and they've definitely run into quota limits (repeatedly). One possible reason for this is that having lots of RAM and SSDs makes the effects mostly go away. Another possible reason is that we haven't been looking closely enough to see that we're experiencing global slowdowns that correlate to filesystems hitting quota limits. We've had issues before with somewhat subtle slowdowns that we didn't understand (cf), so I can't discount that we're having it happen again.

Fixing up KA9Q-unix, or "neck deep in 30 year old codebases.."

I'll preface this by saying - yes, I'm still neck deep in FreeBSD's wifi stack and 802.11ac support, but it turns out it's slow work to fix 15 year old locking related issues that worked fine on 11abg cards, kinda worked ok on 11n cards, and are terrible for these 11ac cards. I'll .. get there.

Anyhoo, I've finally been mucking around with AX.25 packet radio. I've been wanting to do this since I was a teenager and found out about its existence, but back in high school and .. well, until a few years ago really .. I didn't have my amateur radio licence. But, now I do, and I've done a bunch of other stuff with a bunch of other radios. The main stumbling block? All my devices are either Apple products or run FreeBSD - and none of them have useful AX.25 stacks. The main stacks of choice these days run on Linux, Windows or are a full hardware TNC.

So yes, I was avoiding hacking on AX.25 stuff because there wasn't a BSD compatible AX.25 stack. I'm 40 now, leave me be.

But! A few weeks ago I found that someone was still running a packet BBS out of San Francisco. And amazingly, his local node ran on FreeBSD! It turns out Jeremy (KK6JJJ) ported both an old copy of KA9Q and N0ARY-BBS to run on FreeBSD! Cool!

I grabbed my 2m radio (which is already cabled up for digital modes), compiled up his KA9Q port, figured out how to get it to speak to Direwolf, and .. ok. Well, it worked. Kinda.

HAMMER2 and fsck for review

HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there\u2019s now a fsck command, useful if you want a report of data validity rather than any manual repair process.

[The return of startx(1) for non-root users with some caveats

Mark Kettenis (kettenis@) has recently committed changes which restore a certain amount of startx(1)/xinit(1) functionality for non-root users. The commit messages explain the situation:

CVSROOT:    /cvs
Module name:    src
Changes by:    kettenis@cvs.openbsd.org    2019/09/15 06:25:41

Modified files:
    etc/etc.amd64  : fbtab 
    etc/etc.arm64  : fbtab 
    etc/etc.hppa   : fbtab 
    etc/etc.i386   : fbtab 
    etc/etc.loongson: fbtab 
    etc/etc.luna88k: fbtab 
    etc/etc.macppc : fbtab 
    etc/etc.octeon : fbtab 
    etc/etc.sgi    : fbtab 
    etc/etc.sparc64: fbtab 

Log message:
Add ttyC4 to lost of devices to change when logging in on ttyC0 (and in some cases also the serial console) such that X can use it as its VT when running without root privileges.

ok jsg@, matthieu@
CVSROOT:    /cvs
Module name:    xenocara
Changes by:    kettenis@cvs.openbsd.org    2019/09/15 06:31:08

Modified files:
    xserver/hw/xfree86/common: xf86AutoConfig.c 

Log message:
Add modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus.

This makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4).  In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)).

ok jsg@, matthieu@

Beastie Bits

• ASCII table and history. Or, why does Ctrl+i insert a Tab in my terminal?
• Sourcehut makes BSD software better
• Chaosnet for Unx
• The Vim-Inspired Editor with a Linguistic Twist
• bhyvearm64: CPU and Memory Virtualization on Armv8.0-A
• DefCon25 - Are all BSD created Equally - A Survey of BSD Kernel vulnerabilities

Feedback/Questions

• Tim - GSoC project ideas for pf rule syntax translation
• Brad - Steam on FreeBSD
• Ruslan - FreeBSD Quarterly Status Report - Q2 2019

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

The UNIX Philosophy in 2019

Today, Linux and open source rules the world, and the UNIX philosophy is widely considered compulsory. Organizations are striving to build small, focused applications that work collaboratively in a cloud and microservices environment. We rely on the network, as well as HTTP (text) APIs for storing and referencing data. Moreover, nearly all configuration is stored and communicated using text (e.g. YAML, JSON or XML). And while the UNIX philosophy has changed dramatically over the past 5 decades, it hasn’t strayed too far from Ken Thompson’s original definition in 1973:

• We write programs that do one thing and do it well
• We write programs to work together
• And we write programs that handle text streams, because that is a universal interface

Why Use Package Managers?

Valuable research is often hindered or outright prevented by the inability to install software. This need not be the case.

Since I began supporting research computing in 1999, I’ve frequently seen researchers struggle for days or weeks trying to install a single open source application. In most cases, they ultimately failed.

In many cases, they could have easily installed the software in seconds with one simple command, using a package manager such as Debian packages, FreeBSD ports, MacPorts, or Pkgsrc, just to name a few.

Developer websites often contain poorly written instructions for doing “caveman installs”; manually downloading, unpacking, patching, and building the software. The same laborious process must often be followed for other software packages on which it depends, which can sometimes number in the dozens. Many researchers are simply unaware that there are easier ways to install the software they need. Caveman installs are a colossal waste of man-hours. If 1000 people around the globe spend an average of 20 hours each trying to install the same program that could have been installed with a package manager (this is not uncommon), then 20,000 man-hours have been lost that could have gone toward science. How many important discoveries are delayed by this?

The elite research institutions have ample funding and dozens of IT staff dedicated to research computing. They can churn out publications even if their operation is inefficient. Most institutions, however, have few or no IT staff dedicated to research, and cannot afford to squander precious man-hours on temporary, one-off software installs. The wise approach for those of us in that situation is to collaborate on making software deployment easier for everyone. If we do so, then even the smallest research groups can leverage that work to be more productive and make more frequent contributions to science.

Fortunately, the vast majority of open source software installs can be made trivial for anyone to do for themselves. Modern package managers perform all the same steps as a caveman install, but automatically. Package managers also install dependencies for us automatically.

News Roundup

Touchpad, Interrupted

For two years I've been driving myself crazy trying to figure out the source of a driver problem on OpenBSD: interrupts never arrived for certain touchpad devices. A couple weeks ago, I put out a public plea asking for help in case any non-OpenBSD developers recognized the problem, but while debugging an unrelated issue over the weekend, I finally solved it.

It's been a long journey and it's a technical tale, but here it is.

Porting wine to amd64 on NetBSD, second evaluation report

• Summary

Presently, Wine on amd64 is in test phase. It seems to work fine with caveats like LD_LIBRARY_PATH which has to be set as 32-bit Xorg libs don't have ${PREFIX}/emul/netbsd32/lib in its rpath section. The latter is due to us extracting 32-bit libs from tarballs in lieu of building 32-bit Xorg on amd64. As previously stated, pkgsrc doesn't search for pkgconfig files in ${PREFIX}/emul/netbsd32/lib which might have inadvertent effects that I am unaware of as of now. I shall be working on these issues during the final coding period. I would like to thank @leot, @maya and @christos for saving me from shooting myself in the foot many a time. I, admittedly, have had times when multiple approaches, which all seemed right at that time, perplexed me. I believe those are times when having a mentor counts, and I have been lucky enough to have really good ones. Once again, thanks to Google for this wonderful opportunity.

Enhancing Syzkaller Support for NetBSD, Part 2

As a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that supports a variety of operating systems including NetBSD. This report details the work done during the second coding period.

You can also take a look at the first report to learn more about the initial support that we added. : https://blog.netbsd.org/tnf/entry/enhancing_syzkaller_support_for_netbsd

July Update: All about the Pinebook Pro

"So I said I won’t be talking about the BSDs, but I feel like I should at the very least give you a general overview of the RK3399 *BSD functionality. I’ll make it quick. I’ve spoken to *BSD devs whom worked on the RockPro64 and from what I’ve gathered (despite the different *BSDs having varying degree of support for the RK3399 SOC) many of the core features are already supported, which bodes well for *BSD on the Pro. That said, some of the things you’d require on a functional laptop – such as the LCD (using eDP) for instance – will not work on the Pinebook Pro using *BSD as of today. So clearly a degree of work is yet needed for a BSD to run on the device. However, keep in mind that *BSD developers will be receiving their units soon and by the time you receive yours some basic functionality may be available."

Killing a process and all of its descendants

Killing processes in a Unix-like system can be trickier than expected. Last week I was debugging an odd issue related to job stopping on Semaphore. More specifically, an issue related to the killing of a running process in a job. Here are the highlights of what I learned:

Unix-like operating systems have sophisticated process relationships. Parent-child, process groups, sessions, and session leaders. However, the details are not uniform across operating systems like Linux and macOS. POSIX compliant operating systems support sending signals to process groups with a negative PID number.

Sending signals to all processes in a session is not trivial with syscalls.

Child processes started with exec inherit their parent signal configuration. If the parent process is ignoring the SIGHUP signal, for example, this configuration is propagated to the children.

The answer to the “What happens with orphaned process groups” question is not trivial.

Fast Software, the Best Software

I love fast software. That is, software speedy both in function and interface. Software with minimal to no lag between wanting to activate or manipulate something and the thing happening. Lightness.

Software that’s speedy usually means it’s focused. Like a good tool, it often means that it’s simple, but that’s not necessarily true. Speed in software is probably the most valuable, least valued asset. To me, speedy software is the difference between an application smoothly integrating into your life, and one called upon with great reluctance. Fastness in software is like great margins in a book — makes you smile without necessarily knowing why.

But why is slow bad? Fast software is not always good software, but slow software is rarely able to rise to greatness. Fast software gives the user a chance to “meld” with its toolset. That is, not break flow. When the nerds upon Nerd Hill fight to the death over Vi and Emacs, it’s partly because they have such a strong affinity for the flow of the application and its meldiness. They have invested. The Tool Is Good, so they feel. Not breaking flow is an axiom of great tools.

A typewriter is an excellent tool because, even though it’s slow in a relative sense, every aspect of the machine itself operates as quickly as the user can move. It is focused. There are no delays when making a new line or slamming a key into the paper. Yes, you have to put a new sheet of paper into the machine at the end of a page, but that action becomes part of the flow of using the machine, and the accumulation of paper a visual indication of work completed. It is not wasted work. There are no fundamental mechanical delays in using the machine. The best software inches ever closer to the physical directness of something like a typewriter. (The machine may break down, of course, ribbons need to be changed — but this is maintenance and separate from the use of the tool. I’d be delighted to “maintain” Photoshop if it would lighten it up.)

Beastie Bits

• Register for vBSDCon 2019, Sept 5-7 in Reston VA
• Register for EuroBSDCon 2019, Sept 19-22 in Lillehammer, Norway

Feedback/Questions

• Paulo - FreeNAS Question
• Marc - Changing VT without function keys?
• Caleb - Patch, update, and upgrade management

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv