NOTES

This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

The Death Of Industrial Design And The Era Of Dull Electronics

Host Naming Convention

News Roundup

Open, free, and completely ignored: The strange afterlife of Symbian

TIL: timeout in Bash scripts

It seems like NVMe SSDs have overtaken SATA SSDs for high capacities

A system to organise your life

Tarsnap

This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

- Nelson - Books

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

• Join us and other BSD Fans in our BSD Now Telegram channel

NOTES

This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

The Hype is the Product

Programmers Aren’t So Humble Anymore—Maybe Because Nobody Codes in Perl

News Roundup

Is OpenBSD 10x faster than Linux?

How to install FreeBSD on providers that don't support it with mfsBSD

SSHX

Zvault Status Update

Undeadly Bits

• 4096 colours and flashing text on the console!
• Font caching no longer runs as root
• OpenSSH will now adapt IP QoS to actual sessions and traffic

Tarsnap

This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

• Join us and other BSD Fans in our BSD Now Telegram channel

NOTES

This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

Why FreeBSD is the Right Choice for Embedded Devices

The Day GlusterFS Tried to Kill My Career

News Roundup

DragonFly DRM updated

NetBSD on Raspberry Pi!

Speed up suspend/resume for FreeBSD

Revisiting ZFS's ZIL, separate log devices, and writes

One of my blog articles featured on the BSD Now podcast episode!

New build cluster speeds up daily autobuilds

Tarsnap

This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

• Join us and other BSD Fans in our BSD Now Telegram channel

NOTES

This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

The Server That Wasn't Meant to Exist

ZFS Performance Tuning – Optimizing for your Workload

News Roundup

What would a multi-user web server look like? (A thought experiment)

That Grumpy BSD Guy: A Short Reading List

rsync's defaults are not always enough

jemalloc Postmortem

Beastie Bits

• IPv6 and proxying on DragonFly
• BoxyBSD
• Sysctltui

Tarsnap

This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

• Join us and other BSD Fans in our BSD Now Telegram channel

NOTES

This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

A year of funded FreeBSD

ZFS Performance Tuning – Optimizing for your Workload

News Roundup

Three Ways to Try FreeBSD in Under Five Minutes

FFS optimizations with dirhash

j2k25 hackathon report from kn@: installer, low battery, and more

NetBSD welcomes Google Summer of Code contributors

Tarsnap

This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

• Join us and other BSD Fans in our BSD Now Telegram channel

NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

Security, Performance, and Interoperability; Introducing FreeBSD 14

HardenedBSD November 2023 Status Report

News Roundup

How to create a FreeBSD Jail hosting a remote desktop

A sneak Peak

Programming FreeBSD Reading Process Information

Why Unix kernels have grown caches for directory entries 'name caches'

Always learning, Always Teaching

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

• Join us and other BSD Fans in our BSD Now Telegram channel

NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

If you can use Open Source you can build hardware

Good performance is not just big O

News Roundup

How to add pledge to a program in OpenBSD

Proof You Should Not Run My Code

3D printing on OpenBSD? Yes, that’s a thing!

Getting the right type of certificate

Jenny’s Daily Drivers

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

• Join us and other BSD Fans in our BSD Now Telegram channel

  ---

NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

5 Key reasons why you need a OpenZFS Performance Audit

Musings on Mobility : The Ping from Hell

News Roundup

OpenBGPD 7.9 released

Setting the clock ahead to see what breaks

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• Esteban - pot

• Tim - BSD Talk at SCALE

• Fred - Networking

- Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

How To Set Up a Wireguard VPN Server with Unbound on OpenBSD

Auditing for OpenZFS Storage Performance

News Roundup

Some notes on OpenBSD 7.2 on a Thinkpad X201

fzf

• A Practical Guide to fzf: Building a File Explorer
• A Practical Guide to fzf: Shell Integration
• ***

Replacing postfix with dma

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• Dennis - Thanks

• Luna - Trillian

• Lyubomir - ipfw question

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv ***

NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon

Headlines

FreeBSD Quarterly Status Report Third Quarter 2022

Avoid Infrastructure Vendor Lock-in by leveraging MinIO and OpenZFS

Announcing the FreeBSD/Firecracker platform

News Roundup

How Much Faster Is Making A Tar Archive Without Gzip?

PostgreSQL from packages on OpenBSD

Upgrading an NVMe zpool from 222G to 1TB drives

PSA: Don't use Reddit for Linux or BSD related questions

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• Hinnerk - vnet jails
  Tom’s response example: https://adventurist.me/posts/00304

• Hugo - Apple M2

• kevin - emacs backspace
  )

  ---

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

  ---

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

If you like BSDNow, consider supporting us on Patreon

What Makes a Good Time to Use OpenZFS Slog and When Should You Avoid It

OpenBSD 7.0 is out

News Roundup

OpenBSD and Wayland

Unlocking UVM faults yields significant performance boost

Beastie Bits

PLAN 9 DESKTOP GUIDE
libvirt and DragonFly
EuroBSDCon 2021 videos are available
Issue#1 of OpenBSD Webzine
The Beastie has landed.
It’s 1998 and you are Sun Microsystems...

• Reply link that's down RSA/SHA1 signature type disabled by default in OpenSSH *** ###Tarsnap
• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• Dan - IPFS
• Jack - IPFS

• Johnny - AdvanceBSD

  ---

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

  ---

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

OpenZFS 2.1 is out

FreeBSD TCP Performance System Controls

News Roundup

IPFS OpenBSD

Tips for running an online conference

My Fanless OpenBSD Desktop

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• Bruce - Upgrading
• Chris - SMB Followup
• dmilith - kTLS

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv ***

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

FreeBSD Performance Observability

Advance!BSD – thoughts on a not-for-profit project to support *BSD (1/2)

News Roundup

Maintainership Change :: Lumina Desktop Environment

Study the past if you would define the Future

How to Handle Secrets on the Command Line

Following NetBSD, DragonFlyBSD Now Has "COVID"

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• Jim - freebsd kde
• michal - zfs question
• tim - lumina and snapshots ***
• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv ***

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

The Open-Source Software bubble that is and the blogging bubble that was

Building Customized FreeBSD Images

News Roundup

Updating to Minecraft 1.17 in FreeBSD

Upgrading a FreeBSD 12.2 jail to FreeBSD 13 using mkjail

DragonFlyBSD 6.0 Is Performing Very Well Against Ubuntu Linux, FreeBSD 13.0

An OpenBSD Consumer Gateway Launch

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• CY - bearssl
• Marc - that tarsnap ad
• nycbug

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv ***

NOTES
This episode of BSDNow is brought to you by Tarsnap

Headlines

My New Project: zedfs.com

Have you ever had an idea that keeps coming back to you over and over again? For a week? For a month? I know that feeling. My new project was born from this feeling.
On this blog, I mix content a lot. I have written personal posts (not many of them, but still), FreeBSD development posts, development posts, security posts, and ZFS posts. This mixed content can be problematic sometimes. I share a lot of stuff here, and readers don’t know what to expect next. I am just excited by so many things, and I want to share that excitement with you!

---

TrueNAS CORE is Ready for Deployment

TrueNAS 12.0 RC1 was released yesterday and with it, TrueNAS CORE is ready for deployment. The merger of FreeNAS and TrueNAS into a unified software image can now begin its path into mainstream use. TrueNAS CORE is the new FreeNAS and is on schedule.
The TrueNAS 12.0 BETA process started in June and has been the most successful BETA release ever with more than 3,000 users and only minor issues. Ars Technica provided a detailed technical walkthrough of the original BETA. There is a long list of features and performance improvements. During the BETA process, TrueNAS 12.0 demonstrated over 1.2 Million IOPS and over 23GB/s on a TrueNAS M60.

---

News Roundup

Interprocess Communication in FreeBSD 11: Performance Analysis

Interprocess communication, IPC, is one of the most fundamental functions of a modern operating system, playing an essential role in the fabric of contemporary applications. This report conducts an investigation in FreeBSD of the real world performance considerations behind two of the most common IPC mechanisms; pipes and sockets. A simple benchmark provides a fair sense of effective bandwidth for each, and analysis using DTrace, hardware performance counters and the operating system’s source code is presented. We note that pipes outperform sockets by 63% on average across all configurations, and further that the size of userspace transmission buffers has a profound effect on performance — larger buffers are beneficial up to a point (∼ 32-64 KiB) after which performance collapses as a result of devastating cache exhaustion. A deep scrutiny of the probe effects at play is also presented, justifying the validity of conclusions drawn from these experiments.

---

Back To The Future: Unix Wildcards Gone Wild

First of all, this article has nothing to do with modern hacking techniques like ASLR bypass, ROP exploits, 0day remote kernel exploits or Chrome's Chain-14-Different-Bugs-To-Get-There... Nope, nothing of the above. This article will cover one interesting old-school Unix hacking technique, that will still work nowadays in 2013.

---

Unix Wars

Dozens of different operating systems have been developed over the years, but only Unix has grown in so many varieties. There are three main branches. Four factors have facilitated this growth...

---

Tarsnap

• This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.

Feedback/Questions

• Chris - installing FreeBSD 13-current
• Dane - FreeBSD History Lesson
• Marc - linux compat
• Mason - apropos battery

• Paul - a topic idea

  ---

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

  ---

Headlines

Text processing in the shell

This article is part of a self-published book project by Balthazar Rouberol and Etienne Brodu, ex-roommates, friends and colleagues, aiming at empowering the up and coming generation of developers. We currently are hard at work on it!

One of the things that makes the shell an invaluable tool is the amount of available text processing commands, and the ability to easily pipe them into each other to build complex text processing workflows. These commands can make it trivial to perform text and data analysis, convert data between different formats, filter lines, etc.

When working with text data, the philosophy is to break any complex problem you have into a set of smaller ones, and to solve each of them with a specialized tool.

Rebalancing data on ZFS mirrors

One of the questions that comes up time and time again about ZFS is “how can I migrate my data to a pool on a few of my disks, then add the rest of the disks afterward?”

If you just want to get the data moved and don’t care about balance, you can just copy the data over, then add the new disks and be done with it. But, it won’t be distributed evenly over the vdevs in your pool.

Don’t fret, though, it’s actually pretty easy to rebalance mirrors. In the following example, we’ll assume you’ve got four disks in a RAID array on an old machine, and two disks available to copy the data to in the short term.

News Roundup

Using OpenBSD relayd to Add Security Headers

I am a huge fan of OpenBSD’s built-in httpd server as it is simple, secure, and quite performant. With the modern push of the large search providers pushing secure websites, it is now important to add security headers to your website or risk having the search results for your website downgraded. Fortunately, it is very easy to do this when you combine httpd with relayd. While relayd is principally designed for layer 3 redirections and layer 7 relays, it just so happens that it makes a handy tool for adding the recommended security headers. My website automatically redirects users from http to https and this gets achieved using a simple redirection in /etc/httpd.conf So if you have a configuration similar to mine, then you will still want to have httpd listen on the egress interface on port 80. The key thing to change here is to have httpd listen on 127.0.0.1 on port 443.

How we set up our ZFS filesystem hierarchy in our ZFS pools

Our long standing practice here, predating even the first generation of our ZFS fileservers, is that we have two main sorts of filesystems, home directories (homedir filesystems) and what we call 'work directory' (workdir) filesystems. Homedir filesystems are called /h/NNN (for some NNN) and workdir filesystems are called /w/NNN; the NNN is unique across all of the different sorts of filesystems. Users are encouraged to put as much stuff as possible in workdirs and can have as many of them as they want, which mattered a lot more in the days when we used Solaris DiskSuite and had fixed-sized filesystems.

Speeding up ZSH

https://web.archive.org/web/20200315184849/https://blog.jonlu.ca/posts/speeding-up-zsh

I was opening multiple shells for an unrelated project today and noticed how abysmal my shell load speed was. After the initial load it was relatively fast, but the actual shell start up was noticeably slow. I timed it with time and these were the results.

In the future I hope to actually recompile zsh with additional profiling techniques and debug information - keeping an internal timer and having a flag output current time for each command in a tree fashion would make building heat maps really easy.

How do Unix Pipes work

Pipes are cool! We saw how handy they are in a previous blog post. Let’s look at a typical way to use the pipe operator. We have some output, and we want to look at the first lines of the output. Let’s download The Brothers Karamazov by Fyodor Dostoevsky, a fairly long novel.

What we do to enable us to grow our ZFS pools over time

In my entry on why ZFS isn't good at growing and reshaping pools, I mentioned that we go to quite some lengths in our ZFS environment to be able to incrementally expand our pools. Today I want to put together all of the pieces of that in one place to discuss what those lengths are.
Our big constraint is that not only do we need to add space to pools over time, but we have a fairly large number of pools and which pools will have space added to them is unpredictable. We need a solution to pool expansion that leaves us with as much flexibility as possible for as long as possible. This pretty much requires being able to expand pools in relatively small increments of space.

Linux maintains bugs: The real reason ifconfig on Linux is deprecated

In my third installment of FreeBSD vs Linux, I will discuss underlying reasons for why Linux moved away from ifconfig(8) to ip(8).

In the past, when people said, “Linux is a kernel, not an operating system”, I knew that was true but I always thought it was a rather pedantic criticism. Of course no one runs just the Linux kernel, you run a distribution of Linux. But after reviewing userland code, I understand the significant drawbacks to developing “just a kernel” in isolation from the rest of the system.

Clear Your Terminal in Style

if you’re someone like me who habitually clears their terminal, sometimes you want a little excitement in your life. Here is a way to do just that.

This post revolves around the idea of giving a command a percent chance of running. While the topic at hand is not serious, this simple technique has potential in your scripts.

Feedback/Questions

• Guy - AMD GPU Help
• MLShroyer13 - VLANs and Jails
• Master One - ZFS Suspend/resume

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Checksumming in filesystems, and why ZFS is doing it right

One of the best aspects of ZFS is its reliability. This can be accomplished using a few features like copy-on-write approach and checksumming. Today we will look at how ZFS does checksumming and why it does it the proper way. Most of the file systems don’t provide any integrity checking and fail in several scenarios:

• Data bit flips - when the data that we wanted to store are bit flipped by the hard drives, or cables, and the wrong data is stored on the hard drive.
• Misdirected writes - when the CPU/cable/hard drive will bit flip a block to which the data should be written.
• Misdirected read - when we miss reading the block when a bit flip occurred.
• Phantom writes - when the write operation never made it to the disk. For example, a disk or kernel may have some bug that it will return success even if the hard drive never made the write. This problem can also occur when data is kept only in the hard drive cache.

Checksumming may help us detect errors in a few of those situations.

DragonFlyBSD Improves Its TMPFS Implementation For Better Throughput Performance

It's been a while since last having any new magical optimizations to talk about by DragonFlyBSD lead developer Matthew Dillon, but on Wednesday he landed some significant temporary file-system "TMPFS" optimizations for better throughput including with swap.

Of several interesting commits merged tonight, the improved write clustering is a big one. In particular, "Reduces low-memory tmpfs paging I/O overheads by 4x and generally increases paging throughput to SSD-based swap by 2x-4x. Tmpfs is now able to issue a lot more 64KB I/Os when under memory pressure."

• https://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/4eb0bb82efc8ef32c4357cf812891c08d38d8860

There's also a new tunable in the VM space as well as part of his commits on Wednesday night. This follows a lot of recent work on dsynth, improved page-out daemon pipelining, and other routine work.

• https://gitweb.dragonflybsd.org/dragonfly.git/commit/bc47dbc18bf832e4badb41f2fd79159479a7d351

This work is building up towards the eventual DragonFlyBSD 5.8 while those wanting to try the latest improvements right away can find their daily snapshots.

News Roundup

Why ZFS is not good at growing and reshaping pools (or shrinking them)

recently read Mark McBride's Five Years of Btrfs (via), which has a significant discussion of why McBride chose Btrfs over ZFS that boils down to ZFS not being very good at evolving your pool structure. You might doubt this judgment from a Btrfs user, so let me say as both a fan of ZFS and a long term user of it that this is unfortunately quite true; ZFS is not a good choice if you want to modify your pool disk layout significantly over time. ZFS works best if the only change in your pools that you do is replacing drives with bigger drives. In our ZFS environment we go to quite some lengths to be able to expand pools incrementally over time, and while this works it both leaves us with unbalanced pools and means that we're basically forced to use mirroring instead of RAIDZ.

(An unbalanced pool is one where some vdevs and disks have much more data than others. This is less of an issue for us now that we're using SSDs instead of HDs.)

Using PKGSRC on Manjaro Linux aarch64 Pinebook-pro

I wanted to see how pkgsrc works on aarch64 Linux Manjaro since it is a very mature framework that is very portable and supported by many architectures – pkgsrc (package source) is a package management system for Unix-like operating systems. It was forked from the FreeBSD ports collection in 1997 as the primary package management system for NetBSD.

One might question why use pkgsrc on Arch based Manjaro, since the pacman package repository is very good on its own. I see alternative pkgsrc as a good automated build framework that offers a way to produce independent build environment /usr/pkg that does not interfere with the current Linux distribution in any way (all libraries are statically built)

I have used the latest Manjaro for Pinebookpro and standard recommended tools as mentioned here https://wiki.netbsd.org/pkgsrc/how_to_use_pkgsrc_on_linux/

A Central Log Host with syslog-ng on FreeBSD

• Part 1

syslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure.

• Part 2

This blog post continues where the blog post A central log host with syslog-ng on FreeBSD left off. Open source solutions to check syslog log messages exist, such as Logcheck or Logwatch. Although these are not too difficult to implement and maintain, I still found these to much. So I went for my own home grown solution to check the syslog messages of the SoCruel.NU central log host.

Beastie Bits

• FreeBSD at Linux Conf 2020 session videos now online
• Unlock your laptop with your phone
• Managing a database of vulnerabilities for a package system: the pkgsrc study
• Hamilton BSD User group will meet again on March 10th](http://studybsd.com/)
• CharmBUG Meeting: March 24th 7pm in Severn, MD ***

Feedback/Questions

• Andrew - ZFS feature Flags
• Sam - TwinCat BSD
• Dacian - Freebsd + amdgpu + Lenovo E595

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

LLDB Threading support now ready for mainline

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.

So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.

Multiple IPSec VPN tunnels with FreeBSD

The FreeBSD handbook describes an IPSec VPN tunnel between 2 FreeBSD hosts (see https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ipsec.html)

But it is also possible to have multiple, 2 or more, IPSec VPN tunnels created and running on a FreeBSD host. How to implement and configure this is described below.

The requirements is to have 3 locations (A, B and C) connected with IPSec VPN tunnels using FreeBSD (11.3-RELEASE).

Each location has 1 IPSec VPN host running FreeBSD (VPN host A, B and C).

VPN host A has 2 IPSec VPN tunnels: 1 to location B (VPN host B) and 1 to location C (VPN host C).

News Roundup

Netflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance

Drew Gallatin of Netflix presented at the recent EuroBSDcon 2019 conference in Norway on the company's network stack optimizations to FreeBSD. Netflix was working on being able to deliver 200Gb/s network performance for video streaming out of Intel Xeon and AMD EPYC servers, to which they are now at 190Gb/s+ and in the process that doubled the potential of EPYC Naples/Rome servers and also very hefty upgrades too for Intel.

Netflix has long been known to be using FreeBSD in their data centers particularly where network performance is concerned. But in wanting to deliver 200Gb/s throughput from individual servers led them to making NUMA optimizations to the FreeBSD network stack. Allocating NUMA local memory for kernel TLS crypto buffers and for backing files sent via sentfile were among their optimizations. Changes to network connection handling and dealing with incoming connections to Nginx were also made.

For those just wanting the end result, Netflix's NUMA optimizations to FreeBSD resulted in their Intel Xeon servers going from 105Gb/s to 191Gb/s while the NUMA fabric utilization dropped from 40% to 13%.

unwind(8); "happy eyeballs"

In case you are wondering why happy eyeballs: It's a variation on this:
https://en.wikipedia.org/wiki/Happy_Eyeballs

unwind has a concept of a best nameserver type. It considers a configured DoT nameserver to be better than doing it's own recursive resolving. Recursive resolving is considered to be better than asking the dhcp provided nameservers.

This diff sorts the nameserver types by quality, as above (validation, resolving, dead...), and as a tie breaker it adds the median of the round trip time of previous queries into the mix.

One other interesting thing about this is that it gets us past captive portals without a check URL, that's why this diff is so huge, it rips out all the captive portal stuff (please apply with patch -E):
17 files changed, 385 insertions(+), 1683 deletions(-)

Please test this. I'm particularly interested in reports from people who move between networks and need to get past captive portals.

Amazon now has FreeBSD ARM 12

Product Overview

FreeBSD is an operating system used to power servers, desktops, and embedded systems. Derived from BSD, the version of UNIX developed at the University of California, Berkeley, FreeBSD has been continually developed by a large community for more than 30 years.

FreeBSD's networking, security, storage, and monitoring features, including the pf firewall, the Capsicum and CloudABI capability frameworks, the ZFS filesystem, and the DTrace dynamic tracing framework, make FreeBSD the platform of choice for many of the busiest web sites and most pervasive embedded networking and storage systems.

OpenSSH U2F/FIDO support in base

I just committed all the dependencies for OpenSSH security key (U2F) support to base and tweaked OpenSSH to use them directly. This means there will be no additional configuration hoops to jump through to use U2F/FIDO2 security keys.

Hardware backed keys can be generated using "ssh-keygen -t ecdsa-sk" (or "ed25519-sk" if your token supports it). Many tokens require to be touched/tapped to confirm this step.

You'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a "key handle" that is used by the security key to derive the real private key at signing time.

So, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything.

Once you have generated a key, you can use it normally - i.e. add it to an agent, copy it to your destination's authorized_keys files (assuming they are running -current too), etc. At authentication time, you will be prompted to tap your security key to confirm the signature operation - this makes theft-of-access attacks against security keys more difficult too.

Please test this thoroughly - it's a big change that we want to have stable before the next release.

Beastie Bits

• DragonFly - git: virtio - Fix LUN scan issue w/ Google Cloud
• Really fast Markov chains in ~20 lines of sh, grep, cut and awk
• FreeBSD Journal Sept/Oct 2019
• Michael Dexter is raising money for Bhyve development
• syscall call-from verification
• FreeBSD Forums Howto Section

Feedback/Questions

• Jeroen - Feedback
• Savo - pfsense ports
• Tin - I want to learn C

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

FreeBSD and custom firmware on the Google Pixelbook

• FreeBSD and custom firmware on the Google Pixelbook

Back in 2015, I jumped on the ThinkPad bandwagon by getting an X240 to run FreeBSD on. Unlike most people in the ThinkPad crowd, I actually liked the clickpad and didn\u2019t use the trackpoint much. But this summer I\u2019ve decided that it was time for something newer. I wanted something..

• lighter and thinner (ha, turns out this is actually important, I got tired of carrying a T H I C C laptop - Apple was right all along);
• with a 3:2 display (why is Lenovo making these Serious Work\u2122 laptops 16:9 in the first place?? 16:9 is awful in below-13-inch sizes especially);
• with a HiDPI display (and ideally with a good size for exact 2x scaling instead of fractional);
• with USB-C ports;
• without a dGPU, especially without an NVIDIA GPU;
• assembled with screws and not glue (I don\u2019t necessarily need expansion and stuff in a laptop all that much, but being able to replace the battery without dealing with a glued chassis is good);
• supported by FreeBSD of course (\u201csome development required\u201d is okay but I\u2019m not going to write big drivers);
• how about something with open source firmware, that would be fun.

I was considering a ThinkPad X1 Carbon from an old generation - the one from the same year as the X230 is corebootable, so that\u2019s fun. But going back in processor generations just doesn\u2019t feel great. I want something more efficient, not less!

And then I discovered the Pixelbook. Other than the big huge large bezels around the screen, I liked everything about it. Thin aluminum design, a 3:2 HiDPI screen, rubber palm rests (why isn\u2019t every laptop ever doing that?!), the \u201cconvertibleness\u201d (flip the screen around to turn it into.. something rather big for a tablet, but it is useful actually), a Wacom touchscreen that supports a pen, mostly reasonable hardware (Intel Wi-Fi), and that famous coreboot support (Chromebooks\u2019 stock firmware is coreboot + depthcharge).

So here it is, my new laptop, a Google Pixelbook.

• Conclusion

Pixelbook, FreeBSD, coreboot, EDK2 good.

Seriously, I have no big words to say, other than just recommending this laptop to FOSS enthusiasts :)

Porting NetBSD to the AMD x86-64: a case study in OS portability

• Abstract

NetBSD is known as a very portable operating system, currently running on 44 different architectures (12 different types of CPU). This paper takes a look at what has been done to make it portable, and how this has decreased the amount of effort needed to port NetBSD to a new architecture. The new AMD x86-64 architecture, of which the specifications were published at the end of 2000, with hardware to follow in 2002, is used as an example.

• Portability

Supporting multiple platforms was a primary goal of the NetBSD project from the start. As NetBSD was ported to more and more platforms, the NetBSD kernel code was adapted to become more portable along the way.

• General

Generally, code is shared between ports as much as possible. In NetBSD, it should always be considered if the code can be assumed to be useful on other architectures, present or future. If so, it is machine-independent and put it in an appropriate place in the source tree. When writing code that is intended to be machine-independent, and it contains conditional preprocessor statements depending on the architecture, then the code is likely wrong, or an extra abstraction layer is needed to get rid of these statements.

• Types

Assumptions about the size of any type are not made. Assumptions made about type sizes on 32-bit platforms were a large problem when 64-bit platforms came around. Most of the problems of this kind had to be dealt with when NetBSD was ported to the DEC Alpha in 1994. A variation on this problem had to be dealt with with the UltraSPARC (sparc64) port in 1998, which is 64-bit, but big endian (vs. the little-endianness of the Alpha). When interacting with datastructures of a fixed size, such as on-disk metadata for filesystems, or datastructures directly interpreted by device hardware, explicitly sized types are used, such as uint32_t, int8_t, etc.

• Conclusions and future work

The port of NetBSD to AMD's x86-64 architecture was done in six weeks, which confirms NetBSD's reputation as being a very portable operating system. One week was spent setting up the cross-toolchain and reading the x86-64 specifications, three weeks were spent writing the kernel code, one week was spent writing the userspace code, and one week testing and debugging it all. No problems were observed in any of the machine-independent parts of the kernel during test runs; all (simulated) device drivers, file systems, etc, worked without modification.

News Roundup

ZFS performance really does degrade as you approach quota limits

Every so often (currently monthly), there is an "OpenZFS leadership meeting". What this really means is 'lead developers from the various ZFS implementations get together to talk about things'. Announcements and meeting notes from these meetings get sent out to various mailing lists, including the ZFS on Linux ones.

• In the September meeting notes, I read a very interesting (to me) agenda item:
  • Relax quota semantics for improved performance (Allan Jude)
  • Problem: As you approach quotas, ZFS performance degrades.
  • Proposal: Can we have a property like quota-policy=strict or loose, where we can optionally allow ZFS to run over the quota as long as performance is not decreased.

This is very interesting to me because of two reasons. First, in the past we have definitely seen significant problems on our OmniOS machines, both when an entire pool hits a quota limit and when a single filesystem hits a refquota limit. It's nice to know that this wasn't just our imagination and that there is a real issue here. Even better, it might someday be improved (and perhaps in a way that we can use at least some of the time).

Second, any number of people here run very close to and sometimes at the quota limits of both filesystems and pools, fundamentally because people aren't willing to buy more space. We have in the past assumed that this was relatively harmless and would only make people run out of space. If this is a known issue that causes serious performance degradation, well, I don't know if there's anything we can do, but at least we're going to have to think about it and maybe push harder at people. The first step will have to be learning the details of what's going on at the ZFS level to cause the slowdown. (It's apparently similar to what happens when the pool is almost full, but I don't know the specifics of that either.)

With that said, we don't seem to have seen clear adverse effects on our Linux fileservers, and they've definitely run into quota limits (repeatedly). One possible reason for this is that having lots of RAM and SSDs makes the effects mostly go away. Another possible reason is that we haven't been looking closely enough to see that we're experiencing global slowdowns that correlate to filesystems hitting quota limits. We've had issues before with somewhat subtle slowdowns that we didn't understand (cf), so I can't discount that we're having it happen again.

Fixing up KA9Q-unix, or "neck deep in 30 year old codebases.."

I'll preface this by saying - yes, I'm still neck deep in FreeBSD's wifi stack and 802.11ac support, but it turns out it's slow work to fix 15 year old locking related issues that worked fine on 11abg cards, kinda worked ok on 11n cards, and are terrible for these 11ac cards. I'll .. get there.

Anyhoo, I've finally been mucking around with AX.25 packet radio. I've been wanting to do this since I was a teenager and found out about its existence, but back in high school and .. well, until a few years ago really .. I didn't have my amateur radio licence. But, now I do, and I've done a bunch of other stuff with a bunch of other radios. The main stumbling block? All my devices are either Apple products or run FreeBSD - and none of them have useful AX.25 stacks. The main stacks of choice these days run on Linux, Windows or are a full hardware TNC.

So yes, I was avoiding hacking on AX.25 stuff because there wasn't a BSD compatible AX.25 stack. I'm 40 now, leave me be.

But! A few weeks ago I found that someone was still running a packet BBS out of San Francisco. And amazingly, his local node ran on FreeBSD! It turns out Jeremy (KK6JJJ) ported both an old copy of KA9Q and N0ARY-BBS to run on FreeBSD! Cool!

I grabbed my 2m radio (which is already cabled up for digital modes), compiled up his KA9Q port, figured out how to get it to speak to Direwolf, and .. ok. Well, it worked. Kinda.

HAMMER2 and fsck for review

HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there\u2019s now a fsck command, useful if you want a report of data validity rather than any manual repair process.

[The return of startx(1) for non-root users with some caveats

Mark Kettenis (kettenis@) has recently committed changes which restore a certain amount of startx(1)/xinit(1) functionality for non-root users. The commit messages explain the situation:

CVSROOT:    /cvs
Module name:    src
Changes by:    kettenis@cvs.openbsd.org    2019/09/15 06:25:41

Modified files:
    etc/etc.amd64  : fbtab 
    etc/etc.arm64  : fbtab 
    etc/etc.hppa   : fbtab 
    etc/etc.i386   : fbtab 
    etc/etc.loongson: fbtab 
    etc/etc.luna88k: fbtab 
    etc/etc.macppc : fbtab 
    etc/etc.octeon : fbtab 
    etc/etc.sgi    : fbtab 
    etc/etc.sparc64: fbtab 

Log message:
Add ttyC4 to lost of devices to change when logging in on ttyC0 (and in some cases also the serial console) such that X can use it as its VT when running without root privileges.

ok jsg@, matthieu@
CVSROOT:    /cvs
Module name:    xenocara
Changes by:    kettenis@cvs.openbsd.org    2019/09/15 06:31:08

Modified files:
    xserver/hw/xfree86/common: xf86AutoConfig.c 

Log message:
Add modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus.

This makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4).  In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)).

ok jsg@, matthieu@

Beastie Bits

• ASCII table and history. Or, why does Ctrl+i insert a Tab in my terminal?
• Sourcehut makes BSD software better
• Chaosnet for Unx
• The Vim-Inspired Editor with a Linguistic Twist
• bhyvearm64: CPU and Memory Virtualization on Armv8.0-A
• DefCon25 - Are all BSD created Equally - A Survey of BSD Kernel vulnerabilities

Feedback/Questions

• Tim - GSoC project ideas for pf rule syntax translation
• Brad - Steam on FreeBSD
• Ruslan - FreeBSD Quarterly Status Report - Q2 2019

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Headlines
###Six Metrics for Measuring ZFS Pool Performance Part 1

The layout of a ZFS storage pool has a significant impact on system performance under various workloads. Given the importance of picking the right configuration for your workload and the fact that making changes to an in-use ZFS pool is far from trivial, it is important for an administrator to understand the mechanics of pool performance when designing a storage system.

• To quantify pool performance, we will consider six primary metrics:
• Read I/O operations per second (IOPS)
• Write IOPS
• Streaming read speed
• Streaming write speed
• Storage space efficiency (usable capacity after parity versus total raw capacity)
• Fault tolerance (maximum number of drives that can fail before data loss)
• For the sake of comparison, we’ll use an example system with 12 drives, each one sized at 6TB, and say that each drive does 100MB/s streaming reads and writes and can do 250 read and write IOPS. We will visualize how the data is spread across the drives by writing 12 multi-colored blocks, shown below. The blocks are written to the pool starting with the brown block on the left (number one), and working our way to the pink block on the right (number 12).

Note that when we calculate data rates and IOPS values for the example system, they are only approximations. Many other factors can impact pool access speeds for better (compression, caching) or worse (poor CPU performance, not enough memory).
There is no single configuration that maximizes all six metrics. Like so many things in life, our objective is to find an appropriate balance of the metrics to match a target workload. For example, a cold-storage backup system will likely want a pool configuration that emphasizes usable storage space and fault tolerance over the other data-rate focused metrics.
Let’s start with a quick review of ZFS storage pools before diving into specific configuration options. ZFS storage pools are comprised of one or more virtual devices, or vdevs. Each vdev is comprised of one or more storage providers, typically physical hard disks. All disk-level redundancy is configured at the vdev level. That is, the RAID layout is set on each vdev as opposed to on the storage pool. Data written to the storage pool is then striped across all the vdevs. Because pool data is striped across the vdevs, the loss of any one vdev means total pool failure. This is perhaps the single most important fact to keep in mind when designing a ZFS storage system. We will circle back to this point in the next post, but keep it in mind as we go through the vdev configuration options.
Because storage pools are made up of one or more vdevs with the pool data striped over the top, we’ll take a look at pool configuration in terms of various vdev configurations. There are three basic vdev configurations: striping, mirroring, and RAIDZ (which itself has three different varieties). The first section will cover striped and mirrored vdevs in this post; the second post will cover RAIDZ and some example scenarios.
A striped vdev is the simplest configuration. Each vdev consists of a single disk with no redundancy. When several of these single-disk, striped vdevs are combined into a single storage pool, the total usable storage space would be the sum of all the drives. When you write data to a pool made of striped vdevs, the data is broken into small chunks called “blocks” and distributed across all the disks in the pool. The blocks are written in “round-robin” sequence, meaning after all the disks receive one row of blocks, called a stripe, it loops back around and writes another stripe under the first. A striped pool has excellent performance and storage space efficiency, but absolutely zero fault tolerance. If even a single drive in the pool fails, the entire pool will fail and all data stored on that pool will be lost.
The excellent performance of a striped pool comes from the fact that all of the disks can work independently for all read and write operations. If you have a bunch of small read or write operations (IOPS), each disk can work independently to fetch the next block. For streaming reads and writes, each disk can fetch the next block in line synchronized with its neighbors. For example, if a given disk is fetching block n, its neighbor to the left can be fetching block n-1, and its neighbor to the right can be fetching block n+1. Therefore, the speed of all read and write operations as well as the quantity of read and write operations (IOPS) on a striped pool will scale with the number of vdevs. Note here that I said the speeds and IOPS scale with the number of vdevs rather than the number of drives; there’s a reason for this and we’ll cover it in the next post when we discuss RAID-Z.
Here’s a summary of the total pool performance (where N is the number of disks in the pool):

• N-wide striped:
• Read IOPS: N * Read IOPS of a single drive
• Write IOPS: N * Write IOPS of a single drive
• Streaming read speed: N * Streaming read speed of a single drive
• Streaming write speed: N * Streaming write speed of a single drive
• Storage space efficiency: 100%
• Fault tolerance: None!

Let’s apply this to our example system, configured with a 12-wide striped pool:

• 12-wide striped:
• Read IOPS: 3000
• Write IOPS: 3000
• Streaming read speed: 1200 MB/s
• Streaming write speed: 1200 MB/s
• Storage space efficiency: 72 TB
• Fault tolerance: None!
• Below is a visual depiction of our 12 rainbow blocks written to this pool configuration:

The blocks are simply striped across the 12 disks in the pool. The LBA column on the left stands for “Logical Block Address”. If we treat each disk as a column in an array, each LBA would be a row. It’s also easy to see that if any single disk fails, we would be missing a color in the rainbow and our data would be incomplete. While this configuration has fantastic read and write speeds and can handle a ton of IOPS, the data stored on the pool is very vulnerable. This configuration is not recommended unless you’re comfortable losing all of your pool’s data whenever any single drive fails.
A mirrored vdev consists of two or more disks. A mirrored vdev stores an exact copy of all the data written to it on each one of its drives. Traditional RAID-1 mirrors usually only support two drive mirrors, but ZFS allows for more drives per mirror to increase redundancy and fault tolerance. All disks in a mirrored vdev have to fail for the vdev, and thus the whole pool, to fail. Total storage space will be equal to the size of a single drive in the vdev. If you’re using mismatched drive sizes in your mirrors, the total size will be that of the smallest drive in the mirror.
Streaming read speeds and read IOPS on a mirrored vdev will be faster than write speeds and IOPS. When reading from a mirrored vdev, the drives can “divide and conquer” the operations, similar to what we saw above in the striped pool. This is because each drive in the mirror has an identical copy of the data. For write operations, all of the drives need to write a copy of the data, so the mirrored vdev will be limited to the streaming write speed and IOPS of a single disk.

Here’s a summary:

• N-way mirror:

• Read IOPS: N * Read IOPS of a single drive

• Write IOPS: Write IOPS of a single drive

• Streaming read speed: N * Streaming read speed of a single drive

• Streaming write speed: Streaming write speed of a single drive

• Storage space efficiency: 50% for 2-way, 33% for 3-way, 25% for 4-way, etc. [(N-1)/N]

• Fault tolerance: 1 disk per vdev for 2-way, 2 for 3-way, 3 for 4-way, etc. [N-1]

• For our first example configuration, let’s do something ridiculous and create a 12-way mirror. ZFS supports this kind of thing, but your management probably will not.

• 1x 12-way mirror:

• Read IOPS: 3000

• Write IOPS: 250

• Streaming read speed: 1200 MB/s

• Streaming write speed: 100 MB/s

• Storage space efficiency: 8.3% (6 TB)

• Fault tolerance: 11

As we can clearly see from the diagram, every single disk in the vdev gets a full copy of our rainbow data. The chainlink icons between the disk labels in the column headers indicate the disks are part of a single vdev. We can lose up to 11 disks in this vdev and still have a complete rainbow. Of course, the data takes up far too much room on the pool, occupying a full 12 LBAs in the data array.

Obviously, this is far from the best use of 12 drives. Let’s do something a little more practical and configure the pool with the ZFS equivalent of RAID-10. We’ll configure six 2-way mirror vdevs. ZFS will stripe the data across all 6 of the vdevs. We can use the work we did in the striped vdev section to determine how the pool as a whole will behave. Let’s first calculate the performance per vdev, then we can work on the full pool:

• 1x 2-way mirror:

• Read IOPS: 500

• Write IOPS: 250

• Streaming read speed: 200 MB/s

• Streaming write speed: 100 MB/s

• Storage space efficiency: 50% (6 TB)

• Fault tolerance: 1

• Now we can pretend we have 6 drives with the performance statistics listed above and run them through our striped vdev performance calculator to get the total pool’s performance:

• 6x 2-way mirror:

• Read IOPS: 3000

• Write IOPS: 1500

• Streaming read speed: 3000 MB/s

• Streaming write speed: 1500 MB/s

• Storage space efficiency: 50% (36 TB)

• Fault tolerance: 1 per vdev, 6 total

• Again, we will examine the configuration from a visual perspective:

Each vdev gets a block of data and ZFS writes that data to all of (or in this case, both of) the disks in the mirror. As long as we have at least one functional disk in each vdev, we can retrieve our rainbow. As before, the chain link icons denote the disks are part of a single vdev. This configuration emphasizes performance over raw capacity but doesn’t totally disregard fault tolerance as our striped pool did. It’s a very popular configuration for systems that need a lot of fast I/O. Let’s look at one more example configuration using four 3-way mirrors. We’ll skip the individual vdev performance calculation and go straight to the full pool:

• 4x 3-way mirror:
• Read IOPS: 3000
• Write IOPS: 1000
• Streaming read speed: 3000 MB/s
• Streaming write speed: 400 MB/s
• Storage space efficiency: 33% (24 TB)
• Fault tolerance: 2 per vdev, 8 total

While we have sacrificed some write performance and capacity, the pool is now extremely fault tolerant. This configuration is probably not practical for most applications and it would make more sense to use lower fault tolerance and set up an offsite backup system.
Striped and mirrored vdevs are fantastic for access speed performance, but they either leave you with no redundancy whatsoever or impose at least a 50% penalty on the total usable space of your pool. In the next post, we will cover RAIDZ, which lets you keep data redundancy without sacrificing as much storage space efficiency. We’ll also look at some example workload scenarios and decide which layout would be the best fit for each.

###2FA with ssh on OpenBSD

Five years ago I wrote about using a yubikey on OpenBSD. The only problem with doing this is that there’s no validation server available on OpenBSD, so you need to use a different OTP slot for each machine. (You don’t want to risk a replay attack if someone succeeds in capturing an OTP on one machine, right?) Yubikey has two OTP slots per device, so you would need a yubikey for every two machines with which you’d like to use it. You could use a bastion—and use only one yubikey—but I don’t like the SPOF aspect of a bastion. YMMV.
After I played with TOTP, I wanted to use them as a 2FA for ssh. At the time of writing, we can’t do that using only the tools in base. This article focuses on OpenBSD; if you use another operating system, here are two handy links.

• SEED CONFIGURATION

The first thing we need to do is to install the software which will be used to verify the OTPs we submit.

# pkg_add login_oath

We need to create a secret - aka, the seed - that will be used to calculate the Time-based One-Time Passwords. We should make sure no one can read or change it.

$ openssl rand -hex 20 > ~/.totp-key
$ chmod 400 ~/.totp-key

Now we have a hexadecimal key, but apps usually want a base32 secret. I initially wrote a small script to do the conversion.
While writing this article, I took the opportunity to improve it. When I initially wrote this utility for my use, python-qrcode hadn’t yet been imported to the OpenBSD ports/packages system. It’s easy to install now, so let’s use it.
Here’s the improved version. It will ask for the hex key and output the secret as a base32-encoded string, both with and without spacing so you can copy-paste it into your password manager or easily retype it. It will then ask for the information needed to generate a QR code. Adding our new OTP secret to any mobile app using the QR code will be super easy!

• SYSTEM CONFIGURATION

We can now move to the configuration of the system to put our new TOTP to use. As you might guess, it’s going to be quite close to what we did with the yubikey.
We need to tweak login.conf. Be careful and keep a root shell open at all times. The few times I broke my OpenBSD were because I messed with login.conf without showing enough care.

• SSHD CONFIGURATION

Again, keeping a root shell around decreases the risk of losing access to the system and being locked outside.
A good standard is to use PasswordAuthentication no and to use public key only. Except… have a guess what the P stands for in TOTP. Yes, congrats, you guessed it!
We need to switch to PasswordAuthentication yes. However, if we made this change alone, sshd would then accept a public key OR a password (which are TOTP because of our login.conf). 2FA uses both at the same time.
To inform sshd we intend to use both, we need to set AuthenticationMethods publickey,password. This way, the user trying to login will first need to perform the traditional publickey authentication. Once that’s done, ssh will prompt for a password and the user will need to submit a valid TOTP for the system.
We could do this the other way around, but I think bots could try passwords, wasting resources. Evaluated in this order, failing to provide a public key leads to sshd immediately declining your attempt.

• IMPROVING SECURITY WITHOUT IMPACTING UX

My phone has a long enough password that most of the time, I fail to type it correctly on the first try. Of course, if I had to unlock my phone, launch my TOTP app and use my keyboard to enter what I see on my phone’s screen, I would quickly disable 2FA.
To find a balance, I have whitelisted certain IP addresses and users. If I connect from a particular IP address or as a specific user, I don’t want to go through 2FA. For some users, I might not even enable 2FA.
To sum up, we covered how to create a seed, how to perform a hexadecimal to base32 conversion and how to create a QR code for mobile applications. We configured the login system with login.conf so that ssh authentication uses the TOTP login system, and we told sshd to ask for both the public key and the Time-based One-Time Password. Now you should be all set to use two-factor ssh authentication on OpenBSD!

##News Roundup
###How ZFS maintains file type information in directories

As an aside in yesterday’s history of file type information being available in Unix directories, I mentioned that it was possible for a filesystem to support this even though its Unix didn’t. By supporting it, I mean that the filesystem maintains this information in its on disk format for directories, even though the rest of the kernel will never ask for it. This is what ZFS does.
The easiest way to see that ZFS does this is to use zdb to dump a directory. I’m going to do this on an OmniOS machine, to make it more convincing, and it turns out that this has some interesting results. Since this is OmniOS, we don’t have the convenience of just naming a directory in zdb, so let’s find the root directory of a filesystem, starting from dnode 1 (as seen before).

# zdb -dddd fs3-corestaff-01/h/281 1
Dataset [....]
[...]
microzap: 512 bytes, 4 entries
[...]
ROOT = 3

# zdb -dddd fs3-corestaff-01/h/281 3
Object lvl iblk dblk dsize lsize %full type
3 1 16K 1K 8K 1K 100.00 ZFS directory
[...]
microzap: 1024 bytes, 8 entries

RESTORED = 4396504 (type: Directory)
ckstst = 12017 (type: not specified)
ckstst3 = 25069 (type: Directory)
.demo-file = 5832188 (type: Regular File)
.peergroup = 12590 (type: not specified)
cks = 5 (type: not specified)
cksimap1 = 5247832 (type: Directory)
.diskuse = 12016 (type: not specified)
ckstst2 = 12535 (type: not specified)

This is actually an old filesystem (it dates from Solaris 10 and has been transferred around with ‘zfs send | zfs recv’ since then), but various home directories for real and test users have been created in it over time (you can probably guess which one is the oldest one). Sufficiently old directories and files have no file type information, but more recent ones have this information, including .demo-file, which I made just now so this would have an entry that was a regular file with type information.
Once I dug into it, this turned out to be a change introduced (or activated) in ZFS filesystem version 2, which is described in ‘zfs upgrade -v’ as ‘enhanced directory entries’. As an actual change in (Open)Solaris, it dates from mid 2007, although I’m not sure what Solaris release it made it into. The upshot is that if you made your ZFS filesystem any time in the last decade, you’ll have this file type information in your directories.
How ZFS stores this file type information is interesting and clever, especially when it comes to backwards compatibility. I’ll start by quoting the comment from zfs_znode.h:

/*
* The directory entry has the type (currently unused on
* Solaris) in the top 4 bits, and the object number in
* the low 48 bits. The "middle" 12 bits are unused.
*/

In yesterday’s entry I said that Unix directory entries need to store at least the filename and the inode number of the file. What ZFS is doing here is reusing the 64 bit field used for the ‘inode’ (the ZFS dnode number) to also store the file type, because it knows that object numbers have only a limited range. This also makes old directory entries compatible, by making type 0 (all 4 bits 0) mean ‘not specified’. Since old directory entries only stored the object number and the object number is 48 bits or less, the higher bits are guaranteed to be all zero.
The reason this needed a new ZFS filesystem version is now clear. If you tried to read directory entries with file type information on a version of ZFS that didn’t know about them, the old version would likely see crazy (and non-existent) object numbers and nothing would work. In order to even read a ‘file type in directory entries’ filesystem, you need to know to only look at the low 48 bits of the object number field in directory entries.

###Everything old is new again

Just because KDE4-era software has been deprecated by the KDE-FreeBSD team in the official ports-repository, doesn’t mean we don’t care for it while we still need to. KDE4 was released on January 11th, 2008 — I still have the T-shirt — which was a very different C++ world than what we now live in. Much of the code pre-dates the availability of C11 — certainly the availability of compilers with C11 support. The language has changed a great deal in those ten years since the original release.
The platforms we run KDE code on have, too — FreeBSD 12 is a long way from the FreeBSD 6 or 7 that were current at release (although at the time, I was more into OpenSolaris). In particular, since then the FreeBSD world has switched over to Clang, and FreeBSD current is experimenting with Clang 7. So we’re seeing KDE4-era code being built, and running, on FreeBSD 12 with Clang 7. That’s a platform with a very different idea of what constitutes correct code, than what the code was originally written for. (Not quite as big a difference as Helio’s KDE1 efforts, though)
So, while we’re counting down to removing KDE4 from the FreeBSD ports tree, we’re also going through and fixing it to work with Clang 7, which defaults to a newer C++ standard and which is quite picky about some things. Some time in the distant past, when pointers were integers and NULL was zero, there was some confusion about booleans. So there’s lots of code that does list.contains(element) > 0 … this must have been a trick before booleans were a supported type in all our compilers. In any case it breaks with Clang 7, since contains() returns a QBool which converts to a nullptr (when false) which isn’t comparable to the integer 0. Suffice to say I’ve spent more time reading KDE4-era code this month, than in the past two years.
However, work is proceeding apace, so if you really really want to, you can still get your old-school kicks on a new platform. Because we care about packaging things right, even when we want to get rid of it.

###OpenBSD netcat demystified

Owing to its versatile functionalities, netcat earns the reputation as “TCP/IP Swiss army knife”. For example, you can create a simple chat app using netcat:

• (1) Open a terminal and input following command:

# nc -l 3003

This means a netcat process will listen on 3003 port in this machine (the IP address of current machine is 192.168.35.176).

• (2) Connect aforemontioned netcat process in another machine, and send a greeting:

# nc 192.168.35.176 3003
hello

Then in the first machine’s terminal, you will see the “hello” text:

# nc -l 3003
hello

A primitive chatroom is built successfully. Very cool! Isn’t it? I think many people can’t wait to explore more features of netcatnow. If you are among them, congratulations! This tutorial may be the correct place for you.
In the following parts, I will delve into OpenBSD’s netcatcode to give a detailed anatomy of it. The reason of picking OpenBSD’s netcat rather than others’ is because its code repository is small (~2000 lines of code) and neat. Furthermore, I also hope this little book can assist you learn more socket programming knowledge not just grasping usage of netcat.
We’re all set. Let’s go!

##Beastie Bits

• What’s in store for NetBSD 9.0
• NetBSD machines at Open Source Conference 2018 Hiroshima
• nmctl adapted with limited privileges: nmctl-0.6.0
• Submit Your Work: Check out SCALE 17x and FOSDEM ’19 CFPs
• OpenBSD 6.4 site is up! (with a partial list of new features)
• Using Alpine to Read Your Email on OpenBSD

##Feedback/Questions

• Morgan - Send/Receive to Manage Fragmentation?
• Ryan - ZFS and mmap
• Marcus - Linux Compat
• Ben - Multiple Pools

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Headlines
##Interview - Michael W. Lucas - mwlucas@michaelwlucas.com / @mwlauthor

• BR: [Welcome Back]
• AJ: What have you been doing since last we talked to you [ed, ssh, and af3e]
• BR: Tell us more about AF3e
• AJ: How did the first Absolute FreeBSD come about?
• BR: Do you have anything special planned for MeetBSD?
• AJ: What are you working on now? [FM:Jails, Git sync Murder]
• BR: What are your plans for next year?
• AJ: How has SEMIBug been going?

Auction at https://mwl.io
Patreon Link:

##Feedback/Questions

• Paul - Recent bhyve related videos (daemon)
• Michael - freebsd-update question
• Sigflup - pkg file search

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

##Headlines
###Essen Hackathon & BSDCam 2018 trip report

• Allan and Benedict met at FRA airport and then headed to the Air Rail terminal for our train to Essen where the Hackathon would happen over the weekend of Aug 10 - 12, 2018. Once there, we did not have to wait long until other early-arrivals would show up and soon we had about 10 people gathered for lunch. After buying some take-out pizzas and bringing it back to the Linuxhotel (there was a training still going on there so we could not get into our rooms yet), we sat in the sunny park and talked. More and more people arrived and soon, people started hacking on their laptops. Some people would not arrive until a few hours before midnight, but we already had a record appearance of 20 people in total.
• On Saturday, we gathered everyone in one of the seminar rooms that had rooms and chairs for us. After some organizational infos, we did an introductory round and Benedict wrote down on the whiteboard what people were interested in. It was not long until groups formed to talk about SSL in base, weird ZFS scrubs that would go over 100% completion (fixed now). Other people started working on ports, fixing bugs, or wrote documentation. The day ended in a BBQ in the Linuxhotel park, which was well received by everyone.
• On Sunday, after attendees packed up their luggage and stored it in the seminar room, we continued hacking until lunchtime. After a quick group picture, we headed to a local restaurant for the social event (which was not open on Saturday, otherwise we would have had it then). In the afternoon, most people departed, a good half of them were heading for BSDCam.
• Commits from the hackathon (the ones from 2018)
• Overall, the hackathon was well received by attendees and a lot of them liked the fact that it was close to another BSD gathering so they could nicely combine the two. Also, people thought about doing their own hackathon in the future, which is an exciting prospect. Thanks to all who attended, helped out here and there when needed. Special Thanks to Netzkommune GmbH for sponsoring the social event and the Linuxhotel for having us.
• Benedict was having a regular work day on Monday after coming back from the hackathon, but flew out to Heathrow on Tuesday. Allan was in London a day earlier and arrived a couple of hours before Benedict in Cambridge. He headed for the Computer Lab even though the main event would not start until Wednesday. Most people gathered at the Maypole pub on Tuesday evening for welcomes, food and drinks.
• On Wednesday, a lot of people met in the breakfast room of Churchill College where most people were staying and went to the Computer Lab, which served as the main venue for BSDCam, together. The morning was spend with introductions and collecting what most people were interested in talking. This unconference style has worked well in the past and soon we had 10 main sessions together for the rest of this and the following two days (full schedule).
• Most sessions took notes, which you can find on the FreeBSD wiki.
• On Thursday evening, we had a nice formal dinner at Trinity Hall.
• BSDCam 2018 was a great success with a lot of fruitful discussions and planning sessions. We thank the organizers for BSDCam for making it happen.
• A special mentions goes out to Robert Watson and his family. Even though he was not there, he had a good reason to miss it: they had their first child born at the beginning of the week. Congratulations and best wishes to all three of them!

###Call for Testing: ZFS Native Encryption for FreeBSD

• A port of the ZoL (ZFS-on-Linux) feature that provides native crypto support for ZFS is ready for testing on FreeBSD
• Most of the porting was done by sef@freebsd.org (Sean Eric Fagan)
• The original ZoL commit is here: https://github.com/zfsonlinux/zfs/pull/5769/commits/5aef9bedc801830264428c64cd2242d1b786fd49
• For an overview, see Tom Caputi’s presentation from the OpenZFS Developers Summit in 2016
• Video: https://youtu.be/frnLiXclAMo
• Slides: https://drive.google.com/file/d/0B5hUzsxe4cdmU3ZTRXNxa2JIaDQ/view?usp=sharing
• WARNING: test in VMs or with spare disks etc, pools created with this code, or upgraded to this version, will no longer be importable on systems that do not support this feature. The on-disk format or other things may change before the final version, so you will likely have to ‘zfs send | zfs recv’ the data on to a new pool
• Thanks for testing to help this feature land in FreeBSD

iXsystems

###Call for Testing: UFS TRIM Consolidation

• Kirk Mckusick posts to the FreeBSD mailing list looking for testers for the new UFS TRIM Consolidation code

When deleting files on filesystems that are stored on flash-memory (solid-state) disk drives, the filesystem notifies the underlying disk of the blocks that it is no longer using. The notification allows the drive to avoid saving these blocks when it needs to flash (zero out) one of its flash pages. These notifications of no-longer-being-used blocks are referred to as TRIM notifications. In FreeBSD these TRIM notifications are sent from the filesystem to the drive using the BIO_DELETE command.
Until now, the filesystem would send a separate message to the drive for each block of the file that was deleted. Each Gigabyte of file size resulted in over 3000 TRIM messages being sent to the drive. This burst of messages can overwhelm the drive’s task queue causing multiple second delays for read and write requests.
This implementation collects runs of contiguous blocks in the file and then consolodates them into a single BIO_DELETE command to the drive. The BIO_DELETE command describes the run of blocks as a single large block being deleted. Each Gigabyte of file size can result in as few as two BIO_DELETE commands and is typically less than ten. Though these larger BIO_DELETE commands take longer to run, they do not clog the drive task queue, so read and write commands can intersperse effectively with them.
Though this new feature has been throughly reviewed and tested, it is being added disabled by default so as to minimize the possibility of disrupting the upcoming 12.0 release. It can be enabled by running ``sysctl vfs.ffs.dotrimcons=1’’. Users are encouraged to test it. If no problems arise, we will consider requesting that it be enabled by default for 12.0.
This support is off by default, but I am hoping that I can get enough testing to ensure that it (a) works, and (b) is helpful that it will be reasonable to have it turned on by default in 12.0. The cutoff for turning it on by default in 12.0 is September 19th. So I am requesting your testing feedback in the near-term. Please let me know if you have managed to use it successfully (or not) and also if it provided any performance difference (good or bad).

• To enable TRIM consolidation use `sysctl vfs.ffs.dotrimcons=1’
• There is also a diff that adds additional statistics: https://lists.freebsd.org/pipermail/freebsd-current/2018-August/070798.html
• You can also watch the volume and latency of BIO_DELETE commands by running gstat with the -d flag

##News Roundup
###ZFS performance

• Aravindh Sampathkumar, a Performance Engineer and Sysadmin posts some simple benchmarks he did on a new ZFS server

This is NOT an all-in post about ZFS performance. I built a FreeBSD+ZFS file server recently at work to serve as an offsite backup server. I wanted to run a few synthetic workloads on it and look at how it fares from performance perspective. Mostly for curiosity and learning purposes.
As stated in the notes about building this server, performance was not one of the priorities, as this server will never face our active workload. What I care about from this server is its ability to work with rsync and keep the data synchronised with our primary storage server. With that context, I ran a few write tests to see how good our solution is and what to expect from it in terms of performance.

• The article then uses FIO to do some benchmarks.
• As the author did, make sure you match the FIO block size to the ZFS record size to avoid write amplification. Either tune FIO or adjust the recordsize property in ZFS
• You also want to consider compression and cache effects

Write Performance: Incompressible: 1600-2600 MB/s, Compressible: 2500-6600 MB/s
Another over 1200 MB/s is enough to keep your 10 gigabit network saturated

• The increased latency that is seen with higher number of writers working, may be the result of the ZFS backpressure system (the write throttle). There is some tuning that can be done there. Specifically, since this machine has 768 GB of ram, you might allow more than 4GB of dirty data, which would mean you’d be able to write larger batches and not have to push back while you wait for a transaction group to flush when dealing with gigabytes/sec of writes

###How to port your OS to EC2

• Colin Percival reflects on his FreeBSD on EC2 maintainership efforts in his blog:

I’ve been the maintainer of the FreeBSD/EC2 platform for about 7.5 years now, and as far as “running things in virtual machines” goes, that remains the only operating system and the only cloud which I work on. That said, from time to time I get questions from people who want to port other operating systems into EC2, and being a member of the open source community, I do my best to help them. I realized a few days ago that rather than replying to emails one by one it would be more efficient to post something publicly; so — for the benefit of the dozen or so people who want to port operating systems to run in EC2, and the curiosity of maybe a thousand more people who use EC2 but will never build AMIs themselves — here’s a rough guide to building EC2 images.
Before we can talk about building images, there are some things you need:
Your OS needs to run on x86 hardware. 64-bit (“amd64”, “x86-64”) is ideal, but I’ve managed to run 32-bit FreeBSD on “64-bit” EC2 instances so at least in some cases that’s not strictly necessary.
You almost certainly want to have drivers for Xen block devices (for all of the pre-Nitro EC2 instances) or for NVMe disks (for the most recent EC2 instances). Theoretically you could make do without these since there’s some ATA emulation available for bootstrapping, but if you want to do any disk I/O after the kernel finishes booting you’ll want to have a disk driver.
Similarly, you need support for the Xen network interface (older instances), Intel 10 GbE SR-IOV networking (some newer but pre-Nitro instances), or Amazon’s “ENA” network adapters (on Nitro instances), unless you plan on having instances which don’t communicate over the network. The ENA driver is probably the hardest thing to port, since as far as I know there’s no way to get your hands on the hardware directly, and it’s very difficult to do any debugging in EC2 without having a working network.
Finally, the obvious: You need to have an AWS account, and appropriate API access keys.
Building a disk image

Building an AMI
I wrote a simple tool for converting disk images into EC2 instances: bsdec2-image-upload. It uploads a disk image to Amazon S3; makes an API call to import that disk image into an EBS volume; creates a snapshot of that volume; then registers an EC2 AMI using that snapshot.
To use bsdec2-image-upload, you’ll first need to create an S3 bucket for it to use as a staging area. You can call it anything you like, but I recommend that you

Create it in a “nearby” region (for performance reasons), and
Set an S3 “lifecycle policy” which deletes objects automatically after 1 day (since bsdec2-image-upload doesn’t clean up the S3 bucket, and those objects are useless once you’ve finished creating an AMI).

Boot configuration
Odds are that your instance started booting and got as far as the boot loader launching the kernel, but at some point after that things went sideways. Now we start the iterative process of building disk images, turning them into AMIs, launching said AMIs, and seeing where they break. Some things you’ll probably run into here:
EC2 instances have two types of console available to them: A serial console and an VGA console. (Or rather, emulated serial and emulated VGA.) If you can have your kernel output go to both consoles, I recommend doing that. If you have to pick one, the serial console (which shows up as the “System Log” in EC2) is probably more useful than the VGA console (which shows up as “instance screenshot”) since it lets you see more than one screen of logs at once; but there’s a catch: Due to some bizarre breakage in EC2 — which I’ve been complaining about for ten years — the serial console is very “laggy”. If you find that you’re not getting any output, wait five minutes and try again.
You may need to tell your kernel where to find the root filesystem. On FreeBSD we build our disk images using GPT labels, so we simply need to specify in /etc/fstab that the root filesystem is on /dev/gpt/rootfs; but if you can’t do this, you’ll probably need to have different AMIs for Nitro instances vs. non-Nitro instances since Xen block devices will typically show up with different device names from NVMe disks. On FreeBSD, I also needed to set the vfs.root.mountfrom kernel environment variable for a while; this also is no longer needed on FreeBSD but something similar may be needed on other systems.
You’ll need to enable networking, using DHCP. On FreeBSD, this means placing ifconfig_DEFAULT=“SYNCDHCP” into /etc/rc.conf; other systems will have other ways of specifying network parameters, and it may be necessary to specify a setting for the Xen network device, Intel SR-IOV network, and the Amazon ENA interface so that you’ll have the necessary configuration across all EC2 instance types. (On FreeBSD, ifconfig_DEFAULT takes care of specifying the network settings which should apply for whatever network interface the kernel finds at boot time.)
You’ll almost certainly want to turn on SSH, so that you can connect into newly launched instances and make use of them. Don’t worry about setting a password or creating a user to SSH into yet — we’ll take care of that later.
EC2 configuration
Now it’s time to make the AMI behave like an EC2 instance. To this end, I prepared a set of rc.d scripts for FreeBSD. Most importantly, they
Print the SSH host keys to the console, so that you can veriy that they are correct when you first SSH in. (Remember, Verifying SSH host keys is more important than flossing every day.)
Download the SSH public key you want to use for logging in, and create an account (by default, “ec2-user”) with that key set up for you.
Fetch EC2 user-data and process it via configinit to allow you to configure the system as part of the process of launching it.
If your OS has an rc system derived from NetBSD’s rc.d, you may be able to use these scripts without any changes by simply installing them and enabling them in /etc/rc.conf; otherwise you may need to write your own scripts using mine as a model.
Firstboot scripts
A feature I added to FreeBSD a few years ago is the concept of “firstboot” scripts: These startup scripts are only run the first time a system boots. The aforementioned configinit and SSH key fetching scripts are flagged this way — so if your OS doesn’t support the “firstboot” keyword on rc.d scripts you’ll need to hack around that — but EC2 instances also ship with other scripts set to run on the first boot:
FreeBSD Update will fetch and install security and critical errata updates, and then reboot the system if necessary.
The UFS filesystem on the “boot disk” will be automatically expanded to the full size of the disk — this makes it possible to specify a larger size of disk at EC2 instance launch time.
Third-party packages will be automatically fetched and installed, according to a list in /etc/rc.conf. This is most useful if configinit is used to edit /etc/rc.conf, since it allows you to specify packages to install via the EC2 user-data.
While none of these are strictly necessary, I find them to be extremely useful and highly recommend implementing similar functionality in your systems.
Support my work!
I hope you find this useful, or at very least interesting. Please consider supporting my work in this area; while I’m happy to contribute my time to supporting open source software, it would be nice if I had money coming in which I could use to cover incidental expenses (e.g., conference travel) so that I didn’t end up paying to contribute to FreeBSD.

Digital Ocean
https://do.co/bsdnow

###Traceability, by Vint Cerf

• A recent article from the August issue of the Communications of the ACM, for your contemplation:

At a recent workshop on cybersecurity in the U.K., a primary topic of consideration was how to preserve the freedom and openness of the Internet while protecting against the harmful behaviors that have emerged in this global medium. That this is a significant challenge cannot be overstated. The bad behaviors range from social network bullying and misinformation to email spam, distributed denial of service attacks, direct cyberattacks against infrastructure, malware propagation, identity theft, and a host of other ills requiring a wide range of technical and legal considerations. That these harmful behaviors can and do cross international boundaries only makes it more difficult to fashion effective responses.
In other columns, I have argued for better software development tools to reduce the common mistakes that lead to vulnerabilities that are exploited. Here, I want to focus on another aspect of response related to law enforcement and tracking down perpetrators. Of course, not all harms are (or perhaps are not yet) illegal, but discovering those who cause them may still be warranted. The recent adoption and implementation of the General Data Protection Regulation (GDPR) in the European Union creates an interesting tension because it highlights the importance and value of privacy while those who do direct or indirect harm must be tracked down and their identities discovered.
In passing, I mention that cryptography has sometimes been blamed for protecting the identity or actions of criminals but it is also a tool for protecting privacy. Arguments have been made for “back doors” to cryptographic systems but I am of the opinion that such proposals carry extremely high risk to privacy and safety. It is not my intent to argue this question in this column.
What is of interest to me is a concept to which I was introduced at the Ditchley workshop, specifically, differential traceability. The ability to trace bad actors to bring them to justice seems to me an important goal in a civilized society. The tension with privacy protection leads to the idea that only under appropriate conditions can privacy be violated. By way of example, consider license plates on cars. They are usually arbitrary identifiers and special authority is needed to match them with the car owners (unless, of course, they are vanity plates like mine: “Cerfsup”). This is an example of differential traceability; the police department has the authority to demand ownership information from the Department of Motor Vehicles that issues the license plates. Ordinary citizens do not have this authority.
In the Internet environment there are a variety of identifiers associated with users (including corporate users). Domain names, IP addresses, email addresses, and public cryptography keys are examples among many others. Some of these identifiers are dynamic and thus ambiguous. For example, IP addresses are not always permanent and may change (for example, temporary IP addresses assigned at Wi-Fi hotspots) or may be ambiguous in the case of Network Address Translation. Information about the time of assignment and the party to whom an IP address was assigned may be needed to identify an individual user. There has been considerable debate and even a recent court case regarding requirements to register users in domain name WHOIS databases in the context of the adoption of GDPR. If we are to accomplish the simultaneous objectives of protecting privacy while apprehending those engaged in harmful or criminal behavior on the Internet, we must find some balance between conflicting but desirable outcomes.
This suggests to me that the notion of traceability under (internationally?) agreed circumstances (that is, differential traceability) might be a fruitful concept to explore. In most societies today, it is accepted that we must be identifiable to appropriate authorities under certain conditions (consider border crossings, traffic violation stops as examples). While there are conditions under which apparent anonymity is desirable and even justifiable (whistle-blowing, for example) absolute anonymity is actually quite difficult to achieve (another point made at the Ditchley workshop) and might not be absolutely desirable given the misbehaviors apparent anonymity invites. I expect this is a controversial conclusion and I look forward to subsequent discussion.

###Remote Access Console using FreeBSD on an RPi3

• Our friend, and FOSDEM Booth Neighbour, Jorge, has posted a tutorial on how he created a remote access console for his SmartOS server and other machines in his homelab
• Parts:
• Raspberry Pi 3 B+
• NavoLabs micro POE Hat
• FT4232H based USB-to-RS232 (4x) adapter
• Official Raspberry Pi case (optional)
• Heat-sink kit (optional)
• USB-to-TTL adaptor (optional)
• Sandisk 16Gb microSD

For the software I ended up using conserver. Below is a very brief tutorial on how to set everything up. I assume you have basic unix skills.

• Get an RPi3 image, make some minor modifications for RPi3+, and write it to the USB stick
• Configure FreeBSD on the RPi3
  • Load the ‘muge’ Ethernet Driver
  • Load USB serial support
  • Load the FTDI driver
  • Enable SSHd and Conserver
  • Configure Conserver
  • Setup log rotation
  • Start Conserver



• And you’re good to go

A small bonus script I wrote to turn on the 2nd LED on the rPI once the system is booted, it will then blink the LED if someone is connected to any of the consoles.

• There is also a followup post with some additional tips: https://blackdot.be/2018/08/freebsd-uart-and-raspberry-pi-3-b/

##Beastie Bits

• Annual Penguin Races
• Mscgen - Message Sequence Chart generator
• This patch makes FreeBSD boot 500 - 800ms faster, please test on your hardware
• FreeBSD’s arc4random() replaced with OpenBSD ChaCha20 implementation
• MeetBSD Devsummit open for registrations
• New Podcast interview with Michael W. Lucas

Tarsnap

##Feedback/Questions
We need more feedback emails. Please write to feedback@bsdnow.tv

Additionally, we are considering a new segment to be added to the end of the show (to make it skippable), where we have a ~15 minute deep dive on a topic. Some initial ideas are on the Virtual Memory subsystem, the Scheduler, Capsicum, and GEOM. What topics would you like to get very detailed explanations of? Many of the explanations may have accompanying graphics, and not be very suitable for audio only listeners, that is why we are planning to put it at the very end of the episode.

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

This episode was brought to you by

Interview - Bryan Cantrill - bryan@joyent.com / @bcantrill

BSD and Solaris history, illumos, dtrace, Joyent, pkgsrc, various topics (and rants)

Feedback/Questions

• Randy writes in
• Jared writes in
• Steve writes in ***

This episode was brought to you by

Interview - Sepherosa Ziehau - sephe@dragonflybsd.org

Features of DragonFlyBSD's network stack

Discussion

Comparing containment methods and privilege separation

• chroot, jails, systrace, capsicum, filesystem permissions, separating users ***

Feedback/Questions

• Brad writes in
• Anonymous writes in
• Benjamin writes in
• Jeroen writes in ***

This episode was brought to you by

Headlines

Even more BSD presentation videos

• More videos from this year's MeetBSD and OpenZFS devsummit were uploaded since last week
• Robert Ryan, At the Heart of the Digital Economy
• FreeNAS & ZFS, The Indestructible Duo - Except for the Hard Drives
• Richard Yao, libzfs_core and ioctl stabilization
• OpenZFS, Company lightning talks
• OpenZFS, Hackathon Presentation and Awards
• Pavel Zakharov, Fast File Cloning
• Rick Reed, Half a billion unsuspecting FreeBSD users
• Alex Reece & Matt Ahrens, Device Removal
• Chris Side, Channel Programs
• David Maxwell, The Unix command pipeline
• Be sure to check out the giant list of videos from last week's episode if you haven't seen them already ***

NetBSD on a Cobalt Qube 2

• The Cobalt Qube was a very expensive networking appliance around 2000
• In 2014, you can apparently get one of these MIPS-based machines for about forty bucks
• This blog post details getting NetBSD installed and set up on the rare relic of our networking past
• If you're an old-time fan of RISC or MIPS CPUs, this'll be a treat for you
• Lots of great pictures of the hardware too ***

OpenBSD vs. AFL

• In their never-ending security audit, some OpenBSD developers have been hitting various parts of the tree with a fuzzer
• If you're not familiar, fuzzing is a semi-automated way to test programs for crashes and potential security problems
• The program being subjected to torture gets all sorts of random and invalid input, in the hopes of uncovering overflows and other bugs
• American Fuzzy Lop, in particular, has provided some interesting results across various open source projects recently
• So far, it's fixed some NULL pointer dereferences in OpenSSH, various crashes in tcpdump and mandoc and a few other things
• AFL has an impressive list of CVEs (vulnerabilities) that it's helped developers discover and fix
• It also made its way into OpenBSD ports, FreeBSD ports and NetBSD's pkgsrc very recently, so you can try it out for yourself ***

GNOME 3 hits the FreeBSD ports tree

• While you've been able to run GNOME 3 on PC-BSD and OpenBSD for a while, it hasn't actually hit the FreeBSD ports tree.. until now
• Now you can play with GNOME 3 and all its goodies (as well as Cinnamon 2.2, which this also brings in) on vanilla FreeBSD
• Be sure to check the commit message and /usr/ports/UPDATING if you're upgrading from GNOME 2
• You might also want to go back and listen to our interview with Joe Marcus Clark about GNOME's portability ***

Interview - Brendan Gregg - bgregg@netflix.com / @brendangregg

Performance tuning, benchmarks, debugging

News Roundup

DragonFlyBSD 4.0 released

• A new major version of DragonFly, 4.0.1, was just recently announced
• This version includes support for Haswell GPUs, lots of SMP improvements (including some in PF) and support for up to 256 CPUs
• It's also the first release to drop support for i386, so it joins PCBSD in the 64 bit-only club
• Check the release notes for all the details, including networking and kernel improvements, as well as some crypto changes ***

Can we talk about FreeBSD vs Linux

• Hackernews had a recent thread about discussing Linux vs BSD, and the trolls stayed away for once
• Rather than rehashing why one is "better" than the other, it was focused on explaining some of the differences between ecosystems and communities
• If you're one of the many people who watch our show just out of curiosity about the BSD world, this might be a good thread to read
• Someone in the comments even gave bsdnow.tv a mention as a good resource to learn, thanks guy ***

OpenBSD IPSEC tunnel guide

• If you've ever wanted to connect two networks with OpenBSD gateways, this is the article for you
• It shows how to set up an IPSEC tunnel between destinations, how to lock it down and how to access all the machines on the other network just like they were on your LAN
• The article also explains some of the basics of IPSEC if you're not familiar with all the terminology, so this isn't just for experts
• Though the article itself is a few years old, it mostly still applies to the latest stuff today
• All the tools used are in the OpenBSD base system, so that's pretty handy too ***

DragonFly starts work on IPFW2

• DragonFlyBSD, much like FreeBSD, comes with more than one firewall you can use
• Now it looks like you're going to have yet another choice, as someone is working on a fork of IPFW (which is actually already in its second version, so it should be "IPFW3")
• Not a whole lot is known yet; it's still in heavy development, but there's a brief roadmap page with some planned additions
• The guy who's working on this has already agreed to come on the show for an interview, but we're going to give him a chance to get some more work done first
• Expect that sometime next year, once he's made some progress ***

Feedback/Questions

• Michael writes in
• Samael writes in
• Steven writes in
• Remy writes in
• Michael writes in ***

This episode was brought to you by

Headlines

pfSense 2.1.4 released

• The pfSense team has released 2.1.4, shortly after 2.1.3 - it's mainly a security release
• Included within are eight security fixes, most of which are pfSense-specific
• OpenSSL, the WebUI and some packages all need to be patched (and there are instructions on how to do so)
• It also includes a large number of various other bug fixes
• Update all your routers! ***

DragonflyBSD's pf gets SMP

• While we're on the topic of pf...
• Dragonfly patches their old[er than even FreeBSD's] pf to support multithreading in many areas
• Stemming from a user's complaint, Matthew Dillon did his own work on pf to make it SMP-aware
• Altering your configuration's ruleset can also help speed things up, he found
• When will OpenBSD, the source of pf, finally do the same? ***

ChaCha usage and deployment

• A while back, we talked to djm about some cryptography changes in OpenBSD 5.5 and OpenSSH 6.5
• This article is sort of an interesting follow-up to that, showing which projects have adopted ChaCha20
• OpenSSH offers it as a stream cipher now, OpenBSD uses it for it's random number generator, Google offers it in TLS for Chromium and some of their services and lots of other projects seem to be adopting it
• Both Google's fork of OpenSSL and LibReSSL have upcoming implementations, while vanilla OpenSSL does not
• Unfortunately, this article has one mistake: FreeBSD does not use it - they still use the broken RC4 algorithm ***

BSDMag June 2014 issue

• The monthly online BSD magazine releases their newest issue
• This one includes the following articles: TLS hardening, setting up a package cluster in MidnightBSD, more GIMP tutorials, "saving time and headaches using the robot framework for testing," an interview and an article about the increasing number of security vulnerabilities
• The free pdf file is available for download as always ***

Interview - Craig Rodrigues - rodrigc@freebsd.org

FreeBSD's continuous testing infrastructure

Tutorial

Creating pre-patched OpenBSD ISOs

News Roundup

Preauthenticated decryption considered harmful

• Responding to a post from Adam Langley, Ted Unangst talks a little more about how signify and pkg_add handle signatures
• In the past, the OpenBSD installer would pipe the output of ftp straight to tar, but then verify the SHA256 at the end - this had the advantage of not requiring any extra disk space, but raised some security concerns
• With signify, now everything is fully downloaded and verified before tar is even invoked
• The pkg_add utility works a little bit differently, but it's also been improved in this area - details in the post
• Be sure to also read the original post from Adam, lots of good information ***

FreeBSD 9.3-RC2 is out

• As the -RELEASE inches closer, release candidate 2 is out and ready for testing
• Since the last one, it's got some fixes for NIC drivers, the latest file and libmagic security fixes, some serial port workarounds and various other small things
• The updated bsdconfig will use pkgng style packages now too
• A lesser known fact: there are also premade virtual machine images you can use too ***

pkgsrcCon 2014 wrap-up

• In what may be the first real pkgsrcCon article we've ever had!
• Includes wrap-up discussion about the event, the talks, the speakers themselves, what they use pkgsrc for, the hackathon and basically the whole event
• Unfortunately no recordings to be found... ***

PostgreSQL FreeBSD performance and scalability

• FreeBSD developer kib@ writes a report on PostgreSQL on FreeBSD, and how it scales
• On his monster 40-core box with 1TB of RAM, he runs lots of benchmarks and posts the findings
• Lots of technical details if you're interested in getting the best performance out of your hardware
• It also includes specific kernel options he used and the rest of the configuration
• If you don't want to open the pdf file, you can use this link too ***

Feedback/Questions

• James writes in
• Klemen writes in
• John writes in
• Brad writes in
• Adam writes in ***

This episode was brought to you by

Headlines

FreeBSD 10 as a firewall

• Back in 2012, the author of this site wrote an article stating you should avoid FreeBSD 9 for a firewall and use OpenBSD instead
• Now, with the release of 10.0, he's apparently changed his mind and switched back over
• It mentions the SMP version of pf, general performance advantages and more modern features
• The author is a regular listener of BSD Now, hi Joe! ***

Network Noise Reduction Using Free Tools

• Really long blog post, based on a BSDCan presentation, about fighting spam with OpenBSD
• Peter Hansteen, author of the book of PF, goes through how he uses OpenBSD's spamd and other security features to combat spam and malware
• He goes through his experiences with content filtering and disappointment with a certain proprietary vendor
• Not totally BSD-specific, lots of people can enjoy the article - lots of virus history as well ***

FreeBSD ASLR patches submitted

• So far, FreeBSD hasn't had Address Space Layout Randomization
• ASLR is a nice security feature, see wikipedia for more information
• With a giant patch from Shawn Webb, it might be integrated into a future version (after a vicious review from the security team of course)
• We might have Shawn on the show to talk about it, but he's also giving a presentation at BSDCan about his work with ASLR ***

Old-style pkg_ tools retired

• At last the old pkg_add tools are being retired in FreeBSD
• pkgng is a huge improvement, and now portmgr@ thinks it's time to cut the cord on the legacy toolset
• Ports aren't going away, and probably never will, but for binary package fans and new users that are used to things like apt, pkgng is the way to go
• All pkg_ tools will be considered unsupported on September 1, 2014 - even on older branches ***

Interview - Luke Marsden - luke@hybridcluster.com / @lmarsden

BSD at HybridCluster

Tutorial

Filesharing with chrooted SFTP

News Roundup

FreeBSD on OpenStack

• OpenStack is a cloud computing project
• It consists of "a series of interrelated projects that control pools of processing, storage, and networking resources throughout a datacenter, able to be managed or provisioned through a web-based dashboard, command-line tools, or a RESTful API."
• Until now, there wasn't a good way to run a full BSD instance on OpenStack
• With a project in the vein of Colin Percival's AWS startup scripts, now that's no longer the case! ***

FOSDEM BSD videos

• This year's FOSDEM had seven BSD presentations
• The videos are slowly being uploaded for your viewing pleasure
• Not all of the BSD ones are up yet, but by the time you're watching this they might be!
• Check this directory for most of 'em
• The BSD dev room was full, lots of interest in what's going on from the other communities ***

The FreeBSD challenge finally returns!

• Due to prodding from a certain guy of a certain podcast, the "FreeBSD Challenge" series has finally resumed
• Our friend from the Linux foundation picks up with day 11 and day 12 on his switching from Linux journey
• This time he outlines the upgrade process of going from 9 to 10, using freebsd-update
• There's also some notes about different options for upgrading ports and some extra tips ***

PCBSD weekly digest

• After the big 10.0 release, the PCBSD crew is focusing on bug fixes for a while
• During their "fine tuning phase" users are encouraged to submit any and all bugs via the trac system
• Warden got some fixes and the package manager got some updates as well
• Huge size reduction in PBI format ***

Feedback/Questions

• Derrick writes in
• Sean writes in
• Patrick writes in
• Peter writes in
• Sean writes in ***

This episode was brought to you by

Headlines

FreeBSD 10.0-RELEASE is out

• The long awaited, giant release of FreeBSD is now official and ready to be downloaded
• One of the biggest releases in FreeBSD history, with tons of new updates
• Some features include: LDNS/Unbound replacing BIND, Clang by default (no GCC anymore), native Raspberry Pi support and other ARM improvements, bhyve, hyper-v support, AMD KMS, VirtIO, Xen PVHVM in GENERIC, lots of driver updates, ZFS on root in the installer, SMP patches to pf that drastically improve performance, Netmap support, pkgng by default, wireless stack improvements, a new iSCSI stack, FUSE in the base system... the list goes on and on
• Start up your freebsd-update or do a source-based upgrade ***

OpenSSH 6.5 CFT

• Our buddy Damien Miller announced a Call For Testing for OpenSSH 6.5
• Huge, huge release, focused on new features rather than bugfixes (but it includes those too)
• New ciphers, new key formats, new config options, see the mailing list for all the details
• Should be in OpenBSD 5.5 in May, look forward to it - but also help test on other platforms! ***

DIY NAS story, FreeNAS 9.2.1-BETA

• Another new blog post about FreeNAS!
• Instead of updating the older tutorials, the author started fresh and wrote a new one for 2014
• "I did briefly consider suggesting nas4free for the EconoNAS blog, since it’s essentially a fork off the FreeNAS tree but may run better on slower hardware, but ultimately I couldn’t recommend anything other than FreeNAS"
• Really long article with lots of nice details about his setup, why you might want a NAS, etc.
• Speaking of FreeNAS, they released 9.2.1-BETA with lots of bugfixes ***

OpenBSD needed funding for electricity.. and they got it

• Briefly mentioned at the end of last week's show, but has blown up over the internet since
• OpenBSD in the headlines of major tech news sites: slashdot, zdnet, the register, hacker news, reddit, twitter.. thousands of comments
• They needed about $20,000 to cover electric costs for the server rack in Theo's basement
• Lots of positive reaction from the community helping out so far, and it appears they have reached their goal and got $100,000 in donations
• From Bob Beck: "we have in one week gone from being in a dire situation to having a commitment of approximately $100,000 in donations to the foundation"
• This is a shining example of the BSD community coming together, and even the Linux people realizing how critical BSD is to the world at large ***

Interview - Colin Percival - cperciva@freebsd.org / @cperciva

FreeBSD on Amazon EC2, backups with Tarsnap, 10.0-RELEASE, various topics

Tutorial

Bandwidth monitoring and testing

News Roundup

pfSense talk at Tokyo FreeBSD Benkyoukai

• Isaac Levy will be presenting "pfSense Practical Experiences: from home routers, to High-Availability Datacenter Deployments"
• He's also going to be looking for help to translate the pfSense documentation into Japanese
• The event is on February 17, 2014 if you're in the Tokyo area ***

m0n0wall 1.8.1 released

• For those who don't know, m0n0wall is an older BSD-based firewall OS that's mostly focused on embedded applications
• pfSense was forked from it in 2004, and has a lot more active development now
• They switched to FreeBSD 8.4 for this new version
• Full list of updates in the changelog
• This version requires at least 128MB RAM and a disk/CF size of 32MB or more, oh no! ***

Ansible and PF, plus NTP

• Another blog post from our buddy Michael Lucas
• There've been some NTP amplification attacks recently in the news
• The post describes how he configured ntpd on a lot of servers without a lot of work
• He leverages pf and ansible for the configuration
• OpenNTPD is, not surprisingly, unaffected - use it ***

ruBSD videos online

• Just a quick followup from a few weeks ago
• Theo and Henning's talks from ruBSD are now available for download
• There's also a nice interview with Theo ***

PCBSD weekly digest

• 10.0-RC4 images are available
• Wine PBI is now available for 10
• 9.2 systems will now be able to upgrade to version 10 and keep their PBI library ***

Feedback/Questions

• Sha'ul writes in
• Kjell-Aleksander writes in
• Mike writes in
• Charlie writes in (and gets a reply)
• Kevin writes in ***

This episode was brought to you by

Headlines

FreeBSD's new testing infrastructure

• A new test suite was added to FreeBSD, with 3 powerful machines available
• Both -CURRENT and stable/10 have got the test suite build infrastructure in place
• Designed to help developers test and improve major scalability across huge amounts of CPUs and RAM
• More details available here
• Could the iXsystems monster server be involved...? ***

OpenBSD gets signify

• At long last, OpenBSD gets support for signed releases!
• For "the world's most secure OS" it was very easy to MITM kernel patches, updates, installer isos, everything
• A commit to the -current tree reveals a new "signify" tool is currently being kicked around
• More details in a blog post from the guy who committed it
• Quote: "yeah, briefly, the plan is to sign sets and packages. that's still work in progress." ***

Faces of FreeBSD

• This time they interview Isabell Long
• She's a volunteer staff member on the freenode IRC network
• In 2011, she participated in the Google Code-In contest and became involved with documentation
• "The new committer mentoring process proved very useful and that, plus the accepting community of FreeBSD, are reasons why I stay involved." ***

pkgsrc-2013Q4 branched

• The quarterly pkgsrc branch from NetBSD is out
• 13472 total packages for NetBSD-current/amd64 + 13049 binary packages built with clang!
• Lots of numbers and stats in the announcement
• pkgsrc works on quite a few different OSes, not just NetBSD
• See our interview with Amitai Schlair for a bit about pkgsrc ***

OpenBSD on Google's Compute Engine

• Google Compute Engine is a "cloud computing" platform similar to EC2
• Unfortunately, they only offer poor choices for the OS (Debian and CentOS)
• Recently it's been announced that there is a custom OS option
• It's using a WIP virtio-scsi driver, lots of things still need more work
• Lots of technical and networking details about the struggles to get OpenBSD working on it ***

The Installfest

We'll be showing you the installer of each of the main BSDs. As of the date this episode airs, we're using:

• FreeBSD 10.0
• OpenBSD 5.4
• NetBSD 6.1.2
• DragonflyBSD 3.6
• PCBSD 10.0 ***

News Roundup

Building an OpenBSD wireless access point

• A neat write up we found around the internet about making an OpenBSD wifi router
• Goes through the process of PXE booting, installing base, using a serial console, setting up networking and wireless
• Even includes a puffy sticker on the Soekris box at the end, how cute ***

FreeBSD 4.X jails on 10.0

• Blog entry from our buddy Michael Lucas
• For whatever reason (an "in-house application"), he needed to run a FreeBSD 4 jail in FreeBSD 10
• Talks about the options he had: porting software, virtualizing, dealing with slow old hardware
• He goes through the whole process of making an ancient jail
• It's "an acceptable trade-off, if it means I don’t have to touch actual PHP code." ***

Unscrewed: a story about OpenBSD

• Pretty long blog post about how a network admin used OpenBSD to save the day
• To set the tone, "It was 5am, and the network was down"
• Great war story about replacing expensive routers and networking equipment with cheaper hardware and BSD
• Mentions a lot of the built in tools and how OpenBSD is great for routers and high security applications ***

PCBSD weekly digest

• 10.0-RC3 is out and ready to be tested
• New detection of ATI Hybrid Graphics, they're working on nVidia next
• Re-classifying Linux jails as unsupported / experimental ***

Feedback/Questions

• Daniel writes in
• Erik writes in
• SW writes in
• [Bostjan writes in[(http://slexy.org/view/s20N9bfkum)
• Samuel writes in ***