Headlines

EKCD - Encrypted Crash Dumps in FreeBSD

Some time ago, I was describing how to configure networking crash dumps. In that post, I mentioned that there is also the possibility to encrypt crash dumps. Today we will look into this functionality. Initially, it was implemented during Google Summer of Code 2013 by my friend Konrad Witaszczyk, who made it available in FreeBSD 12. If you can understand Polish, you can also look into his presentation on BSD-PL on which he gave a comprehensive review of all kernel crash dumps features.

The main issue with crash dumps is that they may include sensitive information available in memory during a crash. They will contain all the data from the kernel and the userland, like passwords, private keys, etc. While dumping them, they are written to unencrypted storage, so if somebody took out the hard drive, they could access sensitive data. If you are sending a crash dump through the network, it may be captured by third parties. Locally the data are written directly to a dump device, skipping the GEOM subsystem. The purpose of that is to allow a kernel to write a crash dump even in case a panic occurs in the GEOM subsystem. It means that a crash dump cannot be automatically encrypted with GELI.

Time on Unix

Time, a word that is entangled in everything in our lives, something we’re intimately familiar with. Keeping track of it is important for many activities we do.

Over millennia we’ve developed different ways to calculate it. Most prominently, we’ve relied on the position the sun appears to be at in the sky, what is called apparent solar time.

We’ve decided to split it as seasons pass, counting one full cycle of the 4 seasons as a year, a full rotation around the sun. We’ve also divided the passing of light to the lack thereof as days, a rotation of the earth on itself. Moving on to more precise clock divisions such as seconds, minutes, and hours, units that meant different things at different points in history. Ultimately, as travel got faster, the different ways of counting time that evolved in multiple places had to converge. People had to agree on what it all meant.

See the article for more

News Roundup

Improve ZVOL sync write performance by using a taskq

A central log host with syslog-ng on FreeBSD - Part 1

syslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure.

HEADS UP: NetBSD Entropy Overhaul

This week I committed an overhaul of the kernel entropy system. Please let me know if you observe any snags! For the technical background, see the thread on tech-kern a few months ago: https://mail-index.NetBSD.org/tech-kern/2019/12/21/msg025876.html.

Setting Up NetBSD Kernel Dev Environment

I used T_PAGEFLT’s blog post as a reference for setting my NetBSD kernel development environment since his website is down I’m putting down the steps here so it would be helpful for starters.

Beastie Bits

• You can now use ccache to speed up dsynth even more.
• Improving libossaudio, and the future of OSS in NetBSD
• DragonFlyBSD DHCPCD Import dhcpcd-9.0.2 with the following changes
• Reminder: watch this space for upcoming FreeBSD Office Hours, next is May 13th at 2pm Eastern, 18:00 UTC

Feedback/Questions

• Ghislain - ZFS Question
• Jake - Paypal Donations
• Oswin - Hammer tutorial

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Headlines

Checksumming in filesystems, and why ZFS is doing it right

One of the best aspects of ZFS is its reliability. This can be accomplished using a few features like copy-on-write approach and checksumming. Today we will look at how ZFS does checksumming and why it does it the proper way. Most of the file systems don’t provide any integrity checking and fail in several scenarios:

• Data bit flips - when the data that we wanted to store are bit flipped by the hard drives, or cables, and the wrong data is stored on the hard drive.
• Misdirected writes - when the CPU/cable/hard drive will bit flip a block to which the data should be written.
• Misdirected read - when we miss reading the block when a bit flip occurred.
• Phantom writes - when the write operation never made it to the disk. For example, a disk or kernel may have some bug that it will return success even if the hard drive never made the write. This problem can also occur when data is kept only in the hard drive cache.

Checksumming may help us detect errors in a few of those situations.

DragonFlyBSD Improves Its TMPFS Implementation For Better Throughput Performance

It's been a while since last having any new magical optimizations to talk about by DragonFlyBSD lead developer Matthew Dillon, but on Wednesday he landed some significant temporary file-system "TMPFS" optimizations for better throughput including with swap.

Of several interesting commits merged tonight, the improved write clustering is a big one. In particular, "Reduces low-memory tmpfs paging I/O overheads by 4x and generally increases paging throughput to SSD-based swap by 2x-4x. Tmpfs is now able to issue a lot more 64KB I/Os when under memory pressure."

• https://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/4eb0bb82efc8ef32c4357cf812891c08d38d8860

There's also a new tunable in the VM space as well as part of his commits on Wednesday night. This follows a lot of recent work on dsynth, improved page-out daemon pipelining, and other routine work.

• https://gitweb.dragonflybsd.org/dragonfly.git/commit/bc47dbc18bf832e4badb41f2fd79159479a7d351

This work is building up towards the eventual DragonFlyBSD 5.8 while those wanting to try the latest improvements right away can find their daily snapshots.

News Roundup

Why ZFS is not good at growing and reshaping pools (or shrinking them)

recently read Mark McBride's Five Years of Btrfs (via), which has a significant discussion of why McBride chose Btrfs over ZFS that boils down to ZFS not being very good at evolving your pool structure. You might doubt this judgment from a Btrfs user, so let me say as both a fan of ZFS and a long term user of it that this is unfortunately quite true; ZFS is not a good choice if you want to modify your pool disk layout significantly over time. ZFS works best if the only change in your pools that you do is replacing drives with bigger drives. In our ZFS environment we go to quite some lengths to be able to expand pools incrementally over time, and while this works it both leaves us with unbalanced pools and means that we're basically forced to use mirroring instead of RAIDZ.

(An unbalanced pool is one where some vdevs and disks have much more data than others. This is less of an issue for us now that we're using SSDs instead of HDs.)

Using PKGSRC on Manjaro Linux aarch64 Pinebook-pro

I wanted to see how pkgsrc works on aarch64 Linux Manjaro since it is a very mature framework that is very portable and supported by many architectures – pkgsrc (package source) is a package management system for Unix-like operating systems. It was forked from the FreeBSD ports collection in 1997 as the primary package management system for NetBSD.

One might question why use pkgsrc on Arch based Manjaro, since the pacman package repository is very good on its own. I see alternative pkgsrc as a good automated build framework that offers a way to produce independent build environment /usr/pkg that does not interfere with the current Linux distribution in any way (all libraries are statically built)

I have used the latest Manjaro for Pinebookpro and standard recommended tools as mentioned here https://wiki.netbsd.org/pkgsrc/how_to_use_pkgsrc_on_linux/

A Central Log Host with syslog-ng on FreeBSD

• Part 1

syslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure.

• Part 2

This blog post continues where the blog post A central log host with syslog-ng on FreeBSD left off. Open source solutions to check syslog log messages exist, such as Logcheck or Logwatch. Although these are not too difficult to implement and maintain, I still found these to much. So I went for my own home grown solution to check the syslog messages of the SoCruel.NU central log host.

Beastie Bits

• FreeBSD at Linux Conf 2020 session videos now online
• Unlock your laptop with your phone
• Managing a database of vulnerabilities for a package system: the pkgsrc study
• Hamilton BSD User group will meet again on March 10th](http://studybsd.com/)
• CharmBUG Meeting: March 24th 7pm in Severn, MD ***

Feedback/Questions

• Andrew - ZFS feature Flags
• Sam - TwinCat BSD
• Dacian - Freebsd + amdgpu + Lenovo E595

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv