Headlines

Checksumming in filesystems, and why ZFS is doing it right

One of the best aspects of ZFS is its reliability. This can be accomplished using a few features like copy-on-write approach and checksumming. Today we will look at how ZFS does checksumming and why it does it the proper way. Most of the file systems don’t provide any integrity checking and fail in several scenarios:

• Data bit flips - when the data that we wanted to store are bit flipped by the hard drives, or cables, and the wrong data is stored on the hard drive.
• Misdirected writes - when the CPU/cable/hard drive will bit flip a block to which the data should be written.
• Misdirected read - when we miss reading the block when a bit flip occurred.
• Phantom writes - when the write operation never made it to the disk. For example, a disk or kernel may have some bug that it will return success even if the hard drive never made the write. This problem can also occur when data is kept only in the hard drive cache.

Checksumming may help us detect errors in a few of those situations.

DragonFlyBSD Improves Its TMPFS Implementation For Better Throughput Performance

It's been a while since last having any new magical optimizations to talk about by DragonFlyBSD lead developer Matthew Dillon, but on Wednesday he landed some significant temporary file-system "TMPFS" optimizations for better throughput including with swap.

Of several interesting commits merged tonight, the improved write clustering is a big one. In particular, "Reduces low-memory tmpfs paging I/O overheads by 4x and generally increases paging throughput to SSD-based swap by 2x-4x. Tmpfs is now able to issue a lot more 64KB I/Os when under memory pressure."

• https://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/4eb0bb82efc8ef32c4357cf812891c08d38d8860

There's also a new tunable in the VM space as well as part of his commits on Wednesday night. This follows a lot of recent work on dsynth, improved page-out daemon pipelining, and other routine work.

• https://gitweb.dragonflybsd.org/dragonfly.git/commit/bc47dbc18bf832e4badb41f2fd79159479a7d351

This work is building up towards the eventual DragonFlyBSD 5.8 while those wanting to try the latest improvements right away can find their daily snapshots.

News Roundup

Why ZFS is not good at growing and reshaping pools (or shrinking them)

recently read Mark McBride's Five Years of Btrfs (via), which has a significant discussion of why McBride chose Btrfs over ZFS that boils down to ZFS not being very good at evolving your pool structure. You might doubt this judgment from a Btrfs user, so let me say as both a fan of ZFS and a long term user of it that this is unfortunately quite true; ZFS is not a good choice if you want to modify your pool disk layout significantly over time. ZFS works best if the only change in your pools that you do is replacing drives with bigger drives. In our ZFS environment we go to quite some lengths to be able to expand pools incrementally over time, and while this works it both leaves us with unbalanced pools and means that we're basically forced to use mirroring instead of RAIDZ.

(An unbalanced pool is one where some vdevs and disks have much more data than others. This is less of an issue for us now that we're using SSDs instead of HDs.)

Using PKGSRC on Manjaro Linux aarch64 Pinebook-pro

I wanted to see how pkgsrc works on aarch64 Linux Manjaro since it is a very mature framework that is very portable and supported by many architectures – pkgsrc (package source) is a package management system for Unix-like operating systems. It was forked from the FreeBSD ports collection in 1997 as the primary package management system for NetBSD.

One might question why use pkgsrc on Arch based Manjaro, since the pacman package repository is very good on its own. I see alternative pkgsrc as a good automated build framework that offers a way to produce independent build environment /usr/pkg that does not interfere with the current Linux distribution in any way (all libraries are statically built)

I have used the latest Manjaro for Pinebookpro and standard recommended tools as mentioned here https://wiki.netbsd.org/pkgsrc/how_to_use_pkgsrc_on_linux/

A Central Log Host with syslog-ng on FreeBSD

• Part 1

syslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure.

• Part 2

This blog post continues where the blog post A central log host with syslog-ng on FreeBSD left off. Open source solutions to check syslog log messages exist, such as Logcheck or Logwatch. Although these are not too difficult to implement and maintain, I still found these to much. So I went for my own home grown solution to check the syslog messages of the SoCruel.NU central log host.

Beastie Bits

• FreeBSD at Linux Conf 2020 session videos now online
• Unlock your laptop with your phone
• Managing a database of vulnerabilities for a package system: the pkgsrc study
• Hamilton BSD User group will meet again on March 10th](http://studybsd.com/)
• CharmBUG Meeting: March 24th 7pm in Severn, MD ***

Feedback/Questions

• Andrew - ZFS feature Flags
• Sam - TwinCat BSD
• Dacian - Freebsd + amdgpu + Lenovo E595

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

This episode was brought to you by

Headlines

Faces of FreeBSD continues

• Our first one details Shteryana Shopova, the local organizer for EuroBSDCon 2014 in Sophia
• Gives some information about how she got into BSD
• "I installed FreeBSD on my laptop, alongside the Windows and Slackware Linux I was running on it at the time. Several months later I realized that apart from FreeBSD, I hadn't booted the other two operating systems in months. So I wiped them out."
• She wrote bsnmpd and extended it with the help of a grant from the FreeBSD Foundation
• We've also got one for Kevin Martin
• Started off with a pinball website, ended up learning about FreeBSD from an ISP and starting his own hosting company
• "FreeBSD has been an asset to our operations, and while we have branched out a bit, we still primarily use FreeBSD and promote it whenever possible. FreeBSD is a terrific technology with a terrific community." ***

OpenPF?

• A blog post over at the Dragonfly digest
• What if we had some cross platform development of OpenBSD's firewall?
• Similar to portable OpenSSH or OpenZFS, there could be a centrally-developed version with compatibility glue
• Right now FreeBSD 9's pf is old, FreeBSD 10's pf is old (but has the best performance of any implementation due to custom patches), NetBSD's pf is old (but they're working on a fork) and Dragonfly's pf is old
• Further complicated by the fact that PF itself doesn’t have a version number, since it was designed to just be ‘the pf that came with OpenBSD 5.4’
• Not likely to happen any time soon, but it's good food for thought ***

Year of BSD on the server

• A good blog post about switching servers from Linux to BSD
• 2014 is going to be the year of a lot of switching, due to FreeBSD 10's amazing new features
• This author was particularly taken with pkgng and the more coherent layout of BSD systems
• Similarly, there was also a recent reddit thread, "Why did you choose BSD over Linux?"
• Both are excellent reads for Linux users that are thinking about making the switch, send 'em to your friends ***

Getting to know your portmgr

• This time in the series they interview Bryan Drewery, a fairly new addition to the team
• He started maintaining portupgrade and portmaster, and eventually ended up on the ports management team
• Believe it or not, his wife actually had a lot to do with him getting into FreeBSD full-time
• Lots of fun trivia and background about him
• Speaking of portmgr, our interview for today is... ***

Interview - Baptiste Daroussin - bapt@freebsd.org

The future of FreeBSD's binary packages, ports' features, various topics

News Roundup

pfSense december hang out

• Interview/presentation from pfSense developer Chris Buechler with an accompanying blog post
• "This is the first in what will be a monthly recurring series. Each month, we’ll have a how to tutorial on a specific topic or area of the system, and updates on development and other happenings with the project. We have several topics in mind, but also welcome community suggestions on topics"
• Speaking of pfSense, they recently opened an online store
• We're planning on having a pfSense episode next month! ***

BSDMag December issue is out

• The free monthly BSD magazine gets a new release for December
• Topics include CARP on FreeBSD, more BSD programming, "unix basics for security professionals," some kernel introductions, using OpenBSD as a transparent proxy with relayd, GhostBSD overview and some stuff about SSH ***

OpenBSD gets tmpfs

• In addition to the recently-added FUSE support, OpenBSD now has tmpfs
• To get more testing, it was enabled by default in -current
• Should make its way into 5.5 if everything goes according to plan
• Enables lots of new possibilities, like our ccache and tmpfs guide ***

PCBSD weekly digests

• Catching up with all the work going on in PCBSD land..
• 10.0-RC2 is now available
• The big pkgng 1.2 problems seem to have been worked out ***

Feedback/Questions

• Remy writes in
• Jason writes in
• Rob writes in
• John writes in
• Stuart writes in ***