Headlines

Checksumming in filesystems, and why ZFS is doing it right

One of the best aspects of ZFS is its reliability. This can be accomplished using a few features like copy-on-write approach and checksumming. Today we will look at how ZFS does checksumming and why it does it the proper way. Most of the file systems don’t provide any integrity checking and fail in several scenarios:

• Data bit flips - when the data that we wanted to store are bit flipped by the hard drives, or cables, and the wrong data is stored on the hard drive.
• Misdirected writes - when the CPU/cable/hard drive will bit flip a block to which the data should be written.
• Misdirected read - when we miss reading the block when a bit flip occurred.
• Phantom writes - when the write operation never made it to the disk. For example, a disk or kernel may have some bug that it will return success even if the hard drive never made the write. This problem can also occur when data is kept only in the hard drive cache.

Checksumming may help us detect errors in a few of those situations.

DragonFlyBSD Improves Its TMPFS Implementation For Better Throughput Performance

It's been a while since last having any new magical optimizations to talk about by DragonFlyBSD lead developer Matthew Dillon, but on Wednesday he landed some significant temporary file-system "TMPFS" optimizations for better throughput including with swap.

Of several interesting commits merged tonight, the improved write clustering is a big one. In particular, "Reduces low-memory tmpfs paging I/O overheads by 4x and generally increases paging throughput to SSD-based swap by 2x-4x. Tmpfs is now able to issue a lot more 64KB I/Os when under memory pressure."

• https://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/4eb0bb82efc8ef32c4357cf812891c08d38d8860

There's also a new tunable in the VM space as well as part of his commits on Wednesday night. This follows a lot of recent work on dsynth, improved page-out daemon pipelining, and other routine work.

• https://gitweb.dragonflybsd.org/dragonfly.git/commit/bc47dbc18bf832e4badb41f2fd79159479a7d351

This work is building up towards the eventual DragonFlyBSD 5.8 while those wanting to try the latest improvements right away can find their daily snapshots.

News Roundup

Why ZFS is not good at growing and reshaping pools (or shrinking them)

recently read Mark McBride's Five Years of Btrfs (via), which has a significant discussion of why McBride chose Btrfs over ZFS that boils down to ZFS not being very good at evolving your pool structure. You might doubt this judgment from a Btrfs user, so let me say as both a fan of ZFS and a long term user of it that this is unfortunately quite true; ZFS is not a good choice if you want to modify your pool disk layout significantly over time. ZFS works best if the only change in your pools that you do is replacing drives with bigger drives. In our ZFS environment we go to quite some lengths to be able to expand pools incrementally over time, and while this works it both leaves us with unbalanced pools and means that we're basically forced to use mirroring instead of RAIDZ.

(An unbalanced pool is one where some vdevs and disks have much more data than others. This is less of an issue for us now that we're using SSDs instead of HDs.)

Using PKGSRC on Manjaro Linux aarch64 Pinebook-pro

I wanted to see how pkgsrc works on aarch64 Linux Manjaro since it is a very mature framework that is very portable and supported by many architectures – pkgsrc (package source) is a package management system for Unix-like operating systems. It was forked from the FreeBSD ports collection in 1997 as the primary package management system for NetBSD.

One might question why use pkgsrc on Arch based Manjaro, since the pacman package repository is very good on its own. I see alternative pkgsrc as a good automated build framework that offers a way to produce independent build environment /usr/pkg that does not interfere with the current Linux distribution in any way (all libraries are statically built)

I have used the latest Manjaro for Pinebookpro and standard recommended tools as mentioned here https://wiki.netbsd.org/pkgsrc/how_to_use_pkgsrc_on_linux/

A Central Log Host with syslog-ng on FreeBSD

• Part 1

syslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure.

• Part 2

This blog post continues where the blog post A central log host with syslog-ng on FreeBSD left off. Open source solutions to check syslog log messages exist, such as Logcheck or Logwatch. Although these are not too difficult to implement and maintain, I still found these to much. So I went for my own home grown solution to check the syslog messages of the SoCruel.NU central log host.

Beastie Bits

• FreeBSD at Linux Conf 2020 session videos now online
• Unlock your laptop with your phone
• Managing a database of vulnerabilities for a package system: the pkgsrc study
• Hamilton BSD User group will meet again on March 10th](http://studybsd.com/)
• CharmBUG Meeting: March 24th 7pm in Severn, MD ***

Feedback/Questions

• Andrew - ZFS feature Flags
• Sam - TwinCat BSD
• Dacian - Freebsd + amdgpu + Lenovo E595

• Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

This episode was brought to you by

Headlines

FreeBSD 10.0-RELEASE is out

• The long awaited, giant release of FreeBSD is now official and ready to be downloaded
• One of the biggest releases in FreeBSD history, with tons of new updates
• Some features include: LDNS/Unbound replacing BIND, Clang by default (no GCC anymore), native Raspberry Pi support and other ARM improvements, bhyve, hyper-v support, AMD KMS, VirtIO, Xen PVHVM in GENERIC, lots of driver updates, ZFS on root in the installer, SMP patches to pf that drastically improve performance, Netmap support, pkgng by default, wireless stack improvements, a new iSCSI stack, FUSE in the base system... the list goes on and on
• Start up your freebsd-update or do a source-based upgrade ***

OpenSSH 6.5 CFT

• Our buddy Damien Miller announced a Call For Testing for OpenSSH 6.5
• Huge, huge release, focused on new features rather than bugfixes (but it includes those too)
• New ciphers, new key formats, new config options, see the mailing list for all the details
• Should be in OpenBSD 5.5 in May, look forward to it - but also help test on other platforms! ***

DIY NAS story, FreeNAS 9.2.1-BETA

• Another new blog post about FreeNAS!
• Instead of updating the older tutorials, the author started fresh and wrote a new one for 2014
• "I did briefly consider suggesting nas4free for the EconoNAS blog, since it’s essentially a fork off the FreeNAS tree but may run better on slower hardware, but ultimately I couldn’t recommend anything other than FreeNAS"
• Really long article with lots of nice details about his setup, why you might want a NAS, etc.
• Speaking of FreeNAS, they released 9.2.1-BETA with lots of bugfixes ***

OpenBSD needed funding for electricity.. and they got it

• Briefly mentioned at the end of last week's show, but has blown up over the internet since
• OpenBSD in the headlines of major tech news sites: slashdot, zdnet, the register, hacker news, reddit, twitter.. thousands of comments
• They needed about $20,000 to cover electric costs for the server rack in Theo's basement
• Lots of positive reaction from the community helping out so far, and it appears they have reached their goal and got $100,000 in donations
• From Bob Beck: "we have in one week gone from being in a dire situation to having a commitment of approximately $100,000 in donations to the foundation"
• This is a shining example of the BSD community coming together, and even the Linux people realizing how critical BSD is to the world at large ***

Interview - Colin Percival - cperciva@freebsd.org / @cperciva

FreeBSD on Amazon EC2, backups with Tarsnap, 10.0-RELEASE, various topics

Tutorial

Bandwidth monitoring and testing

News Roundup

pfSense talk at Tokyo FreeBSD Benkyoukai

• Isaac Levy will be presenting "pfSense Practical Experiences: from home routers, to High-Availability Datacenter Deployments"
• He's also going to be looking for help to translate the pfSense documentation into Japanese
• The event is on February 17, 2014 if you're in the Tokyo area ***

m0n0wall 1.8.1 released

• For those who don't know, m0n0wall is an older BSD-based firewall OS that's mostly focused on embedded applications
• pfSense was forked from it in 2004, and has a lot more active development now
• They switched to FreeBSD 8.4 for this new version
• Full list of updates in the changelog
• This version requires at least 128MB RAM and a disk/CF size of 32MB or more, oh no! ***

Ansible and PF, plus NTP

• Another blog post from our buddy Michael Lucas
• There've been some NTP amplification attacks recently in the news
• The post describes how he configured ntpd on a lot of servers without a lot of work
• He leverages pf and ansible for the configuration
• OpenNTPD is, not surprisingly, unaffected - use it ***

ruBSD videos online

• Just a quick followup from a few weeks ago
• Theo and Henning's talks from ruBSD are now available for download
• There's also a nice interview with Theo ***

PCBSD weekly digest

• 10.0-RC4 images are available
• Wine PBI is now available for 10
• 9.2 systems will now be able to upgrade to version 10 and keep their PBI library ***

Feedback/Questions

• Sha'ul writes in
• Kjell-Aleksander writes in
• Mike writes in
• Charlie writes in (and gets a reply)
• Kevin writes in ***