Skip to main content.

Episode 123: ZFS in the trenches

2016-01-06

Direct Download:

Video | HD Video | MP3 Audio | OGG Audio | Torrent

iX Systems Mission Complete

  • Submit your story of how you accomplished a mission with FreeBSD, FreeNAS, or iXsystems hardware, and you could win monthly prizes, and have your story featured in the FreeBSD Journal!

FreeNAS Logo Design Contest

Rules and Requirements


For those of you curious about Kris' new lighting here are the links to what he is using.


This episode was brought to you by

iXsystems - Enterprise Servers and Storage for Open SourceDigitalOcean - Simple Cloud Hosting, Built for DevelopersTarsnap - Online Backups for the Truly Paranoid


Headlines

A Brief look back at 2015

  • As we start the show this week, we begin with a brief look back at BSD in 2015, brought to us by Larry at FOSS force.
  • Aside from his issue with tap-to-click on the touchpad, his PC-BSD experience has been pretty good. (Larry, if you hear this, jump on #pcbsd on FreeNode and we will lend a hand)
  • He mentions that this really isn’t his first time running BSD, apparently back in ye-olden days he got NetBSD up and running on a PowerBook G3, until an update brought that experience to abrupt ending.
  • He gives a shout-out to the FreeBSD Foundation as being a great go-to source for wrapup on the previous year in FreeBSD land, while also mentioning the great 4.4 release of DragonFly, and some of the variants, such as RetroBSD and LiteBSD
  • He leaves us with a tease for 2016 that work is ongoing on Twitter to port over Mopidy, a python based extensible music server

A look forward at BSD events throughout 2016

  • After a quick look back at 2015, now its time to start planning your 2016 schedule. The BSDEvents site has a calendar of all the upcoming conferences / shows where BSD will have a presence this year.
  • There are quite a few items on the agenda, including non BSD specific conferences, such as SCALE / Fosdem and more.
  • Take a look and see, you may be able to find something close your location where you can come hang out with other BSD developers.
  • (or better yet), if a linux conference is coming to your town, think about submitting a BSD talk!
  • Additionally, if getting BSD Certification is something on your 2016 resolutions, you can often take the test at one of these shows, avoiding the need to travel to a testing center.

The 'Hidden' Cost of Using ZFS for Your Home NAS

  • An article was recently posted that seems to be trying to dissuade people from using ZFS for their home NAS
  • It points out what experienced users already know, but many newcomers are not strictly aware of: Expanding a ZFS pool is not always as straightforward as you think it should be
  • ZFS was designed to be expanded, and it handled this very well
  • However, a ZFS pool is made up of VDEVs, and it is these VDEVs that provide the redundancy. RAID-Z VDEVs cannot be changed once they are created. You can replace each disk individually, and the VDEV will grow to its new larger size, but you cannot add additional disks to a RAID-Z VDEV
  • At this point, your option is to add an additional VDEV, although best practises dictate that the new VDEV should use an equal number of disks, to avoid uneven performance
  • So, if you started with a 6 disk RAID-Z2, having to add 6 more disks to grow the pool does seem excessive
  • For the best flexibility, use mirrors. If you had used 6 disks as 3 mirrors of 2 disks each, you could then just add 2 more disks at a time. The downside is that using 2TB disks, you’d only have 6TB of usable space, versus the 8TB you would get from those disks in a RAID-Z2
  • This is the trade-off, mirrors give you better performance and flexibility, but less space efficiency
  • It is important to note that the diagrams in this article make it appear as if all parity information is stored on specific drives. In ZFS parity is spread across all drives. Often times, the data written to the drive is not of a size that can evenly be split across all drives, so the data actually ends up looking like this
  • The errors as I see it in the original article are:
    • It notes that the hidden cost of ZFS is that if you add a second RAID-Z VDEV, you will have a whole second set of parity drives. While this is a cost, it is the cost of making sure your data is safe. If you had an array with more than 12 drives, it is likely that you would to be able to withstand the failure of the larger number of drives
    • The article does not consider the resilver time. If you did create a configuration with a very wide RAID-Z stripe, the failure of a disk would leave the pool degraded for a much longer time, leaving your pool at risk for that longer period.
    • The article does not consider performance. Two RAID-Z2 VDEVs of 6 disks each will give much better performance than a single VDEV of 10 or 12 disks, especially when it comes to IOPS.

ZFS Boot Enviroments now availble in the FreeBSD bootloader

  • It’s been in phabricator for a while (and PC-BSD), but the support for Boot-Environments has now landed upstream in -CURRENT
  • This work was helped by cross-project collaboration when an IllumOS Developer, Toomas Soome, started porting the FreeBSD loader to IllumOS to replace GRUB there
  • This gives Beastie menu the ability to look at the ZFS disk, and dynamically list boot-environments that it finds. (Much nicer than GRUB, which required a pre-written configuration file)
  • This work was extended further, when Toomas Soome also ported the Beastie Menu to the UEFI loader which is now enabled by default for UEFI
  • All of these changes are scheduled to be merged back in time for FreeBSD 10.3 as well.
  • There is also a patch being worked on to support booting from ZFS in UEFI
  • This is exciting times for doing neat things with ZFS on root, these plus Allans forthcoming GELI support will negate the necessity for GRUB on PC-BSD for example (Kris is very happy)

Interview - Josh Paetzel - email@email / @bsdunix4ever

  • ZFS Support

News Roundup

RetroBSD being tested on ESP32

  • More hardware news for RetroBSD and LiteBSD
  • I don’t know much about this hardware, but there is a lot of discussion in the forum threads about it
  • Not sure what you are supposed to accomplish with only 400kb of ram
  • LITEBSD Brings 4.4BSD to PIC32
  • It is interesting to see these super-small boards with only 512kb of memory, but will crypto offload support
  • It is also interesting to see talk of 140mbps WiFi, can the processor actually handle that much traffic? BSD Unix-like OS is Resurrected for Embedded IoT Market
  • Related to the above stories, we also have an article about BSD making a resurgence on various Internet of things devices, which mentions both RetroBSD and LiteBSD
  • The article mentions that this is an exciting development for embedded vars who now have an alternative licensed open-source OS to potentially use

HardenedBSD’s new Binary Updater

  • It looks like there is now another way to update your FreeBSD(hardened) system
  • The post by Shawn Web, details how the new updater will work in future releases of HBSD
  • Right now it looks fairly straight-forward, creating both the base.txz and kernel.txz, along with some data for etcupdate
  • It includes a nice option for the kernel name in the update, allowing different kernels to be installed / updated at will
  • Everything is cryptographically signed and verified using the base system openssl
  • The build system is fairly simple, only requiring “sh/git/openssl” to create the binary updates
  • Planned features also include updating of jails, and ZFS boot-environments

Sometimes, processors need (BSD) love too

  • We have a blog post from Brian Everly, talking about his long journey into legacy processors and the plans for the future to work on better supporting them on OpenBSD ports
  • He begins with the story of his UNIX journey to today, and why this fostered his love for many of these old (and not so old) architectures, such as Sparc64, PPC32, i386.
  • This journey ended up with the purchase of some legacy hardware (ebay is your friend), and the creation of a database listing the major port blockers on each platform
  • This is the great kind of thing folks can do to step up and help a project, even as a weekend hobby it’s great to run some hardware and help test / fix up issues that other developers maybe don’t interact with as much anymore.

Beastie Bits

The standard MWL disclaimer

PC-BSD 11.0-CURRENTJAN2016 Available

NetBSD pkgsrc-2015Q3 statistics

NetBSD pkgsrc-2015Q4 released

First Reproducible builds conference in Athens

The creator of the original ThinkPad design passes away


Feedback/Questions


  • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Latest News

Two Year Anniversary

2015-08-08

We're quickly approaching our two-year anniversary, which will be on episode 105. To celebrate, we've created a unique t-shirt design, available for purchase until the end of August. Shirts will be shipped out around September 1st. Most of the proceeds will support the show, and specifically allow us to buy...

New discussion segment

2015-01-17

We're thinking about adding a new segment to the show where we discuss a topic that the listeners suggest. It's meant to be informative like a tutorial, but more of a "free discussion" format. If you have any subjects you want us to explore, or even just a good name...

How did you get into BSD?

2014-11-26

We've got a fun idea for the holidays this year: just like we ask during the interviews, we want to hear how all the viewers and listeners first got into BSD. Email us your story, either written or a video version, and we'll read and play some of them for...

EuroBSDCon 2014

2014-09-18

As you might expect, both Allan and Kris will be at EuroBSDCon this year. They'll be busy hunting down various BSD developers and forcing them to do interviews, but don't hesitate to say hi if you're a listener!...


Episode 170: Sandboxing Cohabitation

2016-11-30

Direct Download:VideoHD VideoMP3 AudioOGG AudioTorrent This episode was brought to you by Headlines EuroBSDcon 2016 Presentation Slides Due to circumstances beyond the control of the organizers of EuroBSDCon, there were not recordings of the talks given at the event. However, they have collected the slide decks from each of the speakers and assembled them on...

Episode 169: Scheduling your NetBSD

2016-11-23

Direct Download:VideoHD VideoMP3 AudioOGG AudioTorrent This episode was brought to you by Headlines Production ready Ted Unangst brings us a piece on what it means to be Production Ready He tells the story of a project he worked on that picked a framework that was “production ready” They tested time zones, and it all seemed to...

Episode 168: The Post Show Show

2016-11-16

Direct Download:VideoHD VideoMP3 AudioOGG AudioTorrent This episode was brought to you by Headlines Build a FreeBSD 11.0-release Openstack Image with bsd-cloudinit We are going to prepare a FreeBSD image for Openstack deployment. We do this by creating a FreeBSD 11.0-RELEASE instance, installing it and converting it using bsd-cloudinit. We'll use the CloudVPS...

Episode 167: Playing the Long Game

2016-11-09

Direct Download:VideoHD VideoMP3 AudioOGG AudioTorrent This episode was brought to you by Interview - Scott Long - scottl@freebsd.org FreeBSD & Netflix Feedback/Questions Zack - USB Config Jens - VMs, Jails and Containers Ranko - Tarsnap Keys Alex - OpenBSD in Hyper-V Curt - Discussion Segment Send questions, comments, show ideas/topics, or stories you want mentioned...