Skip to main content.

Episode 160: EuroBSD-Dreamin

2016-09-21

Direct Download:

  • Video
  • HD Video
  • MP3 Audio
  • OGG Audio
  • Torrent
  • This episode was brought to you by

    iXsystems - Enterprise Servers and Storage for Open SourceDigitalOcean - Simple Cloud Hosting, Built for DevelopersTarsnap - Online Backups for the Truly Paranoid


    Headlines

    Performance Improvements for FreeBSD Kernel Debugging

    • “We previously explored FreeBSD userspace coredumps. Backtrace’s debugging platform supports FreeBSD kernel coredumps too, and their traces share many features. They are constructed somewhat differently, and in the process of adding support for them, we found a way to improve performance for automated programs accessing them.”
    • “A kernel core is typically only generated in exceptional circumstances. Unlike userspace processes, kernel routines cannot fault without sacrificing the machine’s availability. This means things like page faults and illegal instructions inside the kernel stop the machine, instead of just one process. At that point, in most cases, it is only usable enough to inspect its state in a debugger, or to generate a core file.”
    • No one likes it when this happens. This is why backtrace.io is focused on being able to figure out why it is happening
    • “A FreeBSD kernel core file can be formatted in several different ways. This depends on which type of dump was performed. Full core dumps are ELF files, similar in structure to userspace core files. However, as RAM size grew, this became more difficult to manage. In 2006, FreeBSD introduced minidumps, which are much smaller without making the core file useless. This has been the default dump type since FreeBSD 6.0.”
    • The article goes into detail on the minidump format, and some basic debugging techniques
    • “Libkvm will first determine whether the virtual address lies within the kernel or direct maps. If it lies in the kernel map, libkvm will consult the page table pages to discover the corresponding physical address. If it lies in the direct map, it can simply mask off the direct map base address. If neither of these applies, the address is illegal. This process is encapsulated by vatopa, or “virtual address to physical address”. Once the physical address is determined, libkvm consults the core file’s bitmap to figure out where in the core file it is located.”
    • “minidumps include a sparse bitmap indicating the pages that are included. These pages are dumped sequentially in the last section. Because they are sparse in a not entirely predictable way, figuring the offset into the dump for a particular physical address cannot be reduced to a trivial formula.”
    • The article goes into detail about how lookups against this map are slow, and how they were improved
    • “For typical manual debugger use, the impact of this change isn’t noticeable, which is probably why the hash table implementation has been in use for 10 years. However, for any automated debugging process, the extra latency adds up quickly.”
    • “On a sample 8GB kernel core file (generated on a 128GB server), crashinfo improves from 44 seconds to 9 seconds, and uses 30% less memory”
    • “Backtrace began shipping a version of this performance improvement in ptrace in February 2016. This enables us to also offer significantly faster tracing of FreeBSD kernel cores to customers running current and older releases of FreeBSD. On July 17, 2016, our work improving libkvm scaling was committed to FreeBSD/head. It will ship with FreeBSD 12.0.”

    OpenBSD gunzip pipeline tightening

    • OpenBSD has rethought the way they handle package signing
    • Changing from: 1/ fetch data -> 2/ uncompress it -> 3/ check signature -> 4/ process data
    • To: 1/ fetch data -> 2/ check signature -> 3/ uncompress -> 4/ process data
    • “The solution is to move the signature outside of the gzip header”
    • “Now, Since step 1/ is privsep, as long as step 2 is airtight, 3/ and 4/are no longer vulnerable”
    • Guidelines:
      • small, self-contained code to parse simple gzip headers
      • signify-style signature in the gzip comment. Contains checksums of 64K blocks of the compressed archive
      • don't even think about passing the original gzip header through
      • use as a pipeline step: does not need to download full archive to use it, and never ever pass any data to the gunzip part before it's been verified.
    • “Note that afaik we haven't had any hole in our gunzipping process. Well… waiting for an accident to happen is not how we do things. Hopefully, this should prevent future mishaps.”

    OpenVPN On FreeBSD 10.3

    • “While trying to setup OpenVPN, I noticed there was no up-to-date information with correct instructions. OpenVPN uses EasyRSA to setup keys, it has recently been changed in version 3. As a result of this, the old steps to configure OpenVPN are no longer correct. I went through the process of setting up a VPN using OpenVPN on FreeBSD 10.3.”
    • I know FreeBSD developer Adrian Chadd complained about this exact problem when he was trying to setup a VPN before attending DEFCON
    • The tutorial walks through the basic steps:
      • Install the needed software
      • Configure EasyRSA
      • Create a CA
      • Generate keys and DH params
      • OpenVPN Server Config
      • OpenVPN Client Config
      • Starting the daemon
    • It even finishes off with bonus instructions on Port Forwarding, Firewalls, and Dynamic DNS

    lsop

    • LSOP is the tool a bunch of users have been asking for
    • “a FreeBSD utility to list all processes running with outdated binaries or shared libraries”
    • How does it work? “lsop iterates over all running processes and looks through memory-mapped files with read + execute access; then it checks if those files are still available or have been modified/deleted.”
    • How would you use it? After installing an system update (that doesn’t require a reboot to update the kernel), or upgrade your packages, you still need to know which daemons need to be restarted to use the patched libraries and binaries
    • This tool gives you that list
    • Thanks to Bogdan Boyadzhiev for writing this much needed tool

    News Roundup

    OpenBSD 2016 Fundraising Campaign

    • The OpenBSD fund-raising campaign has given us a status update on the state of 2016.
    • They start by giving us a re-cap of previous years: “2015 was a good year for the foundation financially, with one platinum, one gold, four silver and 3 bronze donors providing half of our total donations. 680 individuals making smaller contributions provided the other half. While the total was down significantly after 2014’s blockbuster year, we again exceeded our goal.”

    • As of Sept 5th, they were at approx $115k out of a total goal of 250k.

    • If you are an OpenBSD user, remember to contribute before the end of the year. Small amounts help, and the money of course goes to great causes such as hackathons and running the OpenBSD infrastructure.

    Update firewall Bad Countries

    • Network and Systems admins know, sometimes when all else fails you need to break out the HUGE ban-hammer. In this case sometimes entire countries get put on the excrement list until the attacks stop.
    • We have a handy GitHub project today, which will assist you in doing exactly that, enter update-fw-BC. (Update firewall by country)
    • This perl script may be your savior when dealing with instances that require major brute force. It specifically works with IPFW, PF and IPTABLES, which will allow it to run across a variety of BSD’s or even Linux.
    • It will ingest a list of IP’s that you feed it (perhaps from another tool such as sshguard) and determine what block the IP belongs to, and match according to country.
    • Detailed setup instructions for the various firewalls are included, and some instructions for FreeBSD, although using it on OpenBSD or other $BSD should also be easy to adapt.

    More utilities via moreutils

    • In most BSDs, the “core” set of utilities and commands are just part of the base system, but on Linux, they are usually provided by the “coreutils” package.
    • However, on Linux and now FreeBSD, there is a “moreutils” package, that provides a number of interesting additional basic utilities, including:
      • chronic: Run a task via crontab, and only generate output if the task fails
      • combine: binary AND two text files together, only displaying lines that are in both files
      • errno: look up the text description of a specific error number
      • ifdata: parse out specific information from ifconfig
      • ifne: if-not-empty, only run a command if the output of the pipe is not blank
      • isutf8: determine if a file or stdin contains utf8
      • lckdo: execute a command with a lock held, to prevent a second copy from spawning
      • mispipe: return the exit code of the first command in a pipe chain, rather than the last
      • parallel: run multiple jobs at once
      • pee: tee standard input to multiple pipes
      • sponge: write standard input to a file, allows you to overwrite a file in place: sort file | sponge file
      • ts: add a timestamp to each line of standard input
      • vidir: edit a directory in vi, great for bulk renames
      • vipe: insert vi into a pipe, edit the content before it is passed to the next command
      • zrun: uncompress the arguments before passing them. Like gzless and friends, but for any command
    • Just goes to show the power of the original UNIX philosophy, chaining together a bunch of small useful tools to do really powerful things

    OpenBSD: SNI support added to libtls, httpd in –current

    • libtls, LibreSSL’s improved API to replace the OpenSSL standard, now has a set of functions to implement SNI (Server Name Indication)
    • Until a few years ago, each different SSL/TLS enabled website required a unique IP address, because typical HTTP Virtual Hosting (differentiating which content to serve based on the Host header in the HTTP request), didn’t work because the request was encrypted.
    • Finally the TLS standard was updated to include the hostname of the site the user is requesting in the TLS handshake, so the server can return the corresponding certificate, and multiple TLS enabled websites can be hosted on a single IP address
    • The new API includes the ability to provide additional keypairs (via tlsconfigaddkeypair{file,mem}())
    • And allow the server to determine what servername the client requested viatlsconnservername()
    • This is much easier to use, and therefore safer and less error prone, than the OpenSSL API
    • The libtls API is used in a number of OpenBSD tools, including the httpd

    Beastie Bits


    Feedback/Questions


    • Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Latest News

New announcement

2017-05-25

Hi, Mr. Dexter. Also, we understand that Brad Davis thinks there should be more real news....

Two Year Anniversary

2015-08-08

We're quickly approaching our two-year anniversary, which will be on episode 105. To celebrate, we've created a unique t-shirt design, available for purchase until the end of August. Shirts will be shipped out around September 1st. Most of the proceeds will support the show, and specifically allow us to buy...

New discussion segment

2015-01-17

We're thinking about adding a new segment to the show where we discuss a topic that the listeners suggest. It's meant to be informative like a tutorial, but more of a "free discussion" format. If you have any subjects you want us to explore, or even just a good name...

How did you get into BSD?

2014-11-26

We've got a fun idea for the holidays this year: just like we ask during the interviews, we want to hear how all the viewers and listeners first got into BSD. Email us your story, either written or a video version, and we'll read and play some of them for...


Episode 220: Opening ZFS in 2017

2017-11-15

Direct Download:HD VideoMP3 AudioTorrent This episode was brought to you by Headlines The First PS4 Kernel Exploit: Adieu The First PS4 Kernel Exploit: Adieu Plenty of time has passed since we first demonstrated Linux running on the PS4. Now we will step back a bit and explain how we managed to jump...

Episode 219: We love the ARC

2017-11-08

Direct Download:HD VideoMP3 AudioTorrent This episode was brought to you by Headlines Papers We Love: ARC: A Self-Tuning, Low Overhead Replacement Cache Ever wondered how the ZFS ARC (Adaptive Replacement Cache) works? How about if Bryan Cantrill presented the original paper on its design? Today is that day. Slides It starts by looking back at a fundamental paper...

Episode 218: A KRACK in the WiFi

2017-11-01

Direct Download:HD VideoMP3 AudioTorrent This episode was brought to you by Headlines FreeBSD 10.4-RELEASE Available FreeBSD 10.4-RELEASE is out. The FreeBSD Project dedicates the FreeBSD 10.4-RELEASE to the memory of Andrey A. Chernov. Some of the highlights: 10.4-RELEASE is the first FreeBSD release to feature full support for eMMC storage, including eMMC partitions, TRIM...

Episode 217: Your questions, part II

2017-10-25

Direct Download:HD VideoMP3 AudioTorrent This episode was brought to you by Headlines OpenBSD 6.2 Released OpenBSD continues their six month release cadence with the release of 6.2, the 44th release On a disappointing note, the song for 6.2 will not be released until December Highlights: Improved hardware support on modern platforms including ARM64/ARMv7 and octeon,...