Skip to main content.

Virtualization with bhyve

2014-01-15

Live demo in BSD Now Episode 020.

Virtualization has been a hot topic for a while now, and there are lots of options for it. On FreeBSD, there's the often-copied jail feature. For a lot of use cases, that's ideal. There's no overhead, they're very secure and they offer supreme flexibility. However, sometimes people need to use other operating systems. You could use a userland application like VirtualBox, but performance leaves something to be desired. There are times when performance needs to be at the top of the list. For those times, you'll need something else.

You'll need bhyve.

Short for the BSD Hypervisor, bhyve is a great virtualization technology in the vein of Xen or KVM, but done BSD style and without legacy support. It's a type 2 hypervisor that lets you run other operating systems under FreeBSD. It was officially released in FreeBSD 10.0. So enough talking, let's get it set up. Keep in mind that bhyve requires an Intel CPU with VT-x and Extended Page Table support.


FreeBSD

Start off by creating a disk image to use for the VM. We'll use a 10GB file.

# truncate -s 10g bhyve1.img

Load the bhyve kernel module.

# kldload vmm

Create a bridge and your tap network interface.

# ifconfig bridge0 create
# ifconfig tap0 create
# sysctl net.link.tap.up_on_open=1

Add your LAN interface and the tap interface to the bridge. The NIC is called igb0 in this case.

# ifconfig bridge0 addm igb0 addm tap0
# ifconfig bridge0 up

Download a copy of FreeBSD to install.

# fetch ftp://ftp.freebsd.org/pub/FreeBSD/ISO-IMAGES-amd64/10.0/FreeBSD-10.0-RC5-amd64-bootonly.iso

Copy the template bhyve script. (Official Instructions)

# cp /usr/share/examples/bhyve/vmrun.sh bhyve1.sh
# chmod +x bhyve1.sh

Configure the number of CPUs, amount of ram, name of disk image.

# vi bhyve1.sh

Start bhyve in installation mode.

# ./bhyve1.sh -i -I FreeBSD-10.0-RC5-amd64-bootonly.iso fbsd10

Do an install as usual, but at the end, choose to enter a shell on the installed system. Because bhyve does not emulate a graphics card, only a serial console, we need to configure the system to boot with the serial console.

# vi /etc/ttys

Change this line:

console none    unknown off secure

To the following:

console "/usr/libexec/getty std.9600"    xterm   on  secure

Now, restart the vm, and choose reboot from the beastie menu to exit.

# reboot

Rebooting exits bhyve, so lets start it again without the install disk.

# ./bhyve1.sh fbsd10

You now have a working FreeBSD in bhyve.

# reboot

Select reboot, or hit ESC and type quit, from the beastie menu to exit. The vmrun script tries to keep your vm running so restarts it when it crashes/exits.


OpenBSD

This requires the grub2-bhyve port. Install it via ports or pkgng. Create an OpenBSD tap device and add it to the bridge

# ifconfig tap1 create
# ifconfig bridge0 addm tap1

Download this working OpenBSD image:

# fetch http://people.freebsd.org/~grehan/flashimg.amd64-20131014.bz2
# bunzip2 flashimg.amd64-20131014.bz2

Create a grub device map, obsd.map, containing:

(hd0) ./obsd.img
Booting the OpenBSD image:
grub-bhyve -m obsd.map -r hd0 -M 512 obsd

This will start grub, enter the following:

kopenbsd -h com0 (hd0,openbsd1)/bsd
boot

Start bhyve

# bhyve -c 2 -m 512M -A -H -P -s 0:0,amd_hostbridge -s \
  1:0,lpc -s 2:0,virtio-net,tap1 -s 3:0,virtio-blk,obsd.img -l com1,stdio -W obsd

The image may fail to boot because it was shutdown improperly, restarting it should resolve this. The root password is: test123

Clean up the VM:

# bhyvectl --destroy --vm=obsd

This tutorial is ongoing and will be updated at a later time to include autostarting.

Originally written by Allan for bsdnow.tv, with minor edits | Last updated: 2014/01/15

Latest News

AsiaBSDCon 2014

2014-03-05

Both Allan and Kris will be going to AsiaBSDCon this year, so episode 28 will be shorter than usual. We'll be back the following week with a huge episode. Hopefully they can get some interviews there!...

Christmas & New Year

2013-12-19

Episode 16 was just uploaded, and that's the last one we'll be doing live for this year. Episode 17 will be on Christmas, and feature a prerecorded interview with Scott Long about his BSD magic over at Netflix. Thanks for watching everyone! We look forward to more BSD Now in...

Welcome iXsystems

2013-12-11

As you may have noticed in Episode 015, BSD Now has gotten our first sponsor! We're very happy to welcome iXsystems to the BSD Now team. In case you aren't familiar with them, they have quite a long history with FreeBSD. Their current CTO is in fact Jordan Hubbard, one of...

Update 2013/11/26

2013-11-26

Hi BSD Now fans. Here's a sneak peek at our upcoming schedule: 11/27: Jordan Hubbard, co-founder of FreeBSD and creator of ports, to talk about FreeBSD's founding and future. The tutorial will be an update to the OpenBSD router guide with some new improvements I've made. 12/04: George Wilson from Delphix...


Episode 034: It's Gonna Get NASty

2014-04-23

Direct Download: Video | HD Video | MP3 Audio | OGG Audio | Torrent This episode was brought to you by Interview - John Hixson - john@ixsystems.com / @bsdwhore FreeNAS development All the tutorials are posted in their entirety at bsdnow.tv Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv If...

Episode 033: Certified Package Delivery

2014-04-16

Direct Download: Video | HD Video | MP3 Audio | OGG Audio | Torrent This episode was brought to you by Headlines BSDCan schedule, speakers and talks This year's BSDCan will kick off on May 14th in Ottawa The list of speakers is also out And finally the talks everyone's looking forward to Lots of great tutorials and...

Episode 032: PXE Dust

2014-04-09

Direct Download: Video | HD Video | MP3 Audio | OGG Audio | Torrent This episode was brought to you by Headlines FreeBSD ASLR status update Shawn Webb gives us a little update on his address space layout randomization work for FreeBSD He's implemented execbase randomization for position-independent executables (which OpenBSD also just enabled globally in...

Episode 031: Edgy BSD Users

2014-04-01

Direct Download: Video | HD Video | MP3 Audio | OGG Audio | Torrent This episode was brought to you by Headlines Preorders for cool BSD stuff The 2nd edition of The Design and Implementation of the FreeBSD Operating System is up for preorder We talked to GNN briefly about it, but he and Kirk have...