OpenBSD FAQ - Disk Setup [FAQ Index]



Disks and partitions

The details of setting up disks in OpenBSD vary between platforms, so you should read the instructions in your platform's INSTALL.<arch> file.

Drive identification

On most platforms, OpenBSD handles mass storage with two drivers: The devices are numbered by the order in which they are detected at boot time, starting at zero. So, the first IDE-like disk will be wd0 and the third SCSI-like disk will be sd2. Removing or adding a disk may impact the identity of other drives on the system. Note that OpenBSD will not necessarily number drives in the same order as your boot ROM.

Partitions and filesystems

The term "partition" can mean two different things in OpenBSD: All OpenBSD platforms use the disklabel program as the primary way to manage filesystem partitions. On the platforms that use fdisk, one MBR partition is used to hold all of the OpenBSD filesystems. This partition can be sliced into 16 disklabel partitions, labeled a through p.

Thus, a device name plus a disklabel identify an OpenBSD filesystem. For example, the identifier sd2a refers to the filesystem on the a partition of the third sd device. Its device files would be /dev/sd2a for the block device and /dev/rsd2a for the raw (character) device. Remembering whether a rarely used command needs a block or a character device is difficult. Therefore, many commands make use of the opendev(3) function, which automatically expands sd0 to /dev/rsd0c or wd0a to /dev/wd0a, as appropriate.

A few labels are special:

Disklabel Unique Identifiers

Disks can also be identified by Disklabel Unique Identifiers (DUIDs), a 16 hex digit number, managed by the diskmap(4) device. This number is a random number generated when a disklabel is first created. You can specify partitions on the disk by appending a period and the partition letter. For example, f18e359c8fa2522b.d is the d partition of the disk f18e359c8fa2522b and will always refer to the same chunk of storage, no matter what order the devices attached to the system, or what kind of interface it is attached to. If you put data on wd2d, then later remove wd1 from the system and reboot, your data is now on wd1d, as your old wd2 is now wd1. However, a drive's DUID won't change after boot.

Using fdisk

The fdisk(8) utility is used on some platforms (i386, amd64 and macppc) to create a partition recognized by the system boot ROM. Normally, only one OpenBSD fdisk partition will be placed on a disk and that partition will then be subdivided into disklabel partitions.

View your partition table with:

# fdisk sd0
Disk: sd0       geometry: 553/255/63 [8883945 Sectors]
Offset: 0       Signature: 0xAA55
         Starting       Ending       LBA Info:
 #: id    C   H  S -    C   H  S [       start:      size   ]
------------------------------------------------------------------------
 0: 12    0   1  1 -    2 254 63 [          63:       48132 ] Compaq Diag.
 1: 00    0   0  0 -    0   0  0 [           0:           0 ] unused
 2: 00    0   0  0 -    0   0  0 [           0:           0 ] unused
*3: A6    3   0  1 -  552 254 63 [       48195:     8835750 ] OpenBSD
Here, the OpenBSD partition (id A6) is marked with a * to indicate that it is the bootable partition.

A totally blank disk will need to have the master boot record's boot code written to the disk before it can boot. Normally, all you need to do is:

# fdisk -iy sd0
Alternatively, use the reinit or update commands in interactive mode.

The -e flag starts interactive editing mode:

# fdisk -e sd0
Enter 'help' for information
fdisk: 1>
Beware that q saves changes and exits the program, while x exits without saving. This is the opposite of what many people are now used to in other environments. Note also that fdisk does not warn before saving the changes.

If your system has a maintenance or diagnostic partition, it is recommended that you leave it in place or install it before installing OpenBSD.

What are disk labels?

Disk labels are used to manage OpenBSD filesystem partitions. They contain certain details about your disk, such as drive geometry and filesystem information, as described in depth in the disklabel(5) man page. Use the disklabel(8) command to edit the labels.

This can help overcome some architectures' disk partitioning limitations. For example, on i386, there are only four primary MBR partitions available. With disk labels, one of these primary partitions contains all your OpenBSD partitions, while the other three are still available for other operating systems.

On platforms using fdisk, you should leave the first logical track unused, both in disklabel and in fdisk. For this reason, the default is to start the first partition at block 64.

Don't put swap at the very beginning of your disk on sparc64. While Solaris often did that, OpenBSD requires the boot partition to be at the beginning of the disk.

Recovering partitions after deleting the disklabel

If you have a damaged partition table, there are various things you can attempt to do to recover it.

A copy of the disklabel for each disk is saved in /var/backups as part of the daily system maintenance. Assuming you still have the /var partition, you can simply read the output, and put it back into disklabel with the -R flag.

In the event that you can no longer see that partition, there are two options: Fix enough of the disk so you can see it, or fix enough of the disk so that you can get your data off. The scan_ffs(8) utility will look through a disk to find partitions. You can use the information it finds to recreate the disklabel. If you just want /var back, you can recreate the partition for /var, then recover the backed up label and add the rest from that. The disklabel(8) utility will both update the kernel's understanding of the disklabel and attempt to write the label to disk. Therefore, even if the area of the disk containing the disklabel is unreadable, you will be able to mount until the next reboot.

The amd64 boot process

Details on the amd64 bootstrapping procedure are given in the boot_amd64(8) man page. The boot process is as follows:
  1. Master Boot Record (MBR) and GUID Partition Table (GPT). The fdisk(8) man page contains the details.
  2. Partition Boot Record (PBR). The first 512 bytes of the boot disk's OpenBSD partition contain the first stage boot loader biosboot(8). It is installed by the installboot(8) utility.
  3. Second stage boot loader /boot. The PBR loads the boot(8) program which has the task of locating and loading the kernel.
So, the very start of the boot process could look like this:
Using drive 0, partition 3.                      <- MBR
Loading....                                      <- PBR
probing: pc0 com0 com1 apm mem[636k 190M a20=on] <- /boot
disk: fd0 hd0+
>> OpenBSD/i386 BOOT 3.26
boot>
booting hd0a:/bsd 4464500+838332 [58+204240+181750]=0x56cfd0
entry point at 0x100120

[ using 386464 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993       <- Kernel
        The Regents of the University of California.  All rights reserved.

Soft updates

Soft updates are based on an idea proposed by Greg Ganger and Yale Patt and developed for FreeBSD by Kirk McKusick. Soft updates imposes a partial ordering on the buffer cache operations which permits the requirement for synchronous writing of directory entries to be removed from the FFS code. A large increase is seen in diskwriting performance as a result.

Enabling soft updates must be done with a mount-time option. When mounting a partition with the mount(8) utility, you can specify that you wish to have soft updates enabled on that partition. Below is a sample fstab(5) entry that has one partition sd0a that we wish to have mounted with soft updates.

/dev/sd0a / ffs rw,softdep 1 1

Duplicating your root partition: /altroot

OpenBSD provides an /altroot facility in the daily(8) scripts. If the environment variable ROOTBACKUP=1 is set in either /etc/daily.local or root's crontab(5), and a partition is specified in fstab(5) as mounting to /altroot with the mount options of xx, every night the entire contents of the root partition will be duplicated to the /altroot partition.

Assuming you want to back up your root partition to the partition specified by the DUID bfb4775bb8397569.a, add the following to /etc/fstab

bfb4775bb8397569.a /altroot ffs xx 0 0
and set the appropriate environment variable in /etc/daily.local:
# echo ROOTBACKUP=1 >>/etc/daily.local
As the /altroot process will capture your /etc directory, this will make sure any configuration changes there are updated daily. This is a "disk image" copy done with dd(1) not a file-by-file copy, so your /altroot partition should be at least the same size as your root partition. Generally, you will want your /altroot partition to be on a different disk that has been configured to be fully bootable should the primary disk fail.

Can I access data on filesystems other than FFS?

Yes. Start with the mount(8) manual which contains examples explaining how to mount some of the most commonly used filesystems. A partial list of supported filesystems and related commands can be obtained with
$ man -k -s 8 mount
Note that support may be limited to read-only operation.

Mounting disk images

To mount a disk image in OpenBSD you must configure a vnd(4) device. For example, if you have an ISO image located at /tmp/ISO.image, you would take the following steps to mount the image.
# vnconfig vnd0 /tmp/ISO.image
# mount -t cd9660 /dev/vnd0c /mnt
Since this is an ISO 9660 image, as used by CDs and DVDs, you must specify type of cd9660 when mounting it.

To unmount the image and unconfigure the vnd(4) device, do:

# umount /mnt
# vnconfig -u vnd0
For more information, refer to vnconfig(8) and mount(8).

Why does df tell me I have over 100% of my disk used?

People are sometimes surprised to find they have negative available disk space, or more than 100% of a filesystem in use, as shown by df(1).

When a filesystem is created with newfs(8), some of the available space is held in reserve from normal users. This provides a margin of error when you accidentally fill the disk, and helps keep disk fragmentation to a minimum. Default for this is 5% of the disk capacity, so if the root user has been carelessly filling the disk, you may see up to 105% of the available capacity in use.

If the 5% value is not appropriate for you, you can change it with the tunefs(8) command.

RAID and disk encryption

The softraid(4) subsystem works by emulating a scsibus(4) with sd(4) devices made by combining a number of OpenBSD disklabel(8) partitions into a virtual disk with the desired RAID level. This virtual disk is treated as any other disk, first partitioned with fdisk (on fdisk platforms) and then disklabels are created as usual. Note that only RAID0, RAID1, RAID5 and crypto are fully supported at the moment.

Installing to a mirror

This section covers installing OpenBSD to a mirrored pair of hard drives, and assumes familiarity with the installation process. Disk setup may vary from platform to platform, and booting from softraid devices isn't supported on all of them. It's currently only possible to boot from RAID1, RAID5 and crypto volumes on i386, amd64 and sparc64.

Before using the install script, you will drop to a shell and set up a softraid(4) device.

The install kernel only has the /dev entries for one wd(4) device and one sd(4) device on boot, so you will need to manually create more disk devices if your desired softraid setup requires them. For example, if you need to support a second and third sd(4) device for a mirrored setup, you could do the following from the shell prompt:

Welcome to the OpenBSD/amd64 X.X installation program.
(I)nstall, (U)pgrade, (A)utoinstall or (S)hell? s
# cd /dev
# sh MAKEDEV sd1 sd2
You now have full support for the sd0, the sd1 and sd2 devices.

Next, we'll initialize the disks with fdisk(8) and create RAID partitions with disklabel(8).

If you're booting from MBR, do:

# fdisk -iy sd0
# fdisk -iy sd1
If you use GPT for UEFI booting, do:
# fdisk -iy -g -b 960 sd0
# fdisk -iy -g -b 960 sd1
Create the partition layout on the first device:
# disklabel -E sd0
Label editor (enter '?' for help at any prompt)
> a a
offset: [64]
size: [39825135] *
FS type: [4.2BSD] RAID
> w
> q
No label changes.
Copy the partition layout to the second device:
# disklabel sd0 > layout
# disklabel -R sd1 layout
# rm layout
Assemble the mirror with the bioctl(8) command:
# bioctl -c 1 -l sd0a,sd1a softraid0
scsibus1 at softraid0: 1 targets
sd2 at scsibus2 targ 0 lun 0: <OPENBSD, SR RAID 1, 005> SCSI2 0/direct fixed
sd2: 10244MB, 512 bytes/sec, 20980362 sec total
This shows that we now have a new SCSI bus and a new disk, sd2. This volume will be automatically detected and assembled when the system boots.

Even if you create multiple RAID arrays, the device name will always be softraid0. There won't be a softraid1 or anything else.

Because the new device probably has a lot of garbage where you expect a master boot record and disklabel, zeroing the first chunk of it is highly recommended. Be very careful with this command; issuing it on the wrong device could lead to a very bad day. This assumes that the new softraid device was created as sd2.

# dd if=/dev/zero of=/dev/rsd2c bs=1m count=1
You are now ready to install OpenBSD on your system. Perform the install as normal by invoking "install" or "exit" at the boot media console. Create all the partitions on your new softraid disk (sd2 in our example here) that should be there, rather than on sd0 or sd1 (the non-RAID disks).

To check on the status of your mirror, issue the following command:

# bioctl sd2
A nightly cron job to check the status might be a good idea.

Rebuilding a mirror

When a drive failure happens, you will replace the failed drive, create the RAID and other disklabel partitions, then rebuild the mirror. Assuming your RAID volume is sd2 and you are replacing the failed device with sd1m, the following commands should work:
# bioctl -R /dev/sd1m sd2
# reboot
These steps can be performed in either single user mode or from the install kernel.

Full disk encryption

Much like RAID, full disk encryption in OpenBSD is handled by the softraid(4) subsystem and bioctl(8) command. This section covers installing OpenBSD to a single encrypted disk, and is a very similar process to the previous one. Note that using both RAID and disk encryption on the same device is not currently supported.

Select (S)hell at the initial prompt.

Welcome to the OpenBSD/amd64 X.X installation program.
(I)nstall, (U)pgrade, (A)utoinstall or (S)hell? s
From here, you'll be given a shell within the live environment to manipulate the disks. For this example, we will install to the sd0 SATA drive, erasing all of its previous contents. You may want to write random data to the drive first with something like the following:
# dd if=/dev/random of=/dev/rsd0c bs=1m
This can be a very time-consuming process, depending on the speed of your CPU and disk, as well as the size of the disk. If you don't write random data to the whole device, it may be possible for an adversary to deduce how much space is actually being used.

Next, we'll initialize the disk with fdisk(8) and create the softraid partition with disklabel(8).

If you're booting from MBR, do:

# fdisk -iy sd0
If you use GPT for UEFI booting, do:
# fdisk -iy -g -b 960 sd0
Next, create the partition layout:
# disklabel -E sd0
Label editor (enter '?' for help at any prompt)
> a a			
offset: [64]
size: [39825135] *
FS type: [4.2BSD] RAID
> w
> q
No label changes.
We'll use the entire disk, but note that the encrypted device can be split up into multiple partitions as if it were a regular hard drive.

Now we can build the encrypted device on our "a" partition.

# bioctl -c C -l sd0a softraid0
New passphrase:
Re-type passphrase:
sd1 at scsibus2 targ 1 lun 0: <OPENBSD, SR CRYPTO, 005> SCSI2 0/direct fixed
sd1: 19445MB, 512 bytes/sector, 39824607 sectors
softraid0: CRYPTO volume attached as sd1
Since the installer does not have many device nodes by default, we'll make sure the /dev/sd1 device is accounted for.
# cd /dev && sh MAKEDEV sd1
All data written to sd1 will now be encrypted with AES in XTS mode.

As in the previous example, we'll overwrite the first megabyte of our new pseudo-device.

# dd if=/dev/zero of=/dev/rsd1c bs=1m count=1
Type exit to return to the main installer, then choose this new device as the one for your installation.
[...]
Available disks are: sd0 sd1.
Which disk is the root disk? ('?' for details) [sd0] sd1
You will be prompted for the passphrase on startup, but all other operations should be handled transparently.

Encrypting external disks

Cryptographic softraid volumes can be set up rather simply. This section explains how you might do so for an external USB flash drive, but can be applied to any disk device. If you already read the section on full disk encryption, this should be very familiar. An outline of the steps is as follows: A quick example runthrough of the steps follows, with sd3 being the USB drive.
# dd if=/dev/random of=/dev/rsd3c bs=1m
# fdisk -iy sd3
# disklabel -E sd3 (create an "a" partition, see above for more info)
# bioctl -c C -l sd3a softraid0
New passphrase:
Re-type passphrase:
softraid0: CRYPTO volume attached as sd4
# dd if=/dev/zero of=/dev/rsd4c bs=1m count=1
# fdisk -iy sd4
# disklabel -E sd4 (create an "i" partition, see above for more info)
# newfs sd4i
# mkdir -p /mnt/secretstuff
# mount /dev/sd4i /mnt/secretstuff
# mv somefile /mnt/secretstuff/
# umount /mnt/secretstuff
# bioctl -d sd4
Next time you need to access the drive, simply use bioctl(8) to attach it and then repeat the last four commands as needed.

The man page for this looks a little scary, as the -d command is described as "deleting" the volume. In the case of crypto, however, it just deactivates the encrypted volume so it can't be accessed until it is activated again with the passphrase.