In this module we will learn about device drivers, which is how Linux (as well as other kernels) handles peripheral devices. We will go through the different categories of devices, how applications interact with them and introduce the concept of device node as an interface to devices. Finally, we will focus on one specific type of device and write a simple driver for it.

Device nodes

From the point of view of Linux, there are three types of devices:

  • Character devices, which are sequential streams. They can be thought of as files and as such can be used in conjunction with the open, close, read and write functions provided by POSIX. An example would be a serial port like /dev/ttyS0.
  • Block devices, which are randomly accessed only in block-size multiples, and for which I/O operations are usually cached and deal with mounted filesystems. An example would be a hard drive partition like /dev/sda1.
  • Network devices, which transfer packets of data, and usually implement a socket interface. Packet reception/transmission functions replace read/write operations. An example with be a network interface like eth0 or wlan0.

Character and block devices have filesystem entries associated with them, called device nodes. These nodes behave as if they were regular files from the point of view of a user and can be used by user-level programs to communicate with the underlying device by using I/O system calls. Device nodes are normally placed in the /dev directory.

Let us consider a simple example and request information about the /dev/sda1 device node.

The first thing to notice is the b character before the permission bits. This indicates that the file is a block device.

Right after the group we can see two numbers, 8 and 1. These are referred to as the major and minor number, respectively.

  • The major number identifies the driver associated with the device: all device nodes of the same type (block or character) with the same number use the same driver.
  • The minor number is used only by the driver to differentiate between the different devices it may control. These may either be different instances of the same device (such as the first and second hard drive partition) or different modes of operation of a given device.
    • Notice how all these devices have the same major number (and therefore are controlled by the same driver) but different minor numbers.

Device nodes can be explicitly created using the mknod program:

$ mknod [-m mode] /dev/name <type> <major> <minor>

Alternatively, the mknod() system call can be used:

int mknod(const char* pathname, mode_t mode, dev_t dev);

Device drivers

At its core, a Linux driver is “nothing more” than a set of functions that can be called by the kernel every time it is necessary to interact with a peripheral device. Linux drivers are almost always written in C, although Rust is starting to make its way into the kernel.

We will start our study of drivers by writing a simple character device driver and will progressively add more features to it. The basic skeleton of a Linux device driver is as follows:

#include <linux/init.h>
#include <linux/module.h>

static int cool_driver_init(void) {
  printk(KERN_ALERT "COOL DRIVER: init\n");
  return 0;
}

static void cool_driver_exit(void) {
  printk(KERN_ALERT "COOL DRIVER: exit\n");
}

module_init(cool_driver_init);
module_exit(cool_driver_exit);

MODULE_AUTHOR("Modular MX");
MODULE_LICENSE("GPL v2");

This constitutes the bare minimum to make a fully functional device driver.

Drivers cannot make use of the C standard library (because, well… it is not available at system startup). Therefore, they must use the interfaces provided by the Linux kernel itself. One such utility is the printk function, which is similar to printf. The KERN_ALERT prefix is a preprocessor macro that specific the priority of the message (it is possible to filter out messages during the initialization sequence).

The two functions, cool_driver_init and cool_driver_exit, are called by the kernel when the driver is loaded and when it is removed, respectively. At this point we are not doing anything particularly interesting here. The macros module_init and module_exit are used to tell the kernel the roles of these functions.

MODULE_AUTHOR and MODULE_LICENSE provide extra information about the driver. Be aware that if you do not specify the license of the driver, or if it is a non-free license, the kernel will complain when loading the driver. (The driver will load, but it will taint the kernel.)

  • Go to your copy of the Linux source tree and create a file in drivers/char named cool_driver.c. Copy the previous code to it.
  • Edit drivers/char/Kconfig and add an appropriate configuration entry for the driver.
config COOL_DRIVER
  tristate "My cool driver"
  default n
  help
    This is my cool driver
    • We will go over the exact meaning of the tristate keyword in a few minutes.
  • Edit drivers/char/Makefile and add an appropriate entry for the object file associated with the driver.
obj-$(CONFIG_COOL_DRIVER) += cool_driver.o
  • Run armmake menuconfig and find the new entry under Device Drivers/Character devices. Enable it (that is, press Y).
  • Save the new configuration and rebuild the kernel image.
$  armmake zImage -j $(nproc)
  • Boot the BeagleBone Black with this kernel image. You should see the COOL DRIVER: init message being displayed as part of the logs during the initialization sequence.

In this case, the driver is embedded into the kernel itself, that is, its code is part of the zImage file that is generated during the build process and then loaded by U-Boot. It is also possible to build a driver as a module, in which case it is built as a separate file that can be dynamically loaded (or “inserted”) into the kernel at runtime.

  • Run armmake menuconfig and locate the entry that corresponds to our driver. Press M to indicate that it will be built as a module.
    • This is why we needed to specify tristate in the Kconfig file (that is, the entry can have one of three possible states). The alternative would be bool.
  • Save the new configuration and rebuild the kernel. You will notice that the compiler itself is not involved here; instead, we are relinking object files without the definitions provided by cool_driver.o.
  • Build the modules by running armmake modules. You will notice that only cool_driver.o is built.
    • File drivers/char/cool_driver.ko is the kernel module. If you request information from it using the file utility, you will notice that it is a regular relocatable object file.
  • Install the new module tree to the root filesystem used by the BeagleBone Black using armmake module_install. Remember to specify a value for INSTALL_MOD_PATH!
  • Boot the BeagleBone Black. You can see that the driver was not loaded this time by running the dmesg program, which prints the messages reported by the kernel. (Entry from the manual)
    $ dmesg | grep 'COOL DRIVER'
  • To work with kernel modules, you can use the lsmod, insmod and rmmod utilities.lsmod shows a list of the modules currently loaded in the kernel.insmod can be used to insert a module into the kernel.rmmod can be used to remove a module from the kernel.
  • Examine the output of lsmod. Insert our driver with insmod. Check the output of lsmod again and you should see our driver listed there.
$ lsmod
(...)
$ sudo insmod /lib/modules/6.11.2/kernel/drivers/char/cool_driver.ko
$ lsmod
Module         Size   Used by
cool driver   12288   0
(...)

This driver can be made a bit more interesting by introducing the file_operations data structure, which is a type defined by the kernel that contains function pointers that will be called when a user-space application interacts with a device node managed by the driver. This type looks like this (note that this is not the entire definition):

struct file_operations {
  int (*open)(struct inode*, struct file*);
  int (*release)(struct inode*, struct file*);
  ssize_t (*read)(struct file*, char __user*, size_t, loff_t*);
  ssize_t (*write)(struct file*, const char __user*, size_t, loff_t*);
};

A driver can define functions that match these signatures and then create a variable of type file_operations, initializing its members to the appropriate functions. After registering this instance of file_operations with the device driver, the functions will be called by the kernel.

As an example, let us assume that we have a character device node called /dev/my_device. If we do something like echo 'hello there' >/dev/my_device, the kernel will first call the open function associated with the driver that manages the node, and then write. The string hello there will be passed as the second argument to write.

The only thing that is missing at this point is how to actually register a driver with the kernel so that it can manage device nodes. There are a couple of functions provided by Linux that can be used for this purpose.

  • int register_chrdev_region(dev_t from, unsigned int count, const char* name);
    This function will register a range of device numbers to be used with the driver from which it is called.
  • void cdev_init(struct cdev* cdev, const struct file_operations* fops);
    Initialize the data structure used by the kernel to represent a character device, associating a group of file_operations with it.

Building on our first version of a character device driver, it can be extended as follows:

#include <linux/init.h>
#include <linux/module.h>

#include <linux/fs.h>
#include <linux/cdev.h>

static const char* const device_name = "cool_driver";

static const int major_number = 400;
static const int minor_number = 0;

static struct cdev* character_device;
static dev_t first_device_number;

static int cool_driver_open(struct inode* inode, struct file* file) {
	printk(KERN_ALERT "COOL DRIVER: open\n");
	return 0;
}

static int cool_driver_release(struct inode* inode, struct file* file) {
	printk(KERN_ALERT "COOL DRIVER: release\n");
	return 0;
}

static ssize_t cool_driver_read(struct file* file, char __user* buffer,
                                size_t count, loff_t* position) {
	printk(KERN_ALERT "COOL DRIVER: read (%u bytes)\n", count);
	return (ssize_t) count;
}

static ssize_t cool_driver_write(struct file* file, const char __user* buffer,
                                 size_t length, loff_t* count) {
	printk(KERN_ALERT "COOL DRIVER: write (%u bytes)\n", length);
	return (ssize_t) length;
}

static const struct file_operations cool_driver_file_operations = {
	.owner = THIS_MODULE,
	.open = cool_driver_open,
	.release = cool_driver_release,
	.read = cool_driver_read,
	.write = cool_driver_write
};

static int cool_driver_init(void) {
	printk(KERN_ALERT "COOL DRIVER: init\n");

	first_device_number = MKDEV(major_number, minor_number);
	register_chrdev_region(first_device_number, 1, device_name);

	character_device = cdev_alloc();
	cdev_init(character_device, &cool_driver_file_operations);
	cdev_add(character_device, first_device_number, 1);

	return 0;
}

static void cool_driver_exit(void) {
	printk(KERN_ALERT "COOL DRIVER: exit\n");

	cdev_del(character_device);
	unregister_chrdev_region(first_device_number, 1);
}

module_init(cool_driver_init);
module_exit(cool_driver_exit);

MODULE_AUTHOR("Modular MX");
MODULE_LICENSE("GPL v2");
  • Rebuild this driver as a module as install it to the root filesystem used by the BeagleBone Black.
  • Boot the board and insert the module using lsmod.
  • Create a character device node with appropriate major and minor numbers so that it is managed by our driver.
$ sudo mknod -m 666 /dev/my_node c 400 0
    • Feel free to inspect this file with ls to confirm that the numbers are correct.
  • Write something to the device node. You will see that open, write and close are called, in that order.
$ echo 'hello there' >/dev/my_node
[ ... ] COOL DRIVER: open
[ ... ] COOL DRIVER: write (12 bytes)
[ ... ] COOL DRIVER: close

Allocation of device numbers

The Linux kernel uses the dev_t type to represent device numbers, that is, both the major and minor numbers. Its internal representation is meant to be an implementation-specific detail, but usually it is a 32-bit quantity that uses 12 bits for the major number and 20 for the minor number. Users are expected to manipulate these quantities through the following macros:

  • MAJOR(dev): Extracts the major number from a dev_t quantity.
  • MINOR(dev): Extracts the minor number from a dev_t quantity.
  • MKDEV(major, minor): Create a dev_t quantity out of a major and minor number.

Under most circumstances, one of the first things that a Linux driver will do during initialization is to obtain one or more device numbers, which are needed to work with the devices managed by the driver. There are two main approaches for this:

  • Static allocation of device numbers through the following function:
int register_chrdev_region(dev_t first, unsigned int count, const char* name);
    • The first parameter, which also happens to be called first, is the beginning device number of the range to allocate. It is often the case that MINOR(first) is 0, but this is not a requirement.
    • The second parameter, count, is the number of contiguous device number that are being requested.
    • The third parameter, name, is the name of the device that should be associated with the number range and will appear in /proc/devices.
    • The return type will be 0 if the allocation was successfully performed, and a negative number in case of an error.
  • Dynamic allocation of device numbers through the following function:
int alloc_chrdev_region(dev_t* dev, unsigned int first_minor, unsigned int count, const char* name);
    • The first parameter is output-only and will, on successful completion, hold the first number in the allocated range.
    • The second parameter is the first minor number to use in the allocated range; it is usually 0.
    • The third and fourth parameters are similar to those provided to register_chrdev_region. The same applies to the function’s return type.

Regardless of the approach used to allocate device numbers, they should be freed when no longer necessary:

void unregister_chrdev_region(dev_t first, unsigned int count);

he kernel uses the type cdev to represent character devices. It is a struct whose internals are meant to be an implementation detail. Before the kernel invokes any operation associated with a character device (like read or write), it is necessary to allocate and register one of these struct objects, which can be done in two ways:

  • By initializing a cdev object that already exists:
void cdev_init(struct cdev* cdev, const struct file_operations* fops);
  • By obtaining a standalone cdev object at runtime:
struct cdev* cdev_alloc(void);
    • Note that if only this function is used, the file_operations must be directly set with something like dev->ops = &my_device_file_operations;.

The cdev data structure has another field that should also be initialized: owner should be set to THIS_MODULE.

Once the cdev data structure is set up, it can be registered with the kernel by using the following function:

int cdev_add(struct cdev* cdev, dev_t first_device_number, unsigned int count);

As soon as cdev_add returns (with a successful return code) the device is “up and running” and its operations can be called by the kernel. This means that it is important to only call this function after the driver is completely ready to handle operations on the device.

Removing a character device from the kernel is done by calling the following function:

void cdev_del(struct cdev* cdev);

Drivers and memory management

If a Linux driver requires dynamic memory allocation, it can use the kmalloc and kfree functions provided by the kernel.

void* kmalloc(size_t size, int flags);
void kfree(const void* address);

This interface should be familiar to anyone who has used malloc and free.

One obvious difference is the flags parameter for kmalloc, though. It is used to indicate the kernel how the memory is supposed to be used, and in turn control the behavior of the allocator used by the kernel.

An in-depth discussion of the available options for this parameter is outside the scope of this module. You are encouraged to consult the official documentation. In practice, GFP_KERNEL is used most of the time, but be aware that there are more options which could be useful in certain scenarios.

The region allocated by kmalloc is contiguous in physical memory. This is important when, for example, allocating a buffer that can be accessed by a DMA device on a physically addressed bus.

The kernel provides another option for allocating memory in the form of vmalloc, which allocates a contiguous region in the virtual address space. The pages associated with this region don’t have to necessarily be contiguous in physical memory, but the kernel sees them as a contiguous range of addresses. Memory allocated with vmalloc should be freed with vfree.

void* vmalloc(unsigned long size);
void vfree(const void* address);

Addresses allocated by vmalloc cannot be used outside of the microprocessor, because they make sense only on top of the processor’s MMU. A good rule of thumb is to use vmalloc when allocating memory for a large sequential buffer that exists only in software.

Closing thoughts

It is not humanly possible to cover all the available information about Linux device drivers in the limited amount of time we have. This module was meant to provide an introduction to the subject and (hopefully) give you enough context so that you can further investigate.

Linux is a moving target in many ways, so books written about it tend to become outdated in no time. That being said, the basic idea behind the way the kernel interacts with drivers is somewhat stable. If you would like to know more about drivers in general, a good starting point is the third edition of Linux Device Drivers. The authors have graciously made the chapters freely available online, but if you find the information in that book relevant, consider supporting them by actually buying it.

Labs

  • Extend the character device driver we wrote to “remember” the data that is written into it. Requesting data from the driver should reset its state.
$ sudo mknod -m 666 /dev/my_node c 400 0
$ printf hello >/dev/my_node
$ printf ' there' >/dev/my_node
$ printf '!\n' >/dev/my_node
$ cat /dev/my_node
hello there!
$ printf 'first line\n' >/dev/my_node
$ printf 'second line\n' >/dev/my_node
$ cat /dev/my_node
first line
second line
  • So far, we have been building the driver inside the Linux source tree (that is, in-tree). Figure out how to do an out-of-tree build. Is there anything in particular that needs to be taken into consideration when doing so?
    • Hint: The interfaces used internally by Linux are not guaranteed to be stable, so it is always a good idea to build a module using the header files that correspond to the particular version of the kernel for which they are meant to be a part.