Please note that this tutorial is quite old. For a more up-to-date tutorial, please see this updated tutorial which I've created for version 4.4.1 of the Linux kernel, as used in my Spring 2016 section of this course.


Adding a Simple System Call to the Linux 3.0.4 Kernel


Introduction

This short tutorial will show you how to add a very simple system call to the Linux kernel, version 3.0.4. In this case, we will be adding a system call that simply prints "Hello World" to the kernel log. This article will also explain how to run the system call from a userland program. While the system call added is pretty much useless, it does demonstrate the basics of adding a system call, which is one of the tasks that will be done in Project 1 for this course.

While adding a simple system call to the Linux kernel is not difficult, there are a few files that need to be edited in order to make things work. This tutorial will focus on the x86 (32-bit) version of the kernel, but similar steps could be applied for other architectures.

Writing the "Hello World" system call

Writing the system call itself is pretty much the simplest part of this whole process, so we shall start there. First, create a new directory in the root of a clean copy of the kernel sources created for this tutorial. Call the new directory "hello". In this directory, we need to create two files. The first of which is the implementation of the system call itself, hello.c:

#include <linux/kernel.h>

asmlinkage long sys_hello(void) {
    printk("Hello World\n");
    return 0;
}

In kernel space, we do not have a direct printf that prints to stdout. While it would be possible to use the "write" system call to write to stdout, for the purposes of this tutorial, we leave this out. Instead, we use printk to print to the kernel log. printk works pretty much exactly as you would expect printf to do. There are a few things to notice about this simple .c file other than that. First of all, you do not have a full libc to work with in kernel space. You are much more limited in what you can and cannot use. See the kernel headers to find out more about that. Second, you should notice the asmlinkage tag in front of the return type in the function. This is required for system calls. Do not leave it out. In addition, it is common practice for a system call to return a status code of some sort, usually as a long. In this case, we simply return 0 to represent that the operation executed correctly. If we wanted to return an error, the standard way to do so is to return the negative of some value that you can find in the standard errno.h header file in libc.

After creating the system call, we need to set up the Makefile to build it. Add the following to a Makefile in the same directory as hello.c:

obj-y := hello.o

This is a very simple Makefile, since the build system of the kernel takes care of most of the work for you. This concludes the new files that will need to be added to the kernel sources to make a new system call.

Kernel changes needed to reflect the new system call

There are a few source files in the kernel that will need to be updated in order to reflect the new system call to be added to the kernel. The first, and simplest of these is the root-level Makefile. Find the following line, around line 711 of the root-level Makefile:

core-y		:= kernel/ mm/ fs/ ipc/ security/ crypto/ block/

Assuming you created the new hello directory, add a hello/ to the end, as shown:

core-y		:= kernel/ mm/ fs/ ipc/ security/ crypto/ block/ hello/

It would be a good idea to set the EXTRAVERSION in the Makefile as well, to differentiate it from your normal kernel.

Next, open the include/linux/syscalls.h file, and add a definition for your new system call. Right after the line mentioning sys_setns (around line 849 of the file), add the following:

asmlinkage long sys_hello(void);

This step makes your system call available to other parts of the kernel that may wish to call it. While this is not necessary for this simple system call, it is still good practice for when adding a real system call to the kernel.

Next, open the arch/x86/include/asm/unistd_32.h file, and search for the following lines:

#define __NR_setns		346

#ifdef __KERNEL__

#define NR_syscalls 347

Between the __NR_setns and #ifdef __KERNEL__ lines, add the following:

#define __NR_hello		347

In addition, update the NR_syscalls line to 348, since you have added a new system call.

The final file that needs to be updated is the system call table, which resides in arch/x86/kernel/syscall_table_32.S. Add the definition for your new system call, by adding the following:

	.long sys_hello

You should add this after the definition for the previous latest syscall, which should be a line that mentions sys_setns.

Congratulations, you have now added a new system call to the Linux kernel. Follow the normal steps to build the kernel, and fix any problems you may find in compilation (if you followed this exactly, you should not have any).

Testing your new system call from userland

The next step is to test out your system call to make sure it works. Be sure to reboot into your new kernel, then write the following in a new .c file (not in the kernel source tree):

#define _GNU_SOURCE

#include <stdio.h>
#include <unistd.h>
#include <linux/kernel.h>
#include <sys/syscall.h>

#define __NR_hello 347

long hello_syscall(void) {
    return syscall(__NR_hello);
}

int main(int argc, char *argv[]) {
    hello_syscall();
    return 0;
}

The main point of interest here is the syscall function. This simply performs a system call, and returns the result of the call. It accepts an integer argument that says which system call to perform, as well as any arguments the system call may need passed to it (it uses varargs for this, much like printf or scanf). Most of the rest of the program is fairly self-explanatory for anyone who knows C. Compile this program with your compiler (gcc) and run it. You should not see any output directly on your terminal, but if you open up the kernel log with the dmesg command, you should see your "Hello World" at the end.

Further exploration

You may wish to experiment further with this system call. For instance, make it print to stdout, instead of to the kernel log. To do so, you will need to use the sys_write system call in the kernel.