syscall error handling inside the linux kernel

Question

Hi, I'm writing a new syscall inside the linux kernel. I want to implement an error handling system to return to the user who uses the sycall a code which describes the type of the error. I'd like to know how this is done inside the kernel, since I want to follow the standard. I want to let the user read the errno variable, or something equivalent, to understand the error that happened. Thank you.

P.S. I'm using the latest versione of linux available.

Answer 1

I found that I can simply return the number of the code that I want errno to be set to by returning the code itself in the sycall. All the rest is made by the linux kernel automatically.

Answer 2

You have two options: use existing error codes, or define your own. The majority of the time I've found that existing error codes come close enough to the meaning I want.

Basic error numbers are found in:

include/asm-generic/errno-base.h

and

include/asm-generic/errno.h.

In your system call you can return to your user the negative value of an existing errno value. For example:

return -ENOMEM;

Answer 3

Much of your task is taken care of automatically by libc and the low-level kernel syscall handler (the part written in assembly). The kernel convention for handling error codes is to return a negative error constant, something like -ENOMEM. A zero or a positive number indicates success. This is used throughout the kernel.

If you've defined the new sycall entry point static asmlinkage long my_new_syscall (int param) {...}, it just needs to return -ENOMEM (or the like). If you invoked the system call by using libc's syscall(nr, param), then on an error it will return -1 and ENOMEM (in the positive) will be in errno.

There are many error codes defined in include/asm-generic/{errno,errno-base}.h that you can use (like ENOMEM). If none of those fit your purpose, you can add your own error number to those files, but be aware that you'll also need to modify the user-space-visible kernel headers to show the same number, so it will be more difficult to set up a system to use your modifications. Don't do this if you don't have to.