"short read" from filesystem, when can it happen?

Question

It is obvious that in general the read(2) system call can return less bytes than what was asked to be read. However, quite a few programs assume that when working with a local files, read(2) never returns less than what was asked (unless the file is shorter, of course).

So, my question is: on Linux, in which cases can read(2) return less than what was requested if reading from an open file and EOF is not encountered and the amount being read is a few kilobytes at maximum?

Some guesses:

• Can received signals interrupt a read like that, but not make it fail?
• Can different filesystems affect this behavior? Is there anything special about jffs2?

Answer 1

A received signal only makes read() fail if it hasn't yet read a single byte. Otherwise, it will return partial data.

And I guess alternate filesystems may indeed return short reads in other situations. For example, it makes some sense (to me) to have a network-based filesystem behave just like a network socket wrt short reads (= having them often).

Answer 2

I have to ask: "why do you care about the reason"? If read can return a number of bytes less than the requested amount (which, as you point out, it certainly can) why would you not want to deal with that situation?

Answer 3

If it's really a file you are reading, then you can get short read as the last read before end of file.

Howver, it's generally best to behave as if ANY read could be a short read. If what you are reading is a pipe or an input device (stdin) rather than a file, you can get a short read whenever your buffer is larger than what is currently in the input buffer.

Answer 4

POSIX.1-2008 states:

The value returned may be less than nbyte if the number of bytes left in the file is less than nbyte, if the read() request was interrupted by a signal, or if the file is a pipe or FIFO or special file and has fewer than nbyte bytes immediately available for reading.

Disk-based filesystems generally use uninterruptible reads, which means that the read operation generally cannot be interrupted by a signal. Network-based filesystems sometimes use interruptible reads, which can return partial data or no data. (In the case of NFS this is configurable using the intr mount option.) They sometimes also implement timeouts.

Keep in mind that even /some/arbitrary/file/path may refer to a FIFO or special file, so what you thought was a regular file may not be. It is therefore good practice to handle partial reads even though they may be unlikely.

Answer 5

I am not sure but this situation could arise when the OS is running out of pages in the page cache. You could suggest that flush thread will be invoked in that case, but it depends on the heuristic used in the I/O scheduler. This situation could cause a read to return fewer bytes.