Will blocks of a recently-created subsequently deleted file ever be written back to disc?

Question

Suppose I have a process which creates a file, writes some data to it, then after a short amount of processing (by itself or another process), deletes it and closes all remaining file descriptors.

I am assuming here that there is enough ram to keep the pages in memory until the file is deleted, and that nobody calls sync() in the interim.

Will the blocks of the now-deleted file ever be written back to disc, or are they immediately removed from the dirty-list?

Or does it depend on filesystem? Filesystems like xfs and ext4 have "delayed allocation" which might support this feature, if it is implement.

Answer 1

In the classic Unix file systems, the answer would be "No" (that is, the data for a created and deleted file would not necessarily ever make it to the disk), though some of the directory metadata (modification time) would probably still change. Therefore, what happens does depend in part on the file system in use.

Note that even calling sync() doesn't guarantee that they are written; it only schedules the writing of the data back to disk. Hence the ancient injunction to type the sync command twice before bringing down the system - doing that gave the computer enough time to complete the writing because it can write to disk faster than you can type sync twice (especially if you happen to be using a real Teletype at 110 baud).

---

The POSIX standard says (of the sync() function which is used by the sync command):

The sync() function shall cause all information in memory that updates file systems to be scheduled for writing out to all file systems.

The writing, although scheduled, is not necessarily complete upon return from sync().

If Linux has changed its definition to assure you that 'all data is written to disk', then that is a valid and useful extension. But it isn't the classic behaviour - and beware translating the Linux expertise to other systems.

There are other functions, such as fsync(), that give different, more stringent, promises:

The fsync() function shall request that all data for the open file descriptor named by fildes is to be transferred to the storage device associated with the file described by fildes. The nature of the transfer is implementation-defined. The fsync() function shall not return until the system has completed that action or until an error is detected.

And there are options to file descriptors that give other promises again: O_SYNC, O_DSYNC, O_RSYNC. Look them up in the POSIX standard (open()).

Answer 2

I agree with Jonathan Leffler, but not only for classic Unix file systems: There has been a discussion with a similar topic concerning the ext4 file system.

In a comment, Theodore Ts'o (one of the main developers of the ext4 file system) states: ``... for example, if you create a scratch file, and then delete it 20 seconds later, it will probably never hit the disk.''

Answer 3

I did some research on this and found that, under Linux, it does indeed depend on the filesystem.

ext3 seems to always write back deleted files no matter how small they are or how soon they are deleted. XFS does not always do so, which can result in much better performance for applications which use short-lived temporary files if you have enough ram.

I suspect that the "modern" Linux filesystems (ext4, btrfs) do this too. It's a good thing.

Answer 4

What do you really need to know, here?

If the question is "will it probably be written to disk?" the answer is no, if your processing is brief, but no promises.

If the question is "can I be sure that it won't be written to disk?" the answer is also no. A deleted file is a file like any other as long as it's held open; it's just a file without a name (link).

If the answer is "is it totally free in terms of the disk?" the answer is no again -- for example I'm pretty certain that on a system with quotas, the number of "blocks" in the file would be charged against the user's filesystem quota as soon as you write them.