Encrypted filesystems and the Linux buffer cache

Question

I currently use Berkeley DBs fronted by a Java server for a high-performance disk-backed cache. Provided you warm it up before allowing it to face live traffic, your update rate is low, and your working set fits in memory, the Linux buffer cache does an excellent job. It's measurably faster than memcache, in part because you don't need to context switch to the memcached and back on read. We're very happy with the performance.

We're going to be adding some data to the cache that we're not comfortable leaving on disk in plain text. We've measured and are unhappy with the performance of decrypting during request processing, so we're looking for solutions that decrypt only when the data is loaded from disk and then keep it available in memory.

Before building something that does this, I wanted to find out if we can simply slide in an encrypted filesystem and continue to rely on the OS to manage the cache for us. I haven't found any documentation that tells me at what layer the decryption is done.

So my question is: Can anyone tell me, for any particular Linux encrypted FS, whether the (en|de)cryption is done below the buffer cache (and therefore the cache contains plaintext) or above (and the cache contains ciphertext)?

Answer 1

I haven't played with this, but am pretty sure that buffer cache and VM are not aware of the encryption, so you should see comparable performance with your usage.

Answer 2

The buffer cache sits below the actual filesystem, so it will cache encrypted data. See the diagram at IBM's Anatomy of a Filesystem. Since you want to cache unencrypted data, so long as your encrypted filesystem was created using the 'loop' device the buffer cache will also contain an unencrypted copy of your data, and so it should be fast (at the cost of more memory for FS buffers in-use).