Number of inodes in a partition not matching up to the maximum number of inodes the partition should support

Question

We are using Amazon EBS to store a large number of small files (<10KB) in a 3-level directory structure.

 ~/lists# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             9.9G  3.9G  5.5G  42% /
tmpfs                 854M     0  854M   0% /lib/init/rw
varrun                854M   64K  854M   1% /var/run
varlock               854M     0  854M   0% /var/lock
udev                  854M   80K  854M   1% /dev
tmpfs                 854M     0  854M   0% /dev/shm
/dev/sda2             147G   80G   60G  58% /mnt
/dev/sdj              197G   60G  128G  32% /vol

The partition in question is /vol (size: 200GB)

 ~/lists# df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/sda1             655360   26541  628819    5% /
tmpfs                 186059       3  186056    1% /lib/init/rw
varrun                186059      31  186028    1% /var/run
varlock               186059       2  186057    1% /var/lock
udev                  186059     824  185235    1% /dev
tmpfs                 186059       1  186058    1% /dev/shm
/dev/sda2            19546112 17573097 1973015   90% /mnt
/dev/sdj             13107200 13107200       0  100% /vol

 ~/lists# sudo /sbin/dumpe2fs /dev/sdj | grep "Block size"
dumpe2fs 1.41.4 (27-Jan-2009)
Block size:               4096

The number of inodes for the partition /vol are 13Million+. The block size is 4096. Taking the Block Size as 4096, the number of inodes the 200GB partition (ext3) should support is 52million+ (Maximum Inode Calculation: Volume size in bytes/2^12). So why does the partition only support 13million inode?

Answer 1

I'm pretty sure that inodes are allocated statically when you create the volume (using mfs.ext3 in this case). For whatever reason, mkfs.ext3 decided to reserve 13 Million inodes and now you can't create any more files.

See this 2001 discussion of inodes

The Wikipedia ext3 page has a footnote explaining this more concisely: wiki link

Also, inodes are allocated per file (not block), which is why there are only 13M inodes - mkfs.ext3 must have been configured with an average file size of 8 KB which would account for the issue you're seeing.