Why do du and Perl's -s give different values for the file size?

Question

Updated According to comments:

I have outlog.txt file which contains multiple filenames, e.g: 2345_535_Dell&HP_3PAR_DEAL.txt, similarly there are many filename but not the actual folder where the files are located and so in code am appending filenames to folderpath to get actual file location. Now,I want to get disk usage of all the files present in outlog.txt and also total disk usage of all files present in outlog.txt.

I have tried two approaches perl -s and my ($size) = split(' ', du `"$folderpath/$_"`) but both approaches are giving me different values and also when I am using du than am getting some numeric value but it does not give me unit, is there a way I can get human readable without using -h option as it is not working on my system ?

Background Information

My goal is to get the size of a file, and currently I am using perl -s to get filesize. I have also tried du and am getting different values for the size of the same file. I am not able to understand how this works.

Q: Why do du and perl -s give different values for size? How do they internally work? Which of the two different values is the more accurate one? Also, I'm not sure why du -h filename gives me an illegal expression error:

bash-2.03$ du -h test.txt
/usr/bin/du: illegal option -- h
usage: du [-a][-d][-k][-r][-o|-s][-L] [file ...]

Code:

my $folderpath = 'the_path';
open my $IN, '<', 'path/to/infile';
my $total;
while (<$IN>) {
    chomp;
    my $size = -s "$folderpath/$_";
    print "$_ => $size\n";
    $total += $size;
}
print "Total => $total\n";

Courtesy: RickF

Update:

Q: How can I get the disk usage value instead of file size for each file present, meaning how can I get du value for each files rather than perl -s values for file ?

OS Information uname :SunOS uname -v :Generic_117350-39

Updated Code: According to brain's approach but still du value prints as zero only and not the actual value, any suggestions ?

Update: If I use my ($size) = split(' ', du "$folderpath/$_"); than I am getting du value but it gives me some number, how can I get it into human readable without using -h option ?

 #!/usr/bin/perl
 use strict;
 use warnings;

my $folderpath = '/Project/upload';
open my $IN, '<', 'outlog.txt';
my $total;
while (<$IN>) {
    chomp;
    #my( $block_size, $blocks ) = ( stat( "$_" ) )[11,12];
    #my $du_size = $block_size * $blocks;
    my ($size) = split(' ', `du "$folderpath/$_"); 
    #my $size = -s "$folderpath/$_";
    print "$_ => $size\n";
    $total += $size;
}
print "Total => $total\n";

Answer 1

du stands for "Disk Used", and reports the physical size of the file on disk. If the file is sparse, this may be much smaller than its logical size, which is what -s reports. Both are "accurate", they're just measuring different things.

The error message indicates that the version of du installed on your machine doesn't understand the -h option.

Answer 2

du reports actual disk usage, Perl's -s reports the size of the file. So, if a file is four bytes long it will have a size of four bytes, but disk usage of four kilobytes (depending on how your filesystem is setup).

You will also see a difference in the sizes of sparse files. Sparse files take up less space than they claim to have in them.

Answer 3

By default, du displays the number of blocks used by a file (where each block is 512 bytes on most systems), while perl's -s displays bytes.

As to why your copy of du doesn't have a -h option, you don't tell us what operating system you're using; it appears to include a horribly outdated version of the program.

UPDATE: to get disk usage in perl, you can use the Filesys::DiskUsage module.

Answer 4

If you want du to give the same results as Perl's -s, try du -b. If your du supports it, this gives the "apparent size", which is different from disk usage, as others have stated.

But to do this you'll have to update your du.

Update for OP's updated code: Make sure that the file exists within your current working directory. You may have to prepend the directory to make sure that Perl is finding the file.

It may also clarify things if you get away from using $_ everywhere:

while( my $line = <$IN> ) {
  chomp $line;
  my( $block_size, $blocks ) = ( stat( $line ) )[11,12];
  ...
}

This way you're safe from unintentional changes to $_.

Answer 5

Everyone else has already noted that the file size and disk use are different beasts. However, if you want to get the same answer as du, you can use the stat to get the block size and the number of blocks that a file uses:

From Perl, to get the disk used (blocks unavailable for use for other files):

 # to match du
 my( $block_size, $blocks ) = ( stat( $file ) )[11,12];
 my $du_size = $block_size * $blocks;