views:

2892

answers:

8

For testing purposes I have to generate a file of a certain size (to test an upload limit).

What is a command to create a file of a certain size on Linux?

+5  A: 

Use this command:

dd if=$INPUT-FILE of=$OUTPUT-FILE bs=$BLOCK-SIZE count=$NUM-BLOCKS

To create a big (empty) file, set $INPUT-FILE=/dev/zero.
Total size of the file will be $BLOCK-SIZE * $NUM-BLOCKS.
New file created will be $OUTPUT-FILE.

Grundlefleck
Why did you ask the question?
PintSizedCat
I had to Google for the answer, so I put it here, so it could be discussed and kept up to date... you know, the point of the whole site?
Grundlefleck
I know people are voting @Grundlefleck down for XP whoring, but he does have a point - one of the ways to use this site as envisioned by Jeff and Joel is to put a question and answer for something you just discovered.
Paul Tomblin
Thanks Paul. Though I'm not so bothered about the points, I am bothered about things I find on Google which may be flawed in some way I'd never find out about unless I asked here. People should feel free to make my Q/A community owned if they think I'm whoring, *shrugs*.
Grundlefleck
To quote from the faq's "It's also perfectly fine to ask and answer your own programming question, but pretend you're on Jeopardy: phrase it in the form of a question."
Craig Angus
You might consider first checking if `$OUTPUT_FILE` already exists. Overwriting /etc/passwd with a 2GB file of zeros is discouraging ;-)
Boldewyn
@Boldewyn: it's a good point, but I think using the command responsibly is implied ;-p
Grundlefleck
+8  A: 
dd if=/dev/zero of=my_file.txt count=12345
Paul Tomblin
Remember that dd's default block size is 512 bytes, so this command will make a file 12345*512 bytes in size.
Andrew Medico
+1  A: 

you could do:

[dsm@localhost:~]$ perl -e 'print "\0" x 100' > filename.ext

Where you replace 100 with the number of bytes you want written.

dsm
+18  A: 
dd if=/dev/zero of=upload_test bs=file_size count=1

Where file_size is the size of your test file in bytes

Ilya Kochetov
Oh, that might be more efficient than my approach because it does it all in one block. Good idea.
Paul Tomblin
Actually, using a huge blocksize will perform much worse once it gets very big, as it will allocate and read that amount into memory before writing. If this is somethig like bs=4GiB you'll probably end up swapping.
Brian
A: 

The trouble with the approaches using dd and perl is that they actually have to write every byte of the file. Using an approach like this you wouldn't have to.

#include <stdio.h>

int main ()
{
  FILE * pFile;
  int filesize = 12345;
  pFile = fopen ( "myfile.txt" , "w" );
  fseek ( pFile , filesize , SEEK_SET );
  fputs ( "" , pFile );
  fclose ( pFile );
  return 0;
}

This will just allocate a large file and doesn't actually need to write many bytes to do so.

Disclaimer: I have not compiled or tested this code.

Tom
Does Linux have support for sparse files? Would this allow you to make a file that appears to have N bytes, but which actually takes fewer on the disk? I know some Unixes can do that.
Paul Tomblin
This doesn't work, it creates an empty file. You can try to write something at size-1 then replace it with NUL.
Flame
Another option if you're dropping to C is to use the truncate() or ftruncate() calls.
Brian
+10  A: 

Just to follow up Tom's post, you can use dd to create sparse files as well:

dd if=/dev/zero of=the_file bs=1 count=0 seek=12345

This will create a file with a "hole" in it on most unixes - the data won't actually be written to disk, or take up any space until something other than zero is written into it.

Brian
Setting count=0 avoids having to subtract a byte from the file size.
andrew
So it does. Thanks - I've updated it.
Brian
+5  A: 

You can do it programmatically:

#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdlib.h>

int main() {
    int fd = creat("/tmp/foo.txt", 0644);
    ftruncate(fd, SIZE_IN_BYTES);
    close(fd);
    return 0;
}

This proceeding is especially useful to subsequently mmap the file into memory.

use the following command to check that the file has the correct size:
# du -B1 --apparent-size /tmp/foo.txt

But:
# du /tmp/foo.txt
will return 0 because it is allocated as Sparse file.

see also: man 2 open and man 2 truncate

bene
+1 Thank you, simple easy and to the point of what I needed.
tristopia
A: 

I suppose the most efficient approach would involve creating the file using open (with the O_CREAT and O_WRONLY flags, unless you want to read from the file in the same program) and setting the size with ftruncate.

spyked