Fastest way to create large file in c++ ?

Question

Create a flat text file in c++ say 50 - 100 MB say the content like 'Added first line' should be inserted in to the file for 40 lakhs time

Answer 1

using old style file io

fopen the file for write.

fseek to the desired file size - 1.

fwrite a single byte

fclose the file

Answer 2

The fastest way to create a file of a certain size is to simply create a zero-length file using creat() or open() and then change the size using chsize(). This will simply allocate blocks on the disk for the file, the contents will be whatever happened to be in those blocks(). It's very fast since no buffer writing needs to take place.

Answer 3

Not sure I understand the question. Do you want to ensure that every character in the file is a printable ASCII character? If so, what about this? Fills the file with "abcdefghabc...."

#include <stdio.h>
int main ()
{
   const int FILE_SiZE = 50000; //size in KB
   const int BUFFER_SIZE = 1024;
   char buffer [BUFFER_SIZE + 1];
   int i;
   for(i = 0; i < BUFFER_SIZE; i++)
      buffer[i] = (char)(i%8 + 'a');
   buffer[BUFFER_SIZE] = '\0';

   FILE *pFile = fopen ("somefile.txt", "w");
   for (i = 0; i < FILE_SIZE; i++)
     fprintf(pFile, buffer);

   fclose(pFile);

   return 0;
}

Answer 4

You haven't mentioned the OS but I'll assume creat/open/close/write are available.

For truly efficient writing and assuming, say, a 4k page and disk block size and a repeated string:

1. open the file.
2. allocate 4k * number of chars in your repeated string, ideally aligned to a page boundary.
3. print repeated string into the memory 4k times, filling the blocks precisely.
4. Use write() to write out the blocks to disk as many times as necessary. You may wish to write a partial piece for the last block to get the size to come out right.
5. close the file.

This bypasses the buffering of fopen() and friends, which is good and bad: their buffering means that they're nice and fast, but they are still not going to be as efficient as this, which has no overhead of working with the buffer.

This can easily be written in C++ or C, but does assume that you're going to use POSIX calls rather than iostream or stdio for efficiency's sake, so it's outside the core library specification.