tags:

views:

220

answers:

7

In scripting languages like Perl, it is possible to read a file into a variable in one shot.

    open(FILEHANDLE,$file);
    $content=<FILEHANDLE>;

What would be the most efficient way to do this in C++?

A: 

maybe not the most efficient, but reads data in one line:

#include<iostream>
#include<vector>
#include<iterator>

main(int argc,char *argv[]){
  // read standard input into vector:
  std::vector<char>v(std::istream_iterator<char>(std::cin),
                     std::istream_iterator<char>());
  std::cout << "read " << v.size() << "chars\n";
}
catwalk
+1  A: 

Here's an iterator-based method.

ifstream file("file", ios::binary);
string fileStr;

istreambuf_iterator<char> inputIt(file), emptyInputIt
back_insert_iterator<string> stringInsert(fileStr);

copy(inputIt, emptyInputIt, stringInsert);
academicRobot
A: 

There should be no \0 in text files.

#include<iostream>
#include<fstream>

using namespace std;

int main(){
  fstream f(FILENAME, fstream::in );
  string s;
  getline( f, s, '\0');

  cout << s << endl;
  f.close();
}
Draco Ater
The question didn't mention text files.
anon
-1: what Neil said.
Felix
-1 This example only reads one line, I have to wonder about the moderations.
piotr
@piotr This example reads the whole text file, it is tested.
Draco Ater
I think everybody assume it would be a text file but it's true it may not be the case.As far as the code goes: maybe it's clearer to directly use ifstream( "filename" ). You don't need to close the file, it is done automatically. And it does read the text file.
Nikko
@Draco Ater I tested with a binary file, probably it did read only until \0. The point is that this example is going to process every character, I prefer iterator based solutions that may be more efficient.
piotr
+5  A: 

The most efficient, but not the C++ way would be:

   FILE* f = fopen(filename, "r");

   // Determine file size
   fseek(f, 0, SEEK_END);
   size_t size = ftell(f);

   char* where = new char[size];

   rewind(f);
   fread(where, sizeof(char), size, f);

   delete[] where;

#EDIT - 2

Just tested the std::filebuf variant also. Looks like it can be called the best C++ approach, even though it's not quite a C++ approach, but more a wrapper. Anyway, here is the chunk of code that works almost as fast as plain C does.

   std::ifstream file(filename, std::ios::binary);
   std::streambuf* raw_buffer = file.rdbuf();

   char* block = new char[size];
   raw_buffer->sgetn(block, size);
   delete[] block;

I've done a quick benchmark here and the results are following. Test was done on reading a 65536K binary file with appropriate (std::ios:binary and rb) modes.

[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 4 tests from IO
[ RUN      ] IO.C_Kotti
[       OK ] IO.C_Kotti (78 ms)
[ RUN      ] IO.CPP_Nikko
[       OK ] IO.CPP_Nikko (106 ms)
[ RUN      ] IO.CPP_Beckmann
[       OK ] IO.CPP_Beckmann (1891 ms)
[ RUN      ] IO.CPP_Neil
[       OK ] IO.CPP_Neil (234 ms)
[----------] 4 tests from IO (2309 ms total)

[----------] Global test environment tear-down
[==========] 4 tests from 1 test case ran. (2309 ms total)
[  PASSED  ] 4 tests.
Kotti
+1 this was going to be my answer. Although, you should add the allocation of `where` for the example to be more clear: `char *where = malloc(sizeof(char) * size);`
Felix
As far as I know, there is a chance that std::filebuf in the C++ library is more efficient
Nikko
Nice benchmark, I'm not surprised by the numbers. For max performance on plain ascii files using good old C io is the way to go. C++ streams are just no match. However, they are less error prone. As long as they are not showing up when profiling I'd prefer using them.
Maik Beckmann
You have to try with std::filebuf s :)
Nikko
Wow, it's actually cool. I don't know why, but at first I didn't trust you. Looks like the best way to combine `iostream` functionality and raw C file reading speed.
Kotti
Heh.. plain C is STILL faster ;)
Felix
+13  A: 

Like this:

#include <fstream>
#include <string>

int main(int argc, char** argv)
{

  std::ifstream ifs("myfile.txt");
  std::string content( (std::istreambuf_iterator<char>(ifs) ),
                       (std::istreambuf_iterator<char>()    ) );

  return 0;
}

The statement

  std::string content( (std::istreambuf_iterator<char>(ifs) ),
                       (std::istreambuf_iterator<char>()    ) );

can be split into

std::string content;
content.assign( (std::istreambuf_iterator<char>(ifs) ),
                (std::istreambuf_iterator<char>()    ) );

which is useful if you want to just overwrite the value of an existing std::string variable.

Maik Beckmann
+1 Very C++ idiomatic. In Linux with gcc 4.4 the resulting system calls are efficient, the file is read 8k at a time.
piotr
If the file size is known, the `std::string::reserve` method can be called before reading the file to allocate space. This should speed up the execution. Much time is lost by reallocating memory for the string.
Thomas Matthews
+1  A: 

This depends on a lot of things, such as what is the size of the file, what is its type (text/binary) etc. Some time ago I benchmarked the following function against versions using streambuf iterators - it was about twice as fast:

unsigned int FileRead( std::istream & is, std::vector <char> & buff ) {
    is.read( &buff[0], buff.size() );
    return is.gcount();
}

void FileRead( std::ifstream & ifs, string & s ) {
    const unsigned int BUFSIZE = 64 * 1024; // reasoable sized buffer
    std::vector <char> buffer( BUFSIZE );

    while( unsigned int n = FileRead( ifs, buffer ) ) {
        s.append( &buffer[0], n );
    }
}
anon
+1  A: 

The most efficient is to create a buffer of the correct size and then read the file into the buffer.

#include <fstream>
#include <vector>

int main()
{
    std::ifstream       file("Plop");
    if (file)
    {
        /*
         * Get the size of the file
         */
        file.seekg(0,std::ios::end);
        std::streampos          length = file.tellg();
        file.seekg(0,std::ios::beg);

        /*
         * Use a vector as the buffer.
         * It is exception safe and will be tidied up correctly.
         * This constructor creates a buffer of the correct length.
         * Because char is a POD data type it is not initialized.
         *
         * Then read the whole file into the buffer.
         */
        std::vector<char>       buffer(length);
        file.read(&buffer[0],length);
    }
}
Martin York