reading from file certain lines at a time in c/c++

Question

Hello,

So i have a gui, designed using QT, c++. I have large amount of data in a text file that I would like to read in this fashion:

load first 50 lines, when the user scrolls down load next 50 lines and so one. When the user scrolls up load previous 50 lines.

Thank you.

Answer 1

Connect your app to the sliderMoved or sliderReleased signals of your QSlider and read the 50 lines you want to display. Or use the valueChange event.

Answer 2

The easiest solution would be to load the file into memory and manipulate it from there:

std::vector<std::string>    lines;

std::string  line;
while(std::getline(file,line)
{
    lines.push_back(line);
}

If the file is way to large.
Then you need to build an index of the file that tells you exactly where each line starts.

std::vector<std::streampos>    index;
index.push_back(file.tellg());

std::string  line;
while(std::getline(file,line)
{
    index.push_back(file.tellg());
}
file.setg(0);
file.clear();  // Resets the EOF flag.

Once you have your index. You can jump around the file and read any particular line.

int jumpTo = 50;
file.seekg(index[jumpTo]); // Jump to line 50.
//
// Read 50 lines. Do not read past the end
// This will set the EOF flag and future reads will fail.
for(int loop=0;loop < 50 && ((jumpTo + loop) < index.size());++loop)
{
    std::string line;
    std::getline(file,line);
}

Answer 3

I am guessing that this file may be huge (say in the Gigabyte range).

Assuming that portability is not an issue, you may want to use memory mapped io. On many unix systems you have the mmap command.

A nice tutorial can be found here.

If you need to do this for windows systems A windows equivalent example can be found here.

What mmap effectively does is maps the file into the current processes memory space. When one wants to read a bit of the file, the OS copies the bits of the file being looked at into RAM for the user to access. This is done using the same mechanism that allows the OS to move memory from idle processes onto disk until it is required again.

The only tricky bit will be dividing the file blocks into lines.