File I/O in C++

Question

I have a data file where I need to read a datum from each line and store it. And then depending on the value of one of those datums store that data in an array so that I can then calculate the median value of all of these data.

The line of data is demographic information and depending on the geographic location, address of a person. I need to capture their age and then find the median of the people that live on a particular street for example.

So the data set is 150,000 records and each record has 26 fields, a lot of those fields are segments of an address and then the other fields are just numbers, age, street number and that sort of thing.

So what I need to do is read through the line and then if a particular field in the record meets a certain condition I need to capture a field from the record and store it in an array so that I can calculate the median of people that live on "Oak Street" for example.

I have the conditional logic and can work the sort out but I'm uncomfortable with the iostream objects in C++, like instantiating an ifstream object and then reading from the file itself.

Oh I forgot that the data was a comma separated value file.

Answer 1

For comma-delimited input:

using namespace std;
ifstream file;
string line;
while(getline(file, line)) {
    istringstream stream(line);
    string data[3];
    for(int ii = 0; ii < sizeof data / sizeof data[0]; ++ii)
        if(!getline(stream, data[ii], ','))
            throw std::runtime_error("invalid data");
    // process data here
}

For whitespace-delimited input (original answer):

using namespace std;
ifstream file;
string line;
while(getline(file, line)) {
    int datum1;
    string datum2;
    double datum3;
    istringstream stream(line);
    if(!(line >> datum1 >> datum2 >> datum3))
        throw std::runtime_error("invalid data");
    // process data here
}

These methods won't win any prizes for performance, but hopefully they're fairly reliable and easy to understand.

Answer 2

This sounds like a perfect problem for an SQL light style embedded data base. Then you could have any number of standard SQL features without having to rewrite the wheel.