I'm trying to read/write the size of multiple structs to a .dat file. I am getting compile errors, what am I doing wrong? (C++)

Question

I've already asked 2 questions kind of related to this project, and i've reached this conclusion. Writing the size of the Struct to the file , and then reading it back is the best way to do this.

I'm creating a program for a homework assignment that will allow me to maintain inventory. I need to read / write multiple structs of the same type to a file.

The problem is... this is really complicated and i'm having trouble wrapping my head around the whole process. I've seen a bunch of examples and i'm trying to put it all together. I'm getting compile errors... and I have zero clue on how to fix them. If you could help me on this I would be so appreciative... thank you. I'm so lost right now...

** HOPEFULLY THE LAST EDIT #3 ***********

My Code:

// Project 5.cpp : main project file.

#include "stdafx.h"
#include <iostream>
#include <fstream>
#include <string>
#include <vector>
#include <algorithm>

using namespace System;
using namespace std;
#pragma hdrstop

int checkCommand (string line);

template<typename Template>
void readFromFile(Template&);

template<typename Template>
void writeToFile(Template&);

template<typename T>
void writeVector(ofstream &out, const vector<T> &vec);

template<typename Template>
void readVector(ifstream& in, vector<Template>& vec);

struct InventoryItem {
    string Item;
    string Description;
    int Quantity;
    int wholesaleCost;
    int retailCost;
    int dateAdded;
} ;


int main(void)
{
    cout << "Welcome to the Inventory Manager extreme! [Version 1.0]" << endl;

    vector<InventoryItem> structList;

    ofstream out("data.dat");

    writeVector( out, structList );

    while (1)
    {

     string line = "";

     cout << endl;
     cout << "Commands: " << endl;
     cout << "1: Add a new record " << endl;
     cout << "2: Display a record " << endl;
     cout << "3: Edit a current record " << endl;
     cout << "4: Exit the program " << endl;
     cout << endl;
     cout << "Enter a command 1-4: ";

     getline(cin , line);


     int rValue = checkCommand(line);
     if (rValue == 1)
     {
      cout << "You've entered a invalid command! Try Again." << endl;
     } else if (rValue == 2){ 
      cout << "Error calling command!" << endl;
     } else if (!rValue) {
      break;
     }
    }


    system("pause");

    return 0;
}

int checkCommand (string line)
{
    int intReturn = atoi(line.c_str());
    int status = 3;

    switch (intReturn)
    {
     case 1:
      break;
     case 2:
      break;
     case 3:
      break;
     case 4:
      status = 0;
      break;
     default:
      status = 1;
      break;
    }
    return status;
}

template <typename Template>
void readFromFile(Template& t)
{
    ifstream in("data.dat");
    readVector(in, t); Need to figure out how to pass the vector structList via a Template
    in.close();
}

template <typename Template>
void writeToFile(Template& t)
{
    ofstream out("data.dat");
    readVector(out, t); Need to figure out how to pass the vector structList via a Template
    out.close();
}

template<typename T>
void writeVector(ofstream &out, const vector<T> &vec)
{
    out << vec.size();

    for(vector<T>::const_iterator i = vec.begin(); i != vec.end(); ++i)
    {
        out << *i; // SUPER long compile error
    }
}

template<typename T>
vector<T> readVector(ifstream &in)
{
    size_t size;
    in >> size;

    vector<T> vec;
    vec.reserve(size);

    for(int i = 0; i < size; ++i)
    {
        T tmp;
        in >> tmp;
        vec.push_back(tmp);
    }

    return vec;
}

My Compile Errors:

1>.\Project 5.cpp(128) : error C2679: binary '<<' : no operator found which takes a right-hand operand of type 'const InventoryItem' (or there is no acceptable conversion)
1>        C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\include\ostream(653): could be 'std::basic_ostream<_Elem,_Traits> &std::operator <<<char,std::char_traits<char>>(std::basic_ostream<_Elem,_Traits> &,const char *)'
1>        with

That is the only error i'm getting now. I see your code is SO Much better. My new compiler error is SUPER long. I've shown where it the error points to. Could you help me just one last time?

Answer 1

EDIT: Trying to clear up FUD:

bind1st is part of STL's functional header. STL existed before boost showed up. It is deprecated in C++0x in favor of the more generic version i.e. bind (aka boost::bind). See Annex D.8 Binders for more information.

Now the real problem (multiple edits may make this look silly, but I'll keep this for posterity's sake):

 write<long>(out, structList.size());

This is the offending line. This expects a long as the second parameter, whereas the vector's size() is of type size_t or unsigned int under the hoods.

Update there was a typo: use size_t and not size_T:

write<size_t>(out, structList.size());

Next part:

 for_each(structList.begin(), structList.end(), bind1st(write<InventoryItem>, out));

This should be structList or some other type. Also, include functional to be able to use bind1st. Add at the top:

#include <functional>

The template bind1st takes a functor. Passing around ordinary function pointers is not possible without some other hacks. You can use boost::bind as an alternative. Or:

for(InventoryItem::iterator i = structList.begin(), f = structList.end();
         i != f; ++i)
    write<InventoryItem>(out, *i);

Now for other nitpicks:

What is:

#include <String>
...
using namespace System;

Are you sure of what you are using here? If you want STL strings you need to include:

#include <string>

void main(void)

is not a standard signature. Use any one of:

int main(void)

or

int main(int argc, char *argv[]);

I/O is usually much easier with the predefined insertion/extraction operators. You can (and really should) use:

istream is(...);
is >> data;

and similarly

ostream os(...);
os << data;

Note also your readFromFile and writeToFile functions need to be fixed to use vector<InventoryItem> instead of vector simply.

Answer 2

Your read and write functions are buggy. In particular, you should be doing something like this instead:

template<typename T>
void write(ofstream &out, const T &t)
{
    out << T;
}

OLD: bind1st requires you do include functional for it to work:

#include <functional>

Instead of dealing with all these functions and such, though, it'd be better to rely on iterators:

template<typename T>
void writeVector(ofstream &out, const vector<T> &vec)
{
    out << vec.size();

    for(vector<T>::const_iterator i = vec.begin(); i != vec.end(); ++i)
    {
        out << *i;
    }
}

template<typename T>
vector<T> readVector(ifstream &in)
{
    size_t size;
    in >> size;

    vector<T> vec;
    vec.reserve(size);

    for(int i = 0; i < size; ++i)
    {
        T tmp;
        in >> tmp;
        vec.push_back(tmp);
    }

    return vec;
}

You'd want functions to read and write your InventoryItem as well, probably:

ostream &operator<<(ostream &out, const InventoryItem &i)
{
    out << i.Item << i.Description;  // FIXME Read/write strings properly.
    out << i.Quantity;
    out << i.wholesaleCost << i.retailCost;
    out << i.dateAdded;
}

istream &operator>>(istream &out, InventoryItem &i)
{
    // Keep in same order as operator<<(ostream &, const InventoryItem &)!
    in >> i.Item >> i.Description;  // FIXME Read/write strings properly.
    in >> i.Quantity;
    in >> i.wholesaleCost >> i.retailCost;
    in >> i.dateAdded;
}

Answer 3

NOTE: This is not an answer to the compilation errors you are getting, but rather a broader view of the persistence problem you are handling.

Serialization and deserialization is not the simplest problem you can work on. My advice would be investing in learning libraries (boost::serialization) and using them. They have already worked out many of the problems you will face at one time or another. Plus they already have different output formats (binary, xml, json...)

The first thing you must decide, that is if you decide to go ahead and implement your own, is what will be the file format and whether it suits all your needs. Will it always be used in the same environment? Will the platform change (32/64bits)? You can decide to make it binary as it is the simplest, or make it readable for a human being. If you decide on XML, JSON or any other more complex formats, just forget it and use a library.

The simplest solution is working on a binary file and it is also the solution that will give you a smallest file. On the other hand, it is quite sensible to architecture changes (say you migrate from a 32 to a 64 bit architecture/OS)

After deciding the format you will need to work on the extra information that is not part of your objects now but needs to be inserted into the file for later retrieval. Then start working (and testing) from the smallest parts to more complex elements.

Another advice would be to start working with the simplest most defined part and build from there on. Start avoiding templates as much as possible, and once you have it clear and working for a given data type, work on how to generalize it for any other type.

Disclaimer: I have written the code directly on the browser, so there could be some errors, typos or just about anything :)

Text

The first simple approach is just writting a textual representation of the text. The advantage is that it is portable and shorter in code (if not simpler) than the binary approach. The resulting files will be bigger but user readable.

At this point you need to know how reading text works with iostreams. Particularly, whenever you try to read a string the system will read characters until it reaches a separator. This means that the following code:

std::string str;
std::cin >> str;

will only read up to the first space, tab or end of line. When reading numbers (ints as an example) the system will read all valid digits up to the first non-valid digit. That is:

int i;
std::cin >> i;

with input 12345a will consume all characters up to 'a'. You need to know this because that will influence the way you persist data for later retrieval.

// input: "This is a long Description"
std::string str;
std::cin >> str; // Will read 'This' but ignore the rest

int a = 1;
int b = 2;
std::cout << a << b; // will produce '12'
// input: 12
int read;
std::cint >> read; // will read 12, not 1

So you pretty much need separators to insert in the output and to parse the input. For sample purposes I will select the '|' character. It must be a character that does not appear in the text fields.

It will also be a good idea to not only separate elements but also add some extra info (size of the vector). For the elements in the vector you can decide to use a different separator. If you want to be able to read the file manually you can use '\n' so that each item is in its own line

namespace textual {
   std::ostream & operator<<( std::ostream& o, InventoryItem const & data )
   {
      return o << data.Item << "|" << data.Description << "|" << data.Quantity
         << "|" << data. ...;
   }
   std::ostream & operator<<( std::ostream & o, std::vector<InventoryItem> const & v )
   {
      o << v.size() << std::endl;
      for ( int i = 0; i < v.size(); ++i ) {
         o << v[i] << std::endl; // will call the above defined operator<<
      }
   }
}

For reading, you will need to split the input by '\n' to get each element and then with '|' to parse the InventoryItem:

namespace textual {
   template <typename T>
   void parse( std::string const & str, T & data )
   {
      std::istringstream st( str ); // Create a stream with the string
      st >> data;  // use operator>>( std::istream
   }

   std::istream & operator>>( std::istream & i, InventoryItem & data )
   {
      getline( i, data.Item, '|' );
      getline( i, data.Description, '|' );

      std::string tmp;
      getline( i, tmp, '|' ); // Quantity in text
      parse( tmp, data.Quantity );
      getline( i, tmp, '|' ); // wholesaleCost in text
      parse( tmp, data. wholesaleCost );
      // ...
      return i;
   }

   std::istream & operator>>( std::istream & i, std::vector<InventoryItem> & data )
   {
      int size;

      std::string tmp;
      getline( i, tmp ); // size line, without last parameter getline splits by lines
      parse( tmp, size ); // obtain size as string

      for ( int i = 0; i < size; ++i )
      {
         InventoryItem data;
         getline( i, tmp ); // read an inventory line
         parse( tmp, data );
      }    
      return i;  
   }
}

In the vector reading function I have used getline + parse to read the integer. That is to guarantee that the next getline() will actually read the first InventoryItem and not the trailing '\n' after the size.

The most important piece of code there is the 'parse' template that is able to convert from a string to any type that has the insertion operator defined. It can be used to read primitive types, library types (string, for example), and user types that have the operator defined. We use it to simplify the rest of the code quite a bit.

Binary

For a binary format, (ignoring architecture, this will be a pain in the ass if you migrate) the simplest way I can think of is writing the number of elemements in the vector as a size_t (whatever the size is in your implementation), followed by all the elements. Each element will printout the binary representation of each of its members. For basic types as int, it will just output the binary format of the int. For strings we will resort to writting a size_t number with the number of characters in the string followed by the contents of the string.

namespace binary
{
   void write( std::ofstream & o, std::string const & str )
   {
      int size = str.size();
      o.write( &size, sizeof(int) ); // write the size
      o.write( str.c_str(), size ); // write the contents
   }
   template <typename T>
   void write_pod( std::ofstream & o, T data ) // will work only with POD data and not arrays
   {
      o.write( &data, sizeof( data ) );
   }
   void write( std::ofstream & o, InventoryItem const & data )
   {
      write( o, data.Item );
      write( o, data.Description );
      write_pod( o, data.Quantity );
      write_pod( o, data. ...
   }
   void write( std::ofstream & o, std::vector<InventoryItem> const & v )
   {
      int size = v.size();
      o.write( &size, sizeof( size ) ); // could use the template: write_pod( o, size )
      for ( int i = 0; i < v.size(); ++i ) {
         write( o, v[ i ] );
      }
   }
}

I have selected a different name for the template that writes basic types than the functions that write strings or InventoryItems. The reason is that we don't want to later on by mistake use the template to write a complex type (i.e. UserInfo containing strings) that will store an erroneous representation in disk.

Retrieval from disk should be fairly similar:

namespace binary {
   template <typename T>
   void read_pod( std::istream & i, T& data)
   {
      i.read( &data, sizeof(data) );
   }
   void read( std::istream & i, std::string & str )
   {
      int size;
      read_pod( i, size );
      char* buffer = new char[size+1]; // create a temporary buffer and read into it
      i.read( buffer, size );
      buffer[size] = 0;
      str = buffer;
      delete [] buffer;
   }
   void read( std::istream & i, InventoryItem & data )
   {
      read( i, data.Item );
      read( i, data.Description );
      read( i, data.Quantity );
      read( i, ...
   }
   void read( std::istream & i, std::vector< InventoryItem > & v )
   {
      v.clear(); // clear the vector in case it is not empty

      int size;
      read_pod( i, size );
      for ( int i = 0; i < size; ++i )
      {
         InventoryItem item;
         read( i, item );
         v.push_back( item );
      }
   }
}

For using this approach, the std::istream and std::ostream must be opened in binary mode.

int main()
{
   std::ifstream persisted( "file.bin", ios:in|ios::binary );
   std::vector<InventoryItem> v;
   binary::read( persisted, v );

   // work on data

   std::ofstream persist( "output.bin", ios::out|ios::binary );
   binary::write( persist, v );
}

All error checking is left as an exercise for the reader :)

If you have any question on any part of the code, just ask.