store run-time variables in C++

Question

This is about a C++ problem. I have an object tracking program that takes images from 0,...,n in a loop. At current frame the computations are based on previous frames, therefore I need to hold those variables, matrices, etc for later use. This program has to be integrated now into another system which will provide an image and I have to return the tracking output. The system does later other processes, so my program has to become function to distribute as DLL.

I need to store my variables and matrices from previous images in order to use them again. I don't know if the best practice is to write them in hard drive and read them again in another instance. If this is the case what is the best way and data type/file to write/read. The systems aims to be real-time.

Thanks

Answer 1

One thing you could look into that IS NOT THREADSAFE is to keep the local variables as static. If you're not familiar with C/C++ static variables, they are stored in the global memory space and "remembered" between function calls. They're like global variables but can only be accessed by the function they're declared in. Run this a couple of times and see what happens.

void foo()
{
  static int x=0;
  x++;
  cout << x << endl;
}

Remember, you cannot have multiple threads call foo because there's only one state now!

Alternatively you could do something where you create a struct that holds a copy of your local state and you pass that in.

struct state
{
  int x
};

void bar(state& s)
{
  s.x++;
  cout << s.x << endl;
}

Answer 2

Not really an answer, just a longer request for details.

Any kind of data persistence issue involves decisions such as:

• required lifetime: app-controlled, thread, process, until host reboot, indefinite...?
• how many concurrent readers/writers will there be for the repository
• how are the readers/writers spaced across networks / hardware (e.g. endian issues, latencies)

You really haven't provided enough detail to make a serious start on this.

If your general hunch is right and a file is a suitable mechanism, you might consider whether memory mapping works well with your requirements... it tends to be faster than streamed file I/O.

Alternatives include a shared memory segment (can live longer than the creating process), heap...?

If your real interest is in serialisation mechanisms, you might have a look at boosts.

Anyway, I'm off home so it'll probably be someone else who answers....

Answer 3

It depends on your platform, but these days rare is the platform that doesn't have oodles of memory to spare. So if you are just saving data from a previous pass, no matter how much, my first go at it would be to save it all in memory somewhere.

If you end up running out of space, my second go would be to look into getting more RAM for your system. If it costs an extra $100, and you aren't making thousands of units, then it may save you money in the long run over engineering hours.

If not, then you can worry about the extra complexity of trying to save and restore from disk in realtime.

Answer 4

Some ideas and suggestions:

Storing the images or attributes of the images?

In general, an image will take up more space than attributes or data calculated from the image. Perhaps you can store the attributes of the images rather than the whole image.

Cache the data

Put as much data in memory as possible, store the rest in a file. The issue to be answered is how much of information must be in memory (such as the last N items or perhaps the first N items).

Multi (task or thread)

Have one thread that caches the images on demand. As the images are received, it puts either the image into memory (or the attributes). When the fixed memory area fills up, it places the image (or attributes) onto external memory (e.g. files). The main thread would request images from the caching thread. After the caching thread removes an image, it replaces that image (or attributes) with one datum from the file. The thread can sleep until either a new image comes its way or an older image is requested.

Answer 5

Hey Guys thanks a lot, I got some ideas already from you:

1. STL
2. memory-mapped files
3. Multi-Threads

I will start working with these solutions and let's see what are the restrictions of each one with the requirement of real-time.

I will come back to post here later the final solution and in case I require more details I will go for an specific topic.

Thanks