tags:

views:

111

answers:

5

This is about a C++ problem. I have an object tracking program that takes images from 0,...,n in a loop. At current frame the computations are based on previous frames, therefore I need to hold those variables, matrices, etc for later use. This program has to be integrated now into another system which will provide an image and I have to return the tracking output. The system does later other processes, so my program has to become function to distribute as DLL.

I need to store my variables and matrices from previous images in order to use them again. I don't know if the best practice is to write them in hard drive and read them again in another instance. If this is the case what is the best way and data type/file to write/read. The systems aims to be real-time.

Thanks

+2  A: 

One thing you could look into that IS NOT THREADSAFE is to keep the local variables as static. If you're not familiar with C/C++ static variables, they are stored in the global memory space and "remembered" between function calls. They're like global variables but can only be accessed by the function they're declared in. Run this a couple of times and see what happens.

void foo()
{
  static int x=0;
  x++;
  cout << x << endl;
}

Remember, you cannot have multiple threads call foo because there's only one state now!

Alternatively you could do something where you create a struct that holds a copy of your local state and you pass that in.

struct state
{
  int x
};

void bar(state& s)
{
  s.x++;
  cout << s.x << endl;
}
miked
+1 static variables are a lot of fun!
karlphillip
+1, use the object orientation thing.
DeadMG
What did the question have to do with static variables?
Tony
My function will be part of a bigger system containing behaviour analysis based on image and video, plus a avatars animation. By real-time in this subject we mean to be able to process 25 frames per second or an image in 0.04 seconds maximum.
harvybcn
I have to return a 3xN matrix of doubles. My function has to be DLL. the matrices I am looking to store for the next instantiation of my function are not required for other process, only for me. Should I return them to be kept in ram. I know in Matlab you can pack variables in hardware to free the ram then you IN/OUT them when needed. Can I do something similar in C++ in a fast way?
harvybcn
@harvybcn: You might want to factor in whether you're generated compressed or raw data, bit-depth, whether it's SD or HD, and get a rough MB/sec figure. Keep in mind that not all computers can handle common HD video formats well - you'll be loading the system quite heavily if you push that all the way to disk.
Tony
To free RAM, you either compress the data while keeping it in RAM, or your write it out of RAM to a hard disk, flash media, optical drive etc.. For writing, memory mapped files are the fastest and often the most convenient mechanism - you can basically arrange the data in the same format that your program uses it in memory, but sync it to disk periodically. Not a trivial task all up - no offense but seems you're biting off a lot given your seeming familiarity with the issues and options....
Tony
Thanks Tony, you are right regarding by familiarity with the subject. I am doing a research work based on confident knowledge of maths but not in programming. The point is we got to produce a demo, otherwise I would give this to a developer. The idea of memory mapped files sounds likely what I am looking for. the functions holds 15 MB of variables independently of the input image's resolution. Your answer has been very helpful.
harvybcn
A: 

Not really an answer, just a longer request for details.

Any kind of data persistence issue involves decisions such as:

  • required lifetime: app-controlled, thread, process, until host reboot, indefinite...?
  • how many concurrent readers/writers will there be for the repository
  • how are the readers/writers spaced across networks / hardware (e.g. endian issues, latencies)

You really haven't provided enough detail to make a serious start on this.

If your general hunch is right and a file is a suitable mechanism, you might consider whether memory mapping works well with your requirements... it tends to be faster than streamed file I/O.

Alternatives include a shared memory segment (can live longer than the creating process), heap...?

If your real interest is in serialisation mechanisms, you might have a look at boosts.

Anyway, I'm off home so it'll probably be someone else who answers....

Tony
+1  A: 

It depends on your platform, but these days rare is the platform that doesn't have oodles of memory to spare. So if you are just saving data from a previous pass, no matter how much, my first go at it would be to save it all in memory somewhere.

If you end up running out of space, my second go would be to look into getting more RAM for your system. If it costs an extra $100, and you aren't making thousands of units, then it may save you money in the long run over engineering hours.

If not, then you can worry about the extra complexity of trying to save and restore from disk in realtime.

T.E.D.
A: 

Some ideas and suggestions:

Storing the images or attributes of the images?

In general, an image will take up more space than attributes or data calculated from the image. Perhaps you can store the attributes of the images rather than the whole image.

Cache the data

Put as much data in memory as possible, store the rest in a file. The issue to be answered is how much of information must be in memory (such as the last N items or perhaps the first N items).

Multi (task or thread)

Have one thread that caches the images on demand. As the images are received, it puts either the image into memory (or the attributes). When the fixed memory area fills up, it places the image (or attributes) onto external memory (e.g. files). The main thread would request images from the caching thread. After the caching thread removes an image, it replaces that image (or attributes) with one datum from the file. The thread can sleep until either a new image comes its way or an older image is requested.

Thomas Matthews
A: 

Hey Guys thanks a lot, I got some ideas already from you:

  1. STL
  2. memory-mapped files
  3. Multi-Threads

I will start working with these solutions and let's see what are the restrictions of each one with the requirement of real-time.

I will come back to post here later the final solution and in case I require more details I will go for an specific topic.

Thanks

harvybcn