Hey guys, I am trying to implement a image recognition program in C++. I had completed the extracted of feature, now trying to save the large amount of numbers. I had 2 implementation in mind, one is to save the data into binary files, to reduce the overhead of computation, or i can use database to keep the information. Binary file is fast to read and write, but hard to access, while database seem an easier interface to work with, but i not sure if the performance is fast enough. Do you guys have any experience regarding fsream or database as the choice for fast persistent storage?
A:
You could use SQLite. It basically gives to database-style access to local files. You can also create in-memory databases with it, but i do not know if it is possible to persist those.
Also, the best choice here heavily depends on your access patterns. If it is just sequenctial write and sequential read, then binary files are the best solution.
Space_C0wb0y
2010-03-11 14:09:02
A:
If you really want speed, consider using the C-style I/O functions such as fopen(0, fread() etc. These can often be notable faster than iostreams, for a number of reasons.
anon
2010-03-11 14:12:26
Thanks for the fast response. Going to try fread for the implementation and see what the time to read and write.
Yijinsei
2010-03-11 14:21:42
@Yijinsei While you are at it, try the non-standard open() and write() functions too.
anon
2010-03-11 14:23:27
A:
For me the advantage of using a database like SQLite is an independant format for my data and the ability to query the data using SQL.
Writing data to a SQLite database proved to be a lot slower than writing it to a simple CSV text file.
It depends what you want to do with the data afterwards? If you simply want to read it in again and do some stuff, use fstream (or C-style I/O) like suggested. If you want to query the data and only get specific data, use SQLite.
You also have to consider how you wish to store the data in a database. If you are going to store the data as a BLOB, you may end up losing the benefits of SQL and sacraficing performance.
dirk
2010-03-11 14:20:52
after i done extraction on the images and saved. I need to use back the data for comparison and there could be like thousand access per query. So i really need a fast approach
Yijinsei
2010-03-11 15:50:26
A:
Hi, I am writing an audio recognition program in C++, and I am using now PostgreSql for data storage. So I am interested in an answer too :). What I can propose is that, the data storage should be chosen according the recognition algorithm. If you compare your image with saved images 1 by 1, then binary files seem to be quicker. But if during recognition you work with several images at once, a database may be more speedy solution.
Draco Ater
2010-03-12 11:09:13