What's the best way to organize data on my tiny embedded system?

Question

I haven't done much programming for Embedded Systems before, and now I have to create some scripts for something relatively tiny (<60MB RAM, almost all of which is already used by other more critical processes, the processor is less ~ 500MHz). I can't run something that is on all the time, so Firebird is out. I suggested sqllite, but people more familiar with the platform have told me that even that is likely to consume too much memory and processing power.

I will be programming the prototype in Perl, and will likely need to re-write the scripts I create in c for better performance later on down the road. I'd like to be able to reuse the data structures I create when I re-do it later on. Most of my experience thus far has been in MySQL, so it would be great if the syntax was simliar to that. Any suggestions?

Sorry I can't be more descriptive about the platform I'm working on.

Answer 1

Berkeley DB.

Answer 2

as a suggestion, you may want to consider lua for scripting, its pretty fast, and can be used in production systems and very easy to bind to c.

do you need a relational database?

quite often on embedded systems you use a simple storage system, like a file based system. in general the more flexibility you want, the more overhead you will need.

• simplest, treat entire memory as a large sequential file.
• slightly more complicated, an allocation table to keep track of multiple files

and so on...

Perhaps a key / value store would give you the best compromise for query and storage.

Answer 3

From what you describe I'd look at Berkley DB or a similar key value store.

You can also serialize your data to C structures from Perl. The traditional way to handle this is to use pack, but its a pain for more complex structures. I have found that Convert::Binary::C is great for working with data destined for C structures.

You can feed CBC a struct declaration and configure it to handle your compiler's endianness, int size and so forth. You can also provide code to execute when packing or unpacking a value. For example if you have a typedef for a fixed point number, you can configure CBC to unpack it into a float in Perl, and then convert back to fixed point when the number is packed.

I have had great success using this tool to work with memory dumps and to prepare images for deployment to embedded systems.