Saving and accessing a lot of ROI polygons (IDL, ImageJ, Java?, XML?)

Question

I need to save and re-load thousands of ROI polygons on a stack of images, actually a 30-frame-per second movie for 30+ minutes, so at least 54,000 frames. There can be multiple ROIs on a single image (frame) in the stack, and the size and number of vertices for each ROI is different. The ROI polygons are tracking the edges of shapes as they change over time.

The movie stack is usually too big to hold in memory at once. I post-process it, churning through it frame by frame to track my shapes. I can save the ROI objects to binary files one frame at a time, but that generates 54,000+ files to keep together. Alternatively, if I try to allocate a container object to hold all 54,000+ ROI frames and then save that to one file, I quickly run out of resources.

I’d like to save all the ROIs to one file that I can randomly access and modify. If I want the third ROI polygon on frame 100, I want to be able to load it immediately. I also want to be able to insert and delete ROIs into the saved file.

What’s the best file format for doing this? Is there an accepted format or application? ImageJ has a native ROI format, but I don’t know much about it. My application is programmed in IDL right now. IDL has its own ROI class, but like I said, I’d rather not write them all out to separate files, and I don’t have the resources to combine them all in one file. XML? Thanks.

Answer 1

The ImageJ ROI format is documented here. I'd consider using a relational database for storage and random access. H2 Database is my favorite, but there are plenty of alternatives.

Answer 2

I have to do something very similar a while ago (polygons, IDL, fast access). My decision was to take all of my ROIs and put them in one file and then have a second file that contained the offset that each polygon started at.

Basically file #1 would contain a list of all of the vertices of every ROI and file #2 contained a list of integers that specified the byte offset where that polygon started, i.e., the ith integer in file #2 was the location in file #1 where polygon_i was located.

I've used this for fast access of nearly 5 million polygons with about 4 billion vertices in total.