Creating huge images

Question

My program has the feature to export a hi-res image of the working canvas to the disk. Users will frequently try to export images of about 20,000 x 10,000 pixels @ 32bpp which equals about 800MB. Add that to the serious memory consumption already going on in your average 3D CAD program and you'll pretty much guarantee an out-of-memory crash on 32-bit platforms.

So now I'm exporting tiles of 1000x1000 pixels which the user has to stitch together afterwards in a pixel editor. Is there a way I can solve this problem without the user doing any work?

I figured I could probably write a small exe that gets command-lined into the process and performs the stitching automatically. It would be a separate process and it would thus have 2GB of ram all to itself. Or is there a better way still? I'd like to support jpg, png and bmp so writing the image as a bytestream to the disk is not really possible.

Answer 1

If you want to do it language-agonistic then you only have two options:

• Either do as you have done or
• do it as the professional graphics applications do it. They keep the images on disk in full resolution but when loading/saving they don't load/save the complete image at once but at runtime create such a tile-based approach in memory (and only load/save the parts that are currently relevant).

The disadvantage of the second aproach is that you have to develop EVERYTHING yourself. You cannot use any OS image handling, image en-/decoders or anything else.

I had a similar problem some time ago and could solve it by using only 1/16bpp (especially as 32bit for images usually contains 8 completely unused bits which do not need to be held in memory - alpha channel). This means I mostly had layers of 1bpp images. However this obviously only works if you use your own application to consume the images.

Answer 2

ImageMagick has a component named composite. The docs say nothing about whether this tool has been built to be as memory efficient as possible, but it shouldn't take long to find out. The download page is here. IM is available on all major platforms, comes with a ton of APIs and interfaces, and can as far as I know used as a standalone EXE or as a DLL.

If composite doesn't work out for you, I'm pretty sure there are other tools around. This will probably be easiest to achieve with uncompressed image data - I can't think of a way of stitching 4 JPEGs together without having them all in memory (although I'm sure that has been already done, too).