Reverse Engineering old paint programs.

Question

I've got a couple of really old MSDos based paint programs. They work on palette indexed image buffers. They have a number of spectacular shape drawing tools, brushes and effects that simply do not exist in any modern paint program- Particularly not whilst staying within the "bounds" of a low color palette indexed image. I would like to reproduce many of these wonderful tools in a modern program, to perhaps make them more accessible to myself and the general public again, without having to boot up an emulator like dosbox. But I have a problem.

While a lot of these brushes and tools and things have obvious functions, whose implementation should be fairly straightforward. But with other tools, their principles of operation are not so obvious. I would be stuck determining a correct or faithful algorithm to implement those.

If you were me, what approach would you take? Are there decompilers/disassemblers readily available that can handle old programs like this? Or would you take some different approach, such as methodically testing the algorithms on different inputs to sort of infer the underlying function? Some combination of different techniques? In the case of one line of these programs, the original author of these (commercial) programs is known, and is now a rabid open source advocate. Should I just try to contact them directly and ask? I'm not particularly experienced with reverse engineering, so I'm at a loss at where to start on this.

edit: it seems that "Ask and ye shall recieve" really works.

Answer 1

IDA Pro handles nearly any binary, and is the gold standard for this kind of work, though contacting the author may prove helpful as well.

Answer 2

If you can convince the author to open source some or all of the program, that's the best method.

Beyond that, disassembly and debugging are going to be your friends. In some cases you may be able to decompile a piece of software if you know what it was compiled with originally. But disassemble it, and run it in a debugger to find the overarching structure, and look for the obvious functions that do the actual brush work.

It may seem daunting at first, but after some time you'll find it's surprisingly understandable.