Simplifying algorithm testing for researchers.

Question

I work in a group that does a large mix of research development and full shipping code.

Half the time I develop processes that run on our real time system ( somewhere between soft real-time & hard real-time, medium real-time? )

The other half I write or optimize processes for our researchers who don't necessarily care about the code at all.

Currently I'm working on a process which I have to fork into two different branches.

There is a research version for one group, and a production version that will need to occasionally be merged with the research code to get the latest and greatest into production.

To test these processes you need to setup a semi complicated testing environment that will send the data we analyze to the process at the correct time (real time system).

I was thinking about how I could make the:

1. Idea
2. Implement
3. Test
4. GOTO #1

Cycle as easy, fast and pain free as possible for my colleagues.

One Idea I had was to embed a scripting language inside these long running processes. So as the process run's they could tweak the actual algorithm & it's parameters.

Off the bat I looked at embedding:

• Lua (useful guide)
• Python (useful guide)

These both seem doable and might actually fully solve the given problem.

Any other bright idea's out there?

Recompiling after a 1-2 line change, redeploying to the test environment and restarting just sucks.

The system is fairly complicated and hopefully I explained it half decently.

Answer 1

Not sure I understand your system, but if the build and deployment is too complicated, maybe you could automate it? If deployment is completely automatic, would that solve the problem?

I don't understand how a scripting language would solve the problem? If you change your algorithm, you still need to restart calculation from the beginning, don't you?

Answer 2

It kind of sounds like what you need is CruiseControl or something similar; every time hyou touch the baseline code, it rebuilds and reruns tests.

Answer 3

Embedding Lua is much easier than embedding Python.

• Lua was designed from the start to be embedded; Python's embeddability was grafted on after the fact.

• Lua is about 20x smaller and simpler than Python.

You don't say much about your build process, but building and testing can be simplified significantly by using a really powerful version of make. I use Andrew Hume's mk, but you would probably be even better off investing the time to master Glenn Fowler's nmake, which can add dependencies on the fly and eliminate the need for a separate configuration step. I don't ordinarily recommend nmake because it is somewhat complicated, but it is very clear that Fowler and his group have built into nmake solutions for lots of scaling and portability problems. For your particular situation, it may be worth the effort to master it.

Answer 4

If you can change enough of the program through a script to be useful, without a full recompile, maybe you should think about breaking the system up into smaller parts. You could have a "server" that handles data loading etc and then the client code that does the actual processing. Each time the system loads new data, it could check and see if the client code has been re-compiled and then use it if that's the case.

I think there would be a couple of advantages here, the largest of which would be that the whole system would be much less complex. Now you're working in one language instead of two. There is less of a chance that people can mess things up when moving from python or lua mode to c++ mode in their heads. By embedding some other language in the system you also run the risk of becoming dependent on it. If you use python or lua to tweek the program, those languages either become a dependency when it becomes time to deploy, or you need to back things out to C++. If you choose to port things to C++ theres another chance for bugs to crop up during the switch.