What's the most effective workflow between people who develop algorithms and developers?

Question

We are developing software with pattern recognition in video. We have 7 mathematicians who are creating algorithms. Plus we have 2 developers that maintain / develop the application with these algorithms. The problem is that mathematicians are using different development tools to create algorithm like Matlab, C, C++. Also because they are not developers the don't give much concerns for memory management or multi-threading. This one of the reason why the app. has a lot of bugs.

If in your company you have similar situation, how do you deal with it? What's the best tools you can recommend to create algorithms? What communication supposed to be between mathematicians and developers? What's in your opinion the most effective to work with high-level tools?

Answer 1

I'd recommend the devs spend a few hours getting used to Matlab, especially the Matlab debugger. If their background is CS then they'll already be familiar with vectors and matrices theoretically if not practically. Other than the matrix being the default data structure, Matlab is C-like and easy enough to interpret for translation into another language.

Answer 2

I have been working with a physics professor lately, and have a little experience with this(although admittedly I'm no expert).

I have had to translate a lot of Matlab code into another language. It has been difficult because a lot of(most) of the operations are absent, including when it comes to precision, and working with matrices and vectors. A good math library needs to be found, or created to fit your needs.

The best way that I have found is to do the following:

1. Get the algorithm to work correctly in the new language.
2. Create some tests to verify that the algorithm is producing desired output. Have your mathematicians verify that your converted solution in fact works, and that you have covered all bases with your tests.
3. Then after it is working, and you can trust your tests, optimize the algorithm to be good coding style, have good design and performance characteristics. Use your regression tests to make sure you aren't breaking anything.

I normally start with a verbatim copy of their algorithms into the other language, and then work from there, regardless of if I do a lot of tests.

It is important to get a working copy first, in case the performance is really not an issue and you need to move on to other things and can come back later to make it faster.

Answer 3

This is your job. How you deal with this is what identifies you as a system developer.

Communicate with your colleagues. Draw and explain, have meetings, agree upon and set standards requirements, follow your plans and talk to your project manager. Make sure that your relevant colleagues are joining up on meetings. Have 1-1 talks etc etc

You cannot blame it on the mathematicians for developers creating bugs. It's their job to worry about implementation, not the mathematicians.

Answer 4

I am not sure whether you devs are rewriting the mathematician's stuff or if you just have to interface to it so I am not sure if my answer is of any use.

However: I work together with a bunch of phd candidates and postdocs on a machine learning library and am a student myself. In that process, I came to translate a lot of algorithms from python/numpy to C++/blas.

This process can be quite tedious - especially with numerical and stochastic algorithms, it is hard to find bugs.

So here is what I did: Get some sample inputs and calculate the results with the python code. Generate unit tests out of these for C++ and then start coding them in C++.

Checking concrete sample inputs with the outputs is essential in this setting.

Answer 5

I agree with Makach.

Let the guys who are creating the algorithms use the tools that they are most familiar with. Because there are two separate (and equally important) tasks to work on in this project. First, there is the creating of an efficient, elegant and appropriate mathematically sound algorithm, then there is the twistedly difficult task of translating it into CPU-speak. The mathimaticians should focus on their first task, and to make it easier for them, allow them to use the toosl they are comfortable with. In terms of man hours, it is a much more efficient use of their time to write MATLAB code, than it would be to have them learn a new programming language.

Your task is to unearth the (presumably) brilliant mathematics that are buried within the gibberish code.

That part is just a perspective on the problem at hand. Here's the actual answer.

Communication, mutual respect, and teaching/learning.

Communication & Mutual Respect

You must communicate with them often. Work closely with them and ask them questions whenever you come across something you're not sure of. This is much easier when there is mutual respect, which means that if you spend all your time criticizing their coding abilities, then they will be forced to spend all their time criticizing your math abilities. Instead, try quick learning-sessions. ("Lunch & Learn" is a fairly common tactic)

Teaching/Learning

The first and most important piece of wisdom to impart to them is commenting. Have them comment the crap out of their code. Tell them that the comments are much more important than the code quality, and that as long as their comments are right, they can leave the rest up to you guys. Because they can. They don't need to have their code look beautiful, for be the fastest, they just need it to make sense to you guys.

To continue this mutual learning scenario, if you notice some very simple very common mistakes they are making, (nothing NEARLY as complicated as multithreading) just give them a quick heads up. "That way works (or doesn't) but here's a way to do it that is a little different but it will make your lives much much easier." Encourage them to reciprocate by trying to notice which nuances or parts of their algorithms which you and your team are having difficulty with and teach a little tutorial about it.

Once you guys get the communication flowing, you'll find it easier and easier to shape their coding style to what is best for your team, and they will also find it easier to understand why you don't see it the same way they do.

Also, as mentioned by Kekoav, make sure they provide a few fully loaded test cases. That means for something like

A -> B -> C -> D -> Solution

They would provide you all the values for A, then what it looks like at B, then what it looks like at C and so on. So that you can be certain that not only is it correct at the end, but it's also correct at every step of the way. Try to have them provide examples that are regular, and also a few of them that are unusual, so that you can be certain your code covers edge cases.