Design principles for creating a domain-specific scientific library

Question

What are good principles in designing a software library for the use of scientists in a specific discipline? By this I mean areas such as chemistry, physics, materials science, microscopy, bioassays, astronomy, etc. I exclude generic libraries such as SciPy, R, Matlab, JAMA, etc. which cover a range of disciplines or are specifically mathematical or for data analysis. (Currently a number of us are involved in building open source libraries in chemistry and it would be valuable to know of experience from other domains).

I appreciate that this is an ambitious task and it is not easy to get funded but it is becoming increasingly important for reproducible and verifiable science.

Answer 1

Good to think about good general API design regardless of how domain-specific it is, so check out this google tech talk. (Joshua Block)

Answer 2

"What are good principles in designing a software library for the use of scientists in a specific discipline?" - first and foremost is being a domain expert, IMO. The kinds of packages that are usually successful in those areas are written by chemists, physicists, biologists, etc.

You have to ask yourself if it's easier for a scientist in one of those areas to learn programming or a programmer to learn a scientific discipline.

Judging by the number of books I see for learning language X in 21 days versus those claiming to confer mastery in chemistry, biology, or physics in the same time frame, I'd guess that scientists are more likely to program their own solutions.

Answer 3

1. It should allow its users--scientists--to do science in that domain instead of programming. The less traction have purely programming concepts, as "handlers", "controllers", "pointers", "memory leaks" etc, the better it fits your end-users;

2. It should allow people, who find your lib interesting, to maintain it after you abandon it. Shorter, it should not be developed as "throwaway code" or it should be done in a "good" wat. See your own question on that matter.

3. To design an API an expert, who will use this library, should be involved. He should know, what a usual problem-solving program requires from the library you're writing and define the main concepts you're to implement. Design your API after what the expert says, not in the way you see it. If you're the expert as well, then whatever, invite another guy, who's not involved in development. The reason is that developers tend to design the API in the way they see natural, the way natural for the implementation. Whereas it should be natural for problems it's involved into.

Answer 4

My advice would be: gauge the common programming skills of the scientific community you want to reach (most probably yours!) and develop something they can use (or that requires just a bit of learning, but not too much). Keep it simple, or hide the complexity in the deep internal parts. Scientists are not professional programmers.