Interview test for applied scientific computing job

Question

Do you know a good and objective question/test to examine applicants for a scientific computing job?

(In fact, this test comes after the candidate passed an interview.)

1st) They need to be intelligent. (Edit, thanks for the words from Trent)

2nd) They would have to deal mostly with programming (C++ and Python, using scientific libraries), numerics and mathematics, but also with engineering and physics topics. Also, C-like or Fortran-like style is not desired... usage of O.O. concepts would be good.

The applicant may have access to compilers, tools, libraries and internet. The duration of the test should be no more than 4 hours.

Answer 1

Find intelligent people and they will be able to adapt to your specific requirements. This is much better than trying to find someone who meets all you "check boxes".

Answer 2

"Convince me that you're the right person for this job"

With this question you can test all the needed skills for a good programmer:

• Intelligence
• Creativity
• Communication
• Self-motivation

... the only thing untested is teamwork ability, but that's a hard thing to test anyway.

P.S. as for knowledge, you can always ask the person to implement a cyclic ring buffer in "the most reusable way he knows" without access to the internet, both in python and C++. Real "usable" code speaks more about people and their coding culture than tons of checkbox questions.

Answer 3

but you should still test to see if they grok computing. There are any number of postings on programming tests for interviews. Don't worry about your specific domain, just see if they can code at all.

It is always eye-opening; never boring. And its better to give a smaller test and discuss it with them, rather than a large test. Ask them why they made various choices, etc. You then get to see if they can clearly explain themselves

Answer 4

I think that you are better off trying to discuss a problem with the person to see how they reason when confronted with that problem. What algorithms do they think of, what methods of approaching the problem are they familiar with? What possible pitfalls do they see?

By allowing the person to discuss with you, you can adapt to their particular skills and there is also less risk of rejecting candidates that get nervous about tests. You can also get a good feel for whether the person fits into your team - you don't want someone that is really good at solving interview-type problems but cannot be bothered with discussing other ways of solving them or whatnot.

Answer 5

I think that you should ask him about his experience. About questions: do not ask about language-feature questions (things like a = b ?? c ?? d), programmer should have idea about algorithms, solving problems and good communication skills in team(this is very important, really)

Answer 6

I can think of a few points you may want to cover:

• knowledge of floating point calculations and how floating point is represented internally (e.g. see What Every Computer Scientist Should Know About Floating-Point Arithmetic)
• some familiarity with approximation algorithms (e.g. Newtonian approximation of roots, Reimann sums)
• common graphing and visualization software
• handling large amounts of data efficiently
• confidence intervals and uncertainty
• statistics and probability

Answer 7

I don't think specific tests are needed for something like that. Just get some smart people that are already working in that area and have a chat with the applicant. Maybe also include something like suggested by Kornel Kisielewicz. Ask him or her about previous projects, technologies used, what he/she liked, what not, etc. If the interviewers are smart, they will be able to tell whether he's right for the job or not.

I was interviewed once for a general software developer position. After a little entry chatting they said something like: "On your resumé you say you know C++ and Java quite well. Tell us all the differences you know between those two, not limited to the languages themselves." From there it was flowing nicely and you can easily throw in questions when they fit. Like when I said C++ is usually translated to machine code while Java to byte code, they would ask what byte code means, what's the advantages and disadvanteges of either approach, etc.

Answer 8

Well, an easy problem is to ask someone to solve a simple ODE system using whatever libraries they wish to use. None of the libraries that I know of are straightforward enough that they can be learnt during the test. For example, solve this system for x=1:10: dx/dt = -k (x^2/x).

A tougher one is to ask someone to solve a stiff ODE system. Here, the choice of algorithm becomes important and a 'guess' probably won't work. For example, most Michaelis-Menten equations are stiff.

dS/dt = - vmax * S/(Ks + S)

where dS is the rate of substrate depletion and vmax and Ks are constants (that you can give the candidate). Choosing the wrong solver here leads to rather disastrous results, as I have found out first hand.

Neither problem is any good on its own, since pretty much anyone could solve either one by trial and error given 4 hours to solve it. But as part of a larger test, they might be useful.

EDIT: What does it prove? Well, ODEs are everywhere in scientific computing. If you haven't at least dealt with them cursorily at some point, it's a big hole in your knowledge. I've only tried to answer the question as asked. "What are specific questions to ask an applicant to a scientific computing job?". To solve the problem the applicant is going to have to know

1) Basic calculus.

2) Some sort of a scientific computing library (MATLAB, SciPy, GSL).

3) If the question is asked appropriately (for example, if they were asked to compute a rate of acceleration), domain knowledge (physics in this case) is also tested.

I think this checks all the boxes :)

Answer 9

I'll probably regret this but here's my very general advice FWIW

1) Do interview the candidate not the job spec.

Have a minimum set of ‘Essential's’ in your spec that you won’t compromise on, after that look at the candidate and see what other skills they bring to the position – they might even bring skills you didn’t know you needed

2) Do test candidates

A test could be a couple of verbal questions, a written test, etc. It doesn’t really matter but you definitely need to get a feel of what the candidate knows and what they don’t know.

3) Do make candidates write code

You’ll learn more about someone by how they approached a problem, how they implemented it and more importantly how they explained what they did to you than anything else. Remember, this is what you’re hiring them to do, this is what they will spend the majority of their day doing

4) Don’t ask lots of ‘memory questions’

Asking someone to remember the exact format of std::algorithm or all the methods in System.Xml.XmlDocument serves no purpose. People use docs and Intellisense – get over it.

5) Do ask open-ended questions

Ask a few questions that don’t have a ‘right’ or ‘wrong’ answer – this way you are giving the candidate a chance to offer their opinion and you’re having a conversation rather than a glorified ‘tick list’. Two way conversation helps because it allows the candidate to relax and as important, it allows you to relax. Having a discussion can tell you a lot about a candidate i.e do they listen to your opinion, how do they respond to counter-arguments

6) Do see if they’ve done their research

Always ask a candidate if they’ve heard of your product/team/game/company. If they can’t tell you just the tinniest bit about the company they are applying to then chances are you don’t want them at your company. It takes 30 seconds to type into Google after all so be brutal about this one. Trust me.

7) Don’t interview on your own

You may be super clever, you may know what you want, you may think you can read people but software development is primarily a collaborative effort so get in a couple of people and don’t just get in another manager or a senior coder – bring in your team and listen to their opinions. Importantly, catch up afterwards and discuss the candidate whilst it’s still fresh in your mind.

8) Don’t rush the interview

The time to allot for interviews depends a lot on the type of position you’re interviewing for i.e junior, experienced, senior however always give yourself an extra half hour on top of how long you think you need. This gives you room to breathe and if you’re done early then fine. Go to second interviews if needed. I frequently do two interviews – the first is technical and the second is more focused on personality and team.

You might not feel it’s appropriate or you don’t have time, however don’t discount it. If you feel that you didn’t get enough out of the candidate first time and there’s still potential then get them back in. If they want to work at your company/team then they’ll come back.

9) Do ask for evidence

If a candidate says ‘contributed significantly to the development of …..’ then ask them what they did, in detail, what they learned from the experience and what they would do again. If they can’t give you evidence and detail then chances are they didn’t ‘contribute significantly’. Beware candidates who use ‘we did’ all the time. Sure, development is done in teams but they must be able to say what they contributed as well as describing the team effort.

10) Do be prepared to be honest and forthright.

Most candidates slightly over-sell themselves. This is fine, their CV is what gets them the interview. However, if you think a candidate is not being entirely up front about their experience then say so. Make sure you explore strengths and weaknesses. Too many interviews end up being an elongated sales exchange. Yep it’s great that the candidate knows the ins and outs of UDP but does he/she know when they’ve made a bad decision and how do they deal with that? Ask them the last time they were wrong or the last bad decision they made. Be careful not to be too heavy handed or judgemental though -Making mistakes is perfectly natural and most of us do it all the time, it’s how we deal with it that’s important.

11) Don’t get taken in by confidence.

There’s a fine line between knowing your stuff and thinking you’re better than everyone else. In a knowledge based profession like programming this is always a temptation for us all. I agree with Jeff Atwood on this one. A good programmer knows that coding is hard and that they make mistakes and a lot of the time their code sucks. Bad programmers don’t – they think everyone else’s code sucks. Caveat Emptor.

Answer 10

Test their math. A few simple questions regarding whatever domain-specific math knowledge they ought to have are very objective and can quickly screen out that 90% of applicants who have no business even being in the field.

We often start our applicants with some basic linear algebra questions. If someone doesn't know what a cross product is, there isn't much point in going on with the interview.

One (good) boss I used to have liked to start phone screen interviews with the question, "what's two to the eighth?" It was astonishing how many people got this wrong. (Flubbing it wouldn't end the phone screen, but it was a very strong predictor of the candidate having some serious problems generally.)

Answer 11

Here's one scientific problem that is rather interesting and difficult, but it can be solved in four hours if your candidates are really good. If not, you can ask them to describe the solution instead of implementing it, or judge incomplete solutions. High school level physics are needed in addition to good programming skills.

In a central heating system there is a water circulation network consisting of pipe pairs connected as a tree structure. At leaves there are radiators, each of which need to emit the same heating power P (constant). At the root the water entering the network (hot pipe of the pipe pair) is heated to temperature T1 (constant). The inner nodes of the tree are pipe pairs, each assumed to be of certain length (constant len).

The heat dissipated (lost) in the hot pipe is U * len * (T1 - T0), where T0 is the room temperature (constant) and U is a heat transfer constant (W/(m*K)). Because of this power loss, the water cools down before reaching radiators (even slightly lower temperature means much less power emitted by the radiator).

Each radiator has a valve that adjusts the flow (kg/s). The flow and the temperature together determine the power radiated by the radiator. You may use the following function:

radiatorPower(flow, temperature) -> power

Implement a data structure describing the network as a tree and an algorithm to determine the required flow for each radiator so that radiatorPower(flow, temperature) equals P.

Notice that the temperature of the returning cold water is controlled by the power emitted and cannot be directly controlled or measured. Also notice that at every branch of the tree the water carried by the pipe pair is split among all the children of the node (radiators or other pipe pairs). The pipe heat loss equation uses the initial T1 at all pipes because it is assumed that the heat loss by pipes higher in the tree is not important for the heat loss calculations (T1 >> T0 and the losses are small).

Water specific heat: c = 4190 J/(kg*K)

Answer 12

Hi

None of the previous answers seemed to mention domain-specific knowledge -- what part of science (which has got quite big in the 21st century) are you working in ? I work with a team of geophysicists working on computational EM problems. There's an awful lot of good scientific computationalists we wouldn't look twice at because their domain is so far from ours.

I'd suggest that it is easier to train someone with the right domain knowledge and the wrong programming knowledge than vice versa.

Regards

Mark