Pair Programming for a job interview

Question

Our company has been thinking about scrapping our interview procedures and bringing each candidate in for a 4-5 hours sit-down with some of the programmers and just do some pair programming.

I like the idea in theory but I am not sure how you can really make it fair for each candidate. How would you rate them? Wouldn't their input really depend on what each programmer was working on that day?

Any thoughts on whether this is a good idea/bad idea or how to make it work is what I am kind of looking for here.

Cheers!

EDIT:

RESULT - AS requested

We are going to conduct the first steps of the interview the same as before. Phone followed by face to face. Instead of bringing them back for a third and final grilling, we are going to bring 3 developers back to sit with all 7 members of the team. We have decided to let the team decide who is then hired.

We have come to this conclusion for a couple of reasons. We believe this will empower the developers by giving them a choice who they are working. The second reason is group dynamic. We think it is really important to have a good group dynamic and it is hard to tell until after you hire a person if they will fit in or not.

So the end result is we are going to go ahead with the pair programming sessions but in a completely different way and for a completely different way than was originally intended.

Any thoughts or criticism of this approach is more than welcome!! (this edit is posted as an answer below so feel free to downvote if you feel this is not the best approach)

Answer 1

I hope you have a bunch of steps ahead of this one. For this to work you need an excellent resume and phone screen. You don't want to spend oodles of time on candidates that you shouldn't be talking to in the first place.

So you suggest an initial interview and possibly have the second interview as the pair programming session? – Ted Smith (1 min ago)

Yeah. You might even think of having a simple coding interview happen over the web using something like CoPilot.

Answer 2

The easiest way is to give each person the same programmer to work with and the exact same piece of code.

The problem you're going to run into, is that hiring isn't like programming. There isn't a step by step process to lead to the right answer as to who to hire. (you can have multiple steps to make the decision easier). You have to evaluate each one on their strengths etc. and essentially make an educated guess as to which is the best one to hire. Sometimes you guess wrong.

The other thing about pair programming you're going to have to watch out for is the amount of time necessary to have each candidate at that stage go through that kind of a test. If I were looking for a job, I would be hesitant to go an interview at a company that would ask me to do that. Why? Because that is a lot of time, and if I am interviewing at multiple places, I could spend literally days just going to interviews for jobs I may not even get or want. Someplaces like Google or MS would be an exception, but most places are like those two. (Not to mention the fact that if they are working on real code, you are essentially asking them to do someone's job for free).

Answer 3

Unless you use pair programming extensively in your real-world development, I'd be very hesitant to use this. I've met any number of high-quality professional developers who have mentioned a strong aversion to pair programming and whose skill would not be well-judged in such a process.

Answer 4

Why not? Also, it's not like interviews are always (or ever) fair. You should evaluate the end results of the new approach against the traditional interview-based approach.

Also, a mini interview before the pair programming session might be good to keep from wasting the programmers' time with people who would be a bad fit.

Answer 5

I like this idea. However I think it might be difficult to do since it would require the candidate to have some knowledge of the project you would pair on with him. Also, 4 to 5 hours seems a bit long. What if you immediately see that it is not going to work out, are you going to sit through the whole session with the candidate?

Good question though. Stuff to think about.

Answer 6

To keep it fair, you'd have to make every participating staff member have a prepared problem to evaluate the candidate on. Preferably something taken form the real world in their company experience, but something that has already been resolved. This is a good chance to evaluate the knowledge on a problem and evaluate not just programming skills.

I hate it when too specific questions are answered. I had an interview once where a programmer was testing my knowledge of the STL which I used extensively and was trying to get me to answer that a custom allocator was needed. I had heard of them but never used them (esp in windows) and was made to feel dumb. IOW, avoid being judgmental.

So my point is, ask practical questions that aren't so much about testing programming knowledge as you can evaluate more qualitative personality and problem-solving approaches if you use the "pair programming" idea.

Good question!

Answer 7

Honestly, that sounds like a great idea, though Jason Punyon is certainly right that you should do a lot of weeding before you waste significant amounts of your developers' time on culls. You get a glimpse at an important metric out of it that's otherwise nearly unobtainable in interviewing: what someone's like to work with.

I don't think there's really any need to be concerned about it being "fair" based on the subject matter or trying to present consistent situations to different candidates, if you maintain the right evaluatory attitude -- that it isn't about whether they "got the right answer" or jumped through the right set of hoops, but what sort of effort, problem-solving, communication aptitude and flexibility they showed. You'd lose most of the benefit of the exercise by turning it into an artificial test, not to mention changing it from something that your developers can get some benefit from (or at least still get some work done during) to a massive waste of their time.

Answer 8

Joel Spolsky has an excellent Guerrilla Guide to Interviewing which talks about, amongst other things, programming tasks.

Trivia: Joel Spolsky is a co-founder of stackoverflow.com

Answer 9

As a personal anecdote, I got smacked around in an interview because of a technique like this. I had gone far in their interview process; passed the resume checks, the code submission and this was the face to face portion of the interview.

I was fresh out of university and had never pair programmed before nor done TDD. They sat me down to do a deck of card exercise and it flopped. Badly! I didn't understand why the interviewer was writing tests that seemed so dumb* (IE "return null;") and they didn't explain why and of course being foreign to TDD I didn't know what questions to ask. The end result was that it looked like I couldn't program my way out of a paper bag.

If you're going to do this type of exercise you need to cater to the interviewee because they're going to be in different spots with their aptitude. This means that you'll get different assessments that may not be based on actual talent and are thus going to be heavily biased.

*Now that I understand TDD, I do understand tests like this and how it's supposed to work, but man did that ever seem stupid at the time!

Answer 10

One particular company uses a technique called extreme interviewing. For the extreme interview they will bring in say 30 developers and group them into 15 pairs. They will explain that they are looking for people who work well with others. That they will make a hiring decision based solely on their ability to work with others.

They will provide a problem for the pairs to solve. They will emphasis that they are not interested in the solution just each programmers ability to work with others. For each pair they will provide an observer of the pair. During the exercise (about 2 to 4 hours in duration), the observer will takes notes about a person ability to pair ... not the solution.

They are amazed how many programmers focus on solving the problem instead of collaborating. Of the 15 pairs, they will identify about 4 to 6 developers for a second interview. Those developers will be asked to come back and spend a week with the team (they get paid). After a week, they decide who to keep. Generally about half of them (2 to 3 developers).

When they are done, they have developers that are able to collaborate and after a week working with various pairs, the team has a strong indication who can effectively develop software. The process is both innovative and effective. They have had a high success rate with those they have hired.

Answer 11

I just had an interview with a San Francisco based company that prides itself on Agile methods/etc. I was to interview the CEO himself. I have about 20 years of experience in the industry, but have never pair programmed or developed using TDD approach. I was told it would be a "programming interview" but had not idea what to expect, and before we started the guy said that he thought that I may agree that all interviews should be done this way. (which in retrospect was nothing more than an arrogant statement).

Anyway, at the interview the exercise was to develop a class using TDD. It took me a second to adjust my thinking on the entire process, again since I had never pair programmed or done TDD. While I stumbled here and there I did ok in the end. but his reply was the I did not exhibit the aggressive back-and-forth nature that they require for their pair programming environment. Now, that could also have been an underhanded way of saying that "I didn't think you did great" kind of message.

Luckily I didn't need the job and to be honest the experience made me realize that I'd rather find a different career than having to be a software engineer that HAS to work in pairs, day in day out, when it came to developing code. Odd thing is that on occasion I have worked with another person on code simultaneously, so anything is possible.

In end I guess it was a good outcome since they didn't think I was a good fit and I didn't care for their working methods. But we would have came to the same conclusion had I talked a for a few minutes more about myself and had he given me a little more info on how they go about their work. Which is to say that there are other ways of finding a good fit candidate than putting them through the stress of pair programming with a complete stranger; bogus way to gauge competency imo.