During interviews, should candidates write code on paper, on a text editor, or an IDE?

Question

This is something I've been wondering about over the past year, as I've been interviewing:

Why are almost all interview coding tasks taking place using pen and paper rather than using a computer with a text editor or a full IDE? I've only had one opportunity this year to actually type in code.

I can see many disadvantages for pen and paper (or whiteboard)

• Handwriting. This is a sore spot for me (takes me ages, I can't read what I write, makes my hand tired, etc.). It is probably also a sore spot for the interviewer who reads it. Having seen enough programmers' handwriting, I don't think that others would win penmanship awards.

• Time - Handwriting (especially on a whiteboard) takes longer. Out the door go nice identifiers. I feel it constraints the questions that can be asked.

• Forces you to deal with space allocation. You can't just insert a line or move lines about. It ends up looking more like a flow diagram than a linear program. And I always feel like having to worry about technical things like that "affects my mojo".

• Making the interviewee uncomfortable.

The only advantages of pen and paper, to me, is high availability and the fact that it is easier to gesture and show things over them afterwards.

Using a text editor or an IDE seems to me to make more sense. You could also rig it to measure time or even do a screen recording.

On the other hand, I can see several disadvantages to computer-based writing (in an onsite interview!) but I feel they can be easily overcome:

• You need a computer - but I'd like to imagine that most companies have an old laptop that can be used just for interviews.

• Candidates can cheat by going online - But they're being watched, and you can disable the network.

• The IDE can catch certain silly mistakes - But hopefully you phone screened for those things, and you can always use a text editor.

• Programmer may not be familiar with editor or keyboard - But honestly, if you can't figure out notepad on a standard windows box or consider yourself a linux programmer and don't know any built-in text editor?

• Hygiene - But you can wipe the keyboard between candidates. You're already risking the handshakes.

So which one is best? And why do most places go for pen-and-paper?

Answer 1

Everyone uses different online tools; if you only use emacs, or vim, or VS, or whatever, with all your custom whiz-bang key-bindings, what are the chances you won't fall on your face using someone else's setup? (Not to mention differences in keyboards, and differences in candidate typing speeds.)

You're right about whiteboard failings, but 'online' has its own set of failings. In a sense, whiteboard puts everyone on a 'common denominator' footing and keeps the focus away from 'coding' and more about problem solving/talking.

These are my hypotheses, anyway, I am not sure I completely agree with everything I write here, but seems like some plausible answers to your question.

Answer 2

I am usually not insistent on candidates writing code. I like a face to face interview and if I feel the candidate can answer some good analytical problems, knows how to design complex business problems, and if he is able to justify the use of design patterns, development methodologies, unit testing, and other such attributes, I am usually confident that he will be able to figure out the code.

coming to your question, you use pen and paper to see if he can visually draw out some components but I would not go with computer testing.

Answer 3

Using a computer may overemphasize how familiar the candidate is with a particular editor.

On the other hand, using a whiteboard will show how the code looks in the candidates mind. Does the candidate think with uniform indentation? Does the candidate know that they will need some space at the top because they are skipping ahead to the loop but need to handle the entry condition?

Whiteboard exercises are more about measuring a candidates ability to program, rather than testing their knowledge with any given domain.

Answer 4

In general, the questions being asked in a job interview should be short and simple enough that they are answerable on a whiteboard without making the interviewee's hand sore. A large part of the interview process is watching the candidate think through a problem. An IDE is great, but you should have a good enough grasp of whatever language you are expert in to do simple things without such a crutch.

Often the kinds of solutions I am looking for as an interviewer are not far removed from pseudocode. I don't care if the candidate can't remember which file he has to #import, or whether it's array.length vs array.size(); these are minor nits that I don't really care about. What I do care about is: can the candidate understand the question, think of a thoughtful solution, and articulate it by walking me through the code on the board?

As an "extra" part of the interview it wouldn't hurt to have the candidate sit down at a computer and see if he/she really isn't totally lost in the target environment (especially if you require Unix skills, it's helpful to put them in front of a command prompt and see if they can sink or swim.)

Answer 5

My group has deliberately rejected the seemingly-popular artificial interviewing constructs and opted go entirely realistic. Thus we give candidates a mostly-functioning but buggy application, in the IDE we use; bug reports for some of the bugs, and feature requests to implement. They get full internet access, just like we have; and use the same bug tracking software we use. We then meet in one of the conference rooms we actually use, and whiteboard problems in a collaborative team environment, tossing around technical challenges, ideas and problem solving - exactly what we do every day.

There is no substitute for engaging with a person on technical, procedural, analytical, theoretical and personal levels because that is what the job actually entails. After 2-3 hours of this it's clear whether the candidate is the right fit. The interview team (a few individuals from our whole team) are always unanimous. Since we've implemented this process, we not only haven't had any bad eggs make it through, but some of our best people have now joined.

I have real problems with all the various puzzles, tricks and attempts to throw off candidates. I don't understand how that actually accomplishes the business' goals. You don't want people who are really good at solving esoteric, pointless logic problems or people who can write an algorithm from memory small enough to fit on a piece of paper. If your workers can do that, then great, but that's not what makes them useful to the team and it's not what makes them good programmers. I think there is a clear distinction between practical, SE-relevant puzzles and all the rest which are still so common in programming interviews. It's entirely possible to put someone through the ringer (which does serve a purpose) without the ringer itself being so arbitrary.

Answer 6

I think handwriting is an expected common denominator of communication. Its lack is significant for the role.

If the person isn't able to communicate their ideas rapidly in hand-written form on a whiteboard so that someone can understand it, they are at a disadvantage in performing any role where communicating technical ideas face-to-face with other people is important.

Naturally, this doesn't mean getting them to write out the exact code as they would want it to appear in the source file; but being able to communicate their ideas about an algorithm clearly to other people on a whiteboard is a good test to have as part of an interview.

It doesn't matter what someone's reasons are for having poor handwriting skills; maybe their childhood schooling was inadequate, maybe they learned English as a second language, maybe they have a learning disability, whatever. The job requires communication skills, including the ability to communicate in written form, and anyone who doesn't have them to a sufficient degree (as demonstrated practically by using those skills in the interview) is underqualified.

I have no problem setting up the interview process to discover that lack so a decision can be made on that basis.

Answer 7

I think there are valid reasons for either one. However, I personally favor pen and paper (or alternatively, a whiteboard) for a variety of reasons:

• Most programmers are used to programming in a certain text editor/IDE. The interviewer is not going to have every IDE ever made, so the candidate might be forced to get over the learning curve of a new editor.
• Similarly, certain IDEs may favor certain programming languages, which gives an unfair advantage to candidates who write in those languages.
• Coding questions in interviews should focus more on the general idea of what the candidate is trying to accomplish, not if they know the exact syntax for a certain expression, or if they remember the name of a certain function. Whiteboards give candidates a lot more freedom to express their intent, without worrying about nitty-gritty details.
• The time constraint is actually a good one, in my opinion. It forces the candidate to plan ahead a little more before writing the code, and to generally be more aware of the time constraints within the interview.

Answer 8

I'd say paper, nor neither.

If paper, keep it simple. Sort an array, find duplicates, remove white space, write a simple linked list, something that can fit on one page. It doesn't matter if they get it right or wrong (that is what part 2 is for), it matters how they approach the problem, or if they are honest (never done that before) and ask for help, and if they defend their approach. You can coach, and watch them learn and solve the problem.

I really think that coding in an interview is a waste of time. Most interviews are an hour long, and I think that you want to see if you mesh with the candidate, not if they can regurgitating memorized code. Why not have them complete a take home test. Give them a screen shot of a problem, 48 hours, and see what they submit. And tell them what you expect.

Answer 9

A compromise might be to allow a simple text editor like Notepad to be used. This would let the interviewee shuffle lines around etc. without too much effort, while not providing any extra "help" (syntax highlighting, auto-completion, etc).

Answer 10

Be realistic, and let the candidate write code using the exact same tools they will use on the job. Do you really expect them to write code with pen and paper once hired?? C'mon.

At least, that's my opinion. I've been asked in past interviews to use a whiteboard, but it's just not the same. You can't expect to take a person out of their element and expect them to perform; especially in an interview where they may be nervous to begin with.

The only real questions that justify code to be written on a whiteboard or in pen and paper are Language Syntax based questions. Anything more technical (and more appropriate for the job) are just setting the candidate up for failure unless you let them use the same tools they'll use on the job.

Answer 11

We give our candidates a take home "test" of sorts.

Our process:

1. Review the resume (involves 3 dev leaders @ no more than 5-10 minutes per), if we like their resume we have HR schedule a 30 minute call with the candidate
2. We have a phone interview with the candidate (involves 3 dev leaders - at the 15 minute mark if at least 2 of the leaders agree to continue we do, otherwise we end the call)
3. If the candidate sounds like a tech / cultural fit we send them the test application
4. After 3 days we will review the application (involves 3 dev leaders - all 3 leaders must agree to bring in the candidate)
5. The candidate is onsite with the entire dev team (3 members at a time will interview)
6. The candidate leaves and the team stands in a circle and gives a thumbs up or down and discusses

The Application:

It's basically an application that we give them after a phone interview that a professional developer should be able to complete in 6-10 hours; we give it to them to complete over the weekend.

We have found this to be extremely beneficial because:

• If they are serious about the job they will do the app and do it well
• You can tell a lot about someone via code (especially code that they labored over and iterated on)
• The template uses technologies our team uses daily, it tells us how familiar they are with the technology. We also give them ful liberty to use any technology they want to to demonstrate their abilities
• We encourage them to "wow" us - some do, some don't - guess who gets the job...
• If they attempt to "fake" or outsource the application we know immediately because they cannot speak to it
• We don't waste time waiting for them to finish something onsite
• We found the whole stick them in a room in front of a computer is an impractical representation of reality - if they are passionate about software they will get it done and do it well
• We can take one look at their application and usually with in 5 minutes determine if they are a solid candidate to bring in in front of the rest of the team
• It is contextual, no theory - they have real a deliverable, and a real problem to solve, using real technology

Ever since we implemented this process we have hired a stream of rock stars.

Answer 12

The whole point of doing white board programming questions is to put the applicant on the spot and take them out of their comfort zone. I agree that the tactics are a bit archaic but it definitely sets applicants apart.

If someone can stand up in front of people and work through a complex problem--explaining their thought process as they go--then it goes a long way towards proving that they really know their stuff.

Answer 13

I would certainly not notify your current employer about finding another job. I would just take leave/vacation for about 2 weeks and look for work.