Using a "take-home" coding component in interview process

Question

In recent interviews I have been asking candidates to code through some questions on the whiteboard. I don't feel I'm getting a clear enough picture of the candidates technical ability with this approach. Granted, the questions might not be good enough, maybe the interview needs to be longer, etc, but I'm wondering if a different approach would be better.

What I'd like to try is to create a simple, working project in Visual Studio and have it checked into source control. The candidate can check that code out from home/wherever and then check back in work representing their response to the assignment that I'll provide. I'm thinking that if the window of time is short enough and the assignment clear enough then the solution will be safe enough from all-out Googling (i.e. they couldn't search for and find the entire solution online). I would then be able to review the candidates work.

Has enough worked with something like this before, either to vet a candidate or as a candidate yourself? Any thoughts in general?

P.S. my first StackOverflow question - hi guys and gals.

EDIT: I've seen comments about asking someone to work for free - I wouldn't mind paying the person for their time.

Answer 1

I would just like to say thanks, in advance, for creating yet another flood of people looking to SO to do their work for them.

Answer 2

As a very experienced developer, I appreciate the take-home opportunity very much versus having to solve trick or gimmick problems on a white board in front of a crowd.

That said, I recently heard from a friend who works at a large semi-conductor company that they are explicitly prohibited from including take-home assignments as part of an interview process. Their legal department advised them of potential liability issues with this practice.

Finally, welcome to StackOverflow!

Answer 3

I'm inclined to say this isn't a great idea. There are two reason for this:

1. You can't know that the person you are interviewing did the work. Once they leave your office, their genius roomate could be doing the work for them. Even if they don't fully outsource it, they can go online, ask people questions, etc. You might not be seeing the true abilities of this person. This means at best it is a gate. If they screw it up, you know they suck, but if they don't, you know nothing.
2. It will discourage the best people from applying. Most other employers don't require this. It will be seen as a hassle and the best people will just avoid applying with you because of it.

Answer 4

Only problem I'd see with this is that the candidates could easily cheat and get someone else to do the work. Who knows, your interview question might even end up on StackOverflow :)

Answer 5

I generally prefer to give the candidate small specific programming questions in a face-to-face interview -- that's the best way to accurately gauge their skill, as they cannot fall back to using google and SO to hide their actual abilities. You can learn a lot from even very simple questions -- e.g. if they are incapable of writing fizzbuzz or a fibonacci implementation on a whiteboard or laptop in their choice of language, then any take-home assignment is bound to be full of cheats.

Second-best is a programming test given at the office with a laptop and internet connection - using the browser and shell history, you can deduce the candidate's thought process, although it is less direct than watching them work.

Between these two options, I don't see how a take-home assignment would add any value. It's simply too difficult to verify if they actually did the work themselves.

Answer 6

You should be able to get an idea about their technical ability by asking questions, IMO, and some coding may be useful, depending on the job, but, largely, this seems as though you may need to have a consultant come in to help with the interview process, if you feel you don't know enough to be able to determine what the person does or doesn't know.

I am not crazy about writing short programs during an interview, but if I must then that is fine, but it should be relatively short.

I remember for one company, before I could interview I had to write three programs, one was for a ternary tree, which was interesting, and if they liked what I wrote then they called me (they did, it wasn't going to be a good fit). This is preferable as it then gives me an opportunity to decide quickly that it isn't worth it. If I go through an interview, then you give me homework, I wasted a couple of hours, as I will be annoyed at homework.

Answer 7

All things considered, I don't think Googling -- or other research techniques -- should be prohibited. Research should be encouraged, and the candidates if hired will probably employ go anywhere they can for answers. Granted there some things a programmer should know cold, but if they don't know the basics they can only go so far with Google.

Once for an interview I had to create an application with installer before I showed in person. I thought it was slop as I started late and rushed it, but the interviewer was quite pleased. I think a coding project type interview requirement will weed out many who are intimidated to even try.

Answer 8

There are a few minor gotchas to this in my mind:

1. Solitary work - If in your workplace it is common for developers to work alone and not help each other than this can work alright. Generally, I like to ask co-workers for help and bounce ideas off them, so the idea of sending a take-home project can backfire that way.

2. Initial setup - Are you giving buggy code that has to be fixed? Code where enhancements are needed? There is the question of what initial work has been done and whether or not going down that rabbit hole will be OK or not.

3. Opening up - What kind of access would they get? I'm thinking of potential security issues here.

I don't think I'd see anything wrong with giving some code and asking for minor changes that shouldn't require consultation or clarification but beware this pitfall that may popup. While I have had this and it hasn't gone well, I'd rather it be done in 2 parts, one where I get the task and code and another where there is a follow-up. Just as a suggestion.

Answer 9

Take home is not a good idea. They might not have the right environment set up at home. But you can set up a PC in your office and let them use it for a while. They probably won't cheat right in your office.

Answer 10

Look at the open source stuff they've worked on and you should be able to get a pretty good idea of their style/capabilities before you even talk to them at all.

If they don't work on any open source projects in their spare time that also tells you something about them.

Answer 11

We use a take home test as part of our interview process. There is the danger of someone googling the answer/solution, or having someone else do the work, but this is easiliy mitigated by:

1. Have the interviewee explain their methodology
2. More importantly, quiz them on portions of their test

I wouldn't penalize someone for being resourceful, as long as they understand the implications of their solution. There are lots of sources of bad information out there.

One caveat - the test should be involved enough to be useful, but not too involved to discourage real professionals from applying. It shouldn't take a competent programmer more than one day. We've even paid people to complete our tests.

Answer 12

I always sent a quiz in response to the resumes.

It had several components each with 5 questions covering the language we used at the time (VBScript/VBA/Visual Basic), SQL, DHTML/ASP. And we went over the quiz in the interview. There were not a lot of gimmicks or tricks, but they were often the kind of thing where I wanted them to run an example and tell me what the output was and why the output was what it was - they basically covered common gotchas - checking object properties when the object wasn't checked to have been found, operations which did and did not short-circuit, values versus references, boxing/unboxing.

Once the quiz was returned, we would schedule interviews - I cannot remember a single case where we did not schedule an interview for someone who returned the quiz. I estimated it would take 30-60 minutes to do the quiz. This was some time saved in the interview, and we had code to discuss. A lot of times people are not allowed to bring code samples of work, and a white board isn't exactly a typical programming environment.

It was only one component of the interview process. If someone else did the work, it wasn't like they were going to be able to hide it. I'm also not expecting them to be prohibited from using SO or Google when performing their work if they're hired either.

And yes, I've seen my quiz questions show up on USENET in the past.

Answer 13

Almost everyone's answers so far revolve around the fact that Google is bad (shiver) or that SO is bad. This is ridiculous. I don't know what drives these purist thoughts, but lets be honest with ourselves here: Research is part of the programmer's toolbox. Who cares if he reuses code? Code reuse is part of software engineering. You'd be better served by putting someone on probation for 90 days to observe them in a real setting rather than giving them interview tests at all. If they have a decent amount of experience, possibly a degree, what point is it to test them? Some people just don't do well on tests but would excell as a great programmer for you. My two cents.

Answer 14

As a complementary answer I would like to say that above all, submerging a candidate with a programming quiz is risky. If your questionaire isn't right, it could be too pointed or too general. It could take up to several hours for questions which seem basic. Remember, under stress, most programmers will make ridiculous errors. Let's try to atomise the problem

This is just an example. Say you want to quiz them on optimisation and redundancy. Someone with experience will tend to cut corners in some cases.

Write a function that will return if a given condition is true or false.

Let's look at two possible right answers:

1.

bool evaluate(int value)
{
    if (/*condition*/)
    {
        return true;
    }
    else
    {
        return false;
    }
}

2.

bool evaluate(int value)
{
    return /*condition*/;
}

Both answers are technically good, but answer 1 is a textbook example of someone who isn't used to writing a lot of code in the language in question.

If you think this is dumbing down your candidates, try it. You'll be surprised.

Answer 15

The problem I forsee with this is for the people who don't have a development environment at home. Personally, the only reason I have Visual Studio at home is because I get it for free from the University (that's also the only reason I don't use Linux as my exclusive OS either.. ya, I'm really cheap ;) ).

Also, things like setting up source control access can be a bit beyond some programmers. It doesn't mean that they are inferior as programmers I think, but that they just don't know and don't want to learn how to do something they may very well never need to do. My workplace experience isn't on the same level as others, but at my only job so far my development environment was already set up for me when I started my first day.

I personally have no problem with using Google or SO though. It should be pretty easy to spot hacked up code, or with a search find if they asked it to be done for them. And research is not a bad thing.

Answer 16

I've never encountered this myself, but I have heard of people requesting interviewees bring a sample of their work to the interview (or to submit it with the resume'). It would be sort of like asking an artist (or model) to see their portfolio.

I'd think asking that up front would be more effective than assigning your interviewees "homework". It also keeps the candidate from feeling like they are possibly being used to do your coding for free.

Answer 17

As an interviewer and as a candidate, I tend to like the "take home question" when they are done right. However, you must ask questions that don't have just a "right" answer. So, for example, don't ask a simple algorithm problem that can be looked up; ask something that requires the developer's style and personality to emerge. One sample problem I've been given asked me to write code that shuffled a deck of cards; there are three or four good and many bad ways of doing that. They were looking for appropriate and readable class design, a reasonable if not necessarily ideal algorithm, and maybe 4 pieces of flair; my "pieces of flair" were passing unit tests, which apparently nobody else provided or thought of. I now include at least a few unit tests on any "take home" programming test, and it made a difference for each of the companies that used that approach.

The most annoying "take home" thing was for a job at a company I was quite interested in, but their request was too open ended. "Show us some sample code" made me realize that, even with pet projects in various states, most of my useful/interesting code was under NDA, so I started thinking about something that would be interesting to them, and started researching a problem in that I soon learned had tons of postdoctoral research around because it wasn't as trivial as my first imagining of the problem. By the time that team was back from their team offsite week or whatever, I already had two or three other job offers, a few spiked out ideas, and a long reading list.

So, as an interviewer

1. Ask something where Googling the "right answer" won't provide a complete solution.
2. Ask something for which multiple reasonable solutions exist. Make the question more than just an algorithm to implement.
3. Ask something which lends itself to interpretation, but isn't excessively ambiguous.
4. Ask something that involves actual design, even if it's a small design.

As a candidate:

1. Make sure your code passes more than the sample input/output case. I saw a bland algorithmic question reused many times at one company that many people failed because they only copy/pasted wrong Googled solutions that failed very simple variations of the problem.

2. Write unit tests and include them when you send in your answer.

Answer 18

IMO it is fine to give such a question to be worked at home. The next step that is more important is having a face-to-face interview where you go through the code submitted, asking for all the ratinales and at the point asking him/her to refactor through an extension to the problem.

It is followed by a couple of companies I know of.. And the results are pretty fine too.

Answer 19

as a pre-screening process, i ask a few simple tech questions via the phone before having them come in (that way, you weed out people who know nothing)

another approach is to offer a timed online test (i know, there are some issues with this - whats to stop the person from having a friend help them do the test?). again, this helps weed out people who dont know how to code at all

i have some more information about this approach in this article: Hiring Programming Staff

--LM