What is the best way to tell an excellent programmer in a job interview?

Question

In the setting of an interview: What is the best way to reliably identify when somebody is an excellent programmer. By this I mean he is one of those that is 10-15 times more efficient / rapid / better than his peers towards the lower end of the spectrum.

Many of us have heard of the FizzBuzz Problem as a way to weed out the weak ones. Certainly, taking 5-10 minutes to solve that problem is a serious indicator that an applicant is a weak candidate. I posit that a good indicator is being able to solve that as quickly as you can write. This doesn't seem sufficient, though.

Is it maybe something like giving him a moderately complicated buggy program, and seeing how fast he can grok it and identify all the problems with it?

Answer 1

There are different time scales at which someone can be "fast": some smart people can solve difficult puzzles in seconds, but some smart people produce a lot of good code in a month, even though they might not be that fast at interview questions.

Ask the candidates if they are active in any open source project where you could review some of their code, and spend some time reading those projects' mailing list archives and commit logs. That will tell you a whole lot more than anything the candidates can demonstrate in an interview. (Of course this cannot replace the interview, since not all good coders do open source work.)

Answer 2

An excellent programmer will be able to work with those lower-spectrum peers too. As long as they can do the test and don't wallow in their ego then you have a good candidate, no?

That fizzbuzz test is kinda funny though. The solution I can think of uses the modulo operator. Which I only know from working out character-sheet mapping coordinates (never mentioned in school or college). Would the average programmer even know about that or have I had crap education?

Answer 3

http://www.inter-sections.net/2007/11/13/how-to-recognise-a-good-programmer/

Answer 4

One criteria that I use is to see the 'kind' of languages and tools he has worked on, either in academic or in professional projects and what exactly he achieved. Has he always worked at the application level using standard libraries (always a C# or VB6 guy?) Or has he done a project using C in Linux dealing with hardcore stuff like pointers, memory management, recursion, process synchornization, mutual exclusion, events etc. If he has always used these core and fundamental concepts under some abstraction layer, I will be doubtful.

This is obviously in addition to making him write code. Nothing is a substitute for that. I do however cater for the fact that some people can write code faster than others, and people have different response time when in an interview limelight.

Answer 5

My apologies to anyone who doesn't care for lengthy answers, but I do think it's pretty important to qualify your candidates before you hire them. Anyone who's conducted a significant amount of interviews in this industry knows that most candidates won't last through the first 15-30 minutes of an interview, so most of this list won't be necessary. Just keep in mind how expensive it is (both financially and emotionally) to fire someone before you dismiss my list as overkill. I've tried to list my interview topics here by order of importance.

General Intelligence (brain teasers/logic puzzles)

• Counting toggled lockers
• Crossing bridges
• Weighing marbles
• Burning fuses

Computer Science Knowledge

• Data structures (stacks, queues, lists, trees)
• Algorithms (sorting, search, shortest path, Euclidean algorithm, Hamiltonian cycle, data compression)
• Recursion/Tail Recursion versus Iterative solutions
• Models of computation/Finite state machines
• String matching/Regular expressions
• Backus-Naur Form

Programming exercises

• GCD, Factorial, Fibonacci, Towers of Hanoi
• String and list reversal
• Determine if a singly-linked list has a loop (can you do it with only two pointers?)
• Find the bug

Knowledge of object-oriented programming techniques and common design patterns

Algorithm analysis (run-time O(n) complexity and storage requirements)

Usage of tools and methodologies

• Debugging
• Version control
• Test-driven development
• Coverage analysis
• Continuous integration

Knowledge of common security vulnerabilities and attacks

• Buffer/heap overflow
• SQL injection
• Cross-site scripting attacks

Basic Mathematics

• Numeral systems (convert from one base to another)
• Probability Theory
• Distance between two points on Cartesian plane (Pythagorean theorem)
• Square root (Heron of Alexandria, successive approximation)

Cryptography

• Public key cryptography
• Symmetric-key cryptography
• Hash functions
• Cryptographic protocols (secret sharing, zero-knowledge proofs)

Discrete Mathematics

• Logic
• Set theory
• Graph theory
• Information theory
• Combinatorics
• Proofs (like existence of irrational numbers, infinite primes)

You might also want to take a look at the book Programming Interviews Exposed. It's a good reference on the topic.

Answer 6

Possibly an "excellent" programmer isn't coming to you for an interview. You probably have to steal him from someone else.

Answer 7

The book "Smart and Gets Things Done: Joel Spolsky's Concise Guide to Finding the Best Technical Talent" may help to find an answer.

Table of content:

• Introduction
• Chapter 1: "Hitting the High Notes"
• Chapter 2: "Finding Great Developers"
• Chapter 3: "A Field Guide to Developers"
• Chapter 4: "Sorting Resumes"
• Chapter 5: "The Phone Screen"
• Chapter 6: "The Guerrilla Guide to Interviewing"
• Chapter 7: "Fixing Suboptimal Teams"
• Appendix: "The Joel Test"

The Article by Joel "The Guerrilla Guide to Interviewing (version 3)" can be helpful as well.

And the article "Done, and Gets Things Smart" by Steve Yegge on the topic.

Answer 8

You can't: http://steve-yegge.blogspot.com/2008/06/done-and-gets-things-smart.html

Answer 9

Any answer must include code samples. Hiring a programmer without seeing his or her code is liking hiring a chef without trying his or her cooking.

Answer 10

Ask them a series of questions which require them to code, and have the questions get harder. If they seem to enjoy the challenge, you probably have a live one.

If they can't answer the first easy question, like "write a for loop" or something stupidly easy, then you know this person can't code.

Answer 11

First of all, there's one way you can get an idea before the interview even starts:

If they have a blog or contribute to one or more open source projects, just look at the code and articles they've written. First of all, if they've done either of these then they have initiative to get things done. Also, you can compare these things to the work experience they have listed on their resume and get an idea if they go home and learn more after work, or if they go home and forget about work after 5pm.

Essentially, Do they have passion about programing or not? That's the real question.

Answer 12

Ah, the eternal question.

I did a lot of interviews this year (I have two candidates scheduled tomorrow), and in my experience, hiring is more about gut feeling and people skills, and less about technical knowledge.

1. Take your time with CVs. Some CVs can be rejected in seconds, some take half an hour. Sometimes I think about candidate based on CV a lot longer than I interview him. A few times I've prepared interview questions specifically for that candidate, even though I typically don't have prepared questions.

2. Technical knowledge - there is a minimum I want, and this is usually pretty easy to tell. If in doubt, during the interview, talk about projects he mentioned in CV, and go as deep as you need to. This is usually more than enough to tell you what he knows and what makes him tick. Education is not important, previous jobs matter, possible personal projects score high.

3. Ask about what he wants to do and where he wants to go with his career - do you need what he has, and can you provide what he wants? Also, near the end of interview, I usually ask about a preferred salary. If he's out of my range, or if I won't pay that much for what he knows, that's where we end the interview.

4. Most importantly, candidate must fit into team, and I must feel confident that we'll be able to work together. I don't need to like him, but I must be able to handle him, and he needs to be able to handle me. If that's not the case, I'll pass, because I won't be able to use his technical knowledge. On the other hand, if this is the case, and if he's a quick learner, his lack of technical knowledge won't prevent me from hiring him.

I've trained girls from HR to pass me any CVs as soon as they get them; I schedule interview personally, as fast as I can (ideally day after tomorrow after receiving CV for good CVs). Then he gets half-an-hour or one hour interview with me and at least one co-worker (usually my boss or team-member), where I get to know him and answer any questions. Even if I reject his application on the spot, he gets 20-30 min tour of the company and I talk about what we do and how we do it. Then I send him to HR for psycho test and a bit of really really basic paper coding/SQL. Both tests almost never play a significant role in my decision, it's more of a verification that I judged correctly in the interview. After results, it's 15-min talk where I make him an offer, and if we negotiate the terms we're both happy with, he's hired.

This is a process I had to fight for through company bureaucracy, after missing a couple of great candidates, and which works because I'm the one that decides about hiring (although I do listen the advice from both HR and co-workers, my word is final). More decision-makers, longer process. The longer the process, the more you have to be Google to get top of the crop.

As soon as I'm certain that is a no-match, I end the interview, he gets the company tour and it's over. This might be as short as two minutes on the phone while scheduling interview. Even if you reject a candidate, sell the company. If you did a good job, good hire can come through word-of-mouth from rejected candidate.

Also, one tip. Do send rejection letters (or e-mails) for each application you get. In my current company I usually leave that to HR (apart from those I tell during the interview), but at one point it was priceless getting delighted response from rejected candidate in the lines of "THANK YOU! You're the first company which actually responded instead of leaving me wondering if they'll reply one day!"

Answer 13

This answer is a little outside the box, but I think it's a valuable point.

The very best programmers rarely interview. They don't have to. If your company is particularly world-changing, or excitingly shrouded in secrecy, or several programmers they respect have gone there, then they might apply, but ordinarily great programmers get jobs through their network of associates, not by sending in résumés.

So: the best way to tell an excellent programmer in a job interview is that he's not there.

Answer 14

You should mainly judge the work that they've already done. Any code or ideas that somebody generates during an anxiety-ridden interview are a poor proxy for what they can actually produce on a team.

To do coding challenges, use IM with something like codepad.com and let them do it from the comfort of their own house. Do you write much of your code on a whiteboard in front of your boss, with a 30 minute deadline and your bonus on the line? I don't.

So, is the interview pointless? No, but the emphasis should be on them explaining what they've done and exactly what they've contributed.

You are also going to be subject to all kinds of psychological biases once you meet someone face to face. Don't accidentally hire a programmer because they made better eye contact or are taller than someone else. To route around these, I would conduct as much of the interview as possible over IM/email before you meet them face to face.

Answer 15

One way to tell the passionate programmers from the "I just want a job" programmers is to ask them what book they are reading this week. Then ask them about the books they have read in the past weeks.

I have found that the passionate programmers are ALWAYS reading, and usually the list will include a few programming / Comp. Sci. books in the recent list.

It's not just about keeping "up with the profession" - passionate programmers have a desire and love for programming, and tend to devour material on a variety of topics - not just the whatever language they are using now, but methodologies, other languages (especially new or "weird" or ancient ones), other aspects of IT (maybe robotics, or AI, or games, or...)

If they don't have a recent booklist at all, then they are probably not much of a programmer, in my experience.

Cheers,

-R

Answer 16

Have them code on a whiteboard. Only way you'll be able to tell if they know how to write code.

Answer 17

Tell me what?

Answer 18

http://www.programmerinterview.com/ is a good resource if you're looking for some practice interview questions. It covers quite a few subjects, and has some clear explanations

Answer 19

The language doesn't matter. Logic does. I mean IDE's and compliers are so good these days that any good programmer can pick up any language (ok maybe not assembler) in a week; become decent in a couple of weeks and be very good in a couple of months.

It's his (her's) brain you need to confirm. And you do that my talking. I ask them to solve simple problems. Not by writing code but by stepping me through their logic to come to a solution.

But I admit, if he can't write a simple loop counting 1 to 10, you got troubles.

Answer 20

Having a good programmer present in the interview is best in my opinion.

Only an expert can judge if the applicant just knows a lot of interview questions or if he is actually thinking about the problems and can go into detail. Remember, you don't want to hire people to solve interview puzzles, you want to hire them to get actual work done.

Puzzles are to exclude people who don't get the basics right. If you want to test for skill, prepare a few things you (or your "good programmer") can go into detail about and focus on the one the applicant has to think about for a while. How does he approach problems that he doesn't immediately know the solution of?

Answer 21

I don't think that you should talk about passion on the interview. Frankly, it sounds like a company that looks for 'passion' really means 'working for no money for the idea'.

Passion doesn't even warrant excellence. I mean I spend almost all my life programming, reading about programming, learning crazy languages like Erlang or Clojure and I don't get paid for any of it. Yet I suck at programming.

I think excellent programmer should have a track of successful projects they have been actively involved in. Thus making a programmer write anything above basic FizzBuzz is unnecessary in the interview. Talk about their past projects and what they did. Are you hiring programmers to solve Rubik's cubes and count marbles or work on long and big and exhausting software projects of more than 50 lines of coe?