How to Recruit Great Developers?

Question

A question for all you interviewers/recruiters out there. How do you evaluate candidates at job interviews? How do you differentiate the good from the bad? (please, no "FizzBuzz" answers...)
Which kind of questions do you feel that have the best results? Personally, I find that asking about all kind of .Net internals subjects usually provide some good results. Usually I also like to push the limits on the candidate's knowledge, to really see what he's capable of.

So which kind of questions (or some other techniques) work best for you?

• Basic Programming Problems (Recursion, Simple Sorting)
• Knowledge in various .Net capabilities (Which parts of the framework the candidate had experienced with).
• Advanced Topics/.Net Internals (I always tend to favor candidates that find it fun to experiment with the SSCLI etc.)
• Software Design
• Experience with Development Tools (Source Control, Team System)
• Performance (Write a function that does X, in the most efficient, fastest way)
• General Development Questions (What's the candidate's opinion regarding code readability vs performance, code comments vs self documenting code etc.)

I'd also like to hear about more ideas, not necessarily relating to question, but just how to find the really good developers out there.

Thanks.

Answer 1

Read The Guerrilla Guide to Interviewing

• Don't focus on their immediate knowledge. Don't ask detailed factual information like what is the name of some API function. It is learned, used, forgotten, googled back and relearned again

• Ask questions to find out about their aptitude and willingness to learn, openness to new ideas and concepts, their readiness to change their old habits into doing things better

• See if they have a passion for what they do. Programming outside of work is a great sign they have an interest in programming rather than think it is a good career path. I have many fellows who work in software development, but I can probably think of only one I would want to work with. For the others it is just a career.

For advanced .NET questions look here:

Questions every good .NET developer should be able to answer?

Good C# Interview Questions for a Senior Dev Position

Altough I'm not sure I'm doing good job giving you the material to scare off your candidates. Keep in mind that advanced framework knowledge does not mean an advanced developer. Some people my know all the little details but still be incapable of doing quality work. On the other hand, a great developer may simply have never come across those issues in his career.

• Regarding algorithmic questions and performance. Don't go into the little tiniest details. Unless the person is straight out of the uni or have long or just recently worked with these matters in practice, he won't likely remember this stuff.

• Your "general development questions" direction is very good. Try to find out if a candidate has a strong opinion about code quality, development process, testing, organization. Should he be reciting the basic ideas and expressing the standard opinions, it might be a sign these matters do not really concern him. If you come into an argue about something, it will be a good sign.

ADDED: A few more things that I forgotten yesterday.

• For universe's sake, avoid braincrunching questions. You know, those like about people on two sides of the river with a single boat, a couple of old women selling apples, two burnable cords to measure time etc. etc. These are mostly (though not all) questions that you can only answer if you already know the answer. Otherwise it may take ages (if ever) to solve the puzzle, and it will certainly be longer than any job interview. Somehow, most recruiters believe that ability to answer those helps weed out bad candidates. The others are saying the reaction of a candidate when faced with a puzzle will show how he handles a difficult situation. Unfortunately it won't work because...

• ... a job interview situation is by definition an artificial environment. Candidates come and expect a decision concerning their life to be taken. They expect anything to take place, talks, tests, games, questions, coding tasks. They also expect to be humiliated by arrogant people putting them into an unresolvable situation by using tricky question, psychological games and all dirty tricks. They can do anything except for behaving naturally. So if you expect through a test situation to measure their genuine response when in production work or on the customer side. forget it. They will try to play the game. Some won't manage it and lose the morale. The others will put a mask and play a role they think you expect of them. Both are bad and both do not provide useful information. The others will just lose interest as soon as they recognize the game they were forced to play so many times before. Remember, you are not interrogating a suspect. You are looking for a great programmer to become your colleague.

The best way would probably be to imagine you are already colleagues, try to easy the candidate and then just have a constructive talk about professional matters. Then you can easily decide whether you would want him to become your real colleague or not.

Answer 2

There are hundreds of books on this topic, but from my experience on both sides of the interview, I would do the following process:

1. Phone interview: ask questions that can be answered without having to write things down. Have the conversation on speakerphone with your team listening in.
2. If they pass the phone interview, bring them in. Have them do a programming test that requires knowledge good enough to do their job everyday. (You would be surprised how well candidates can talk about technical things, but are clueless when applying their knowledge to real-world problems.) We do a test that combines Web programming (C#) and SQL. This test should not take less than 45 minutes, and should be cut off at 2 hours if the candidate cannot pass it. Please note that we encourage and allow questions from the candidate.
3. If they pass the programming test, have them meet w/each team member. I am always looking at this point if I feel like I could work with the candidate in terms of personality.
4. Have the candidate meet with at least 1 non-technical employee, such as your business analyst. We had candidates who made it through the above items, only to act like an jerk to this person. Programmers need to be able to relate to non-technical folks.
5. After the interview(s), meet as a team to discuss the candidate. You'll get some interesting feedback that you may not have considered.
6. Finally, if you want to hire the candidate, establish a 3-6 month time frame where if things don't work out you can let them go w/out bureaucratic issues. (We hire candidates through recruiting firms, as consultants first. After about half a year, if they are successful, they get rolled as an employee.

This process is as much art as it is science. YMMV. Good luck.

Answer 3

The best way is to have an interviewer that is a great programmer, as he will know what he is looking for.

For example, just because you don't happen to remember every package and method in Java doesn't mean you don't know it, it may be that you use too many frameworks and languages to care to memorize what can be googled.

If a person truly understands OOP for example, they can quickly pick up any framework that is well-designed.

If a person knows several different programming methodologies then you can see if they can explain when to use FP and OOP in the same application, for example.

Can they explain when to use one language or framework over another?

Again, though, these are questions that should be asked by someone that is a great programmer, as they can better judge the answers.

Answer 4

First, the qustion title implies more than just the interviewing process. The interview is usually only one part of the entire process. Remember, just as you are interviewing the candiate, they are (or at least should be) interviewing you as well. It has to be about "fit with the group" as well as skill set. I've worked with some incredibly brilliant developers before that had the personality of a stump (or worse, the personality of an jackal having a bad day) and it ultimately always hurts the group as a whole.

That being said, things I usually look for in an interview are how the candidate solves a problem. This can be done in many ways - whiteboard problems, discussion, etc., but it boils down to trying to figure out how they reason about a problem and make decisions. This doesn't mean you should ask the "impoosible" questions (those in which there is no right (or real) answer), although sometimes those can be helpful.

Asking about any past projects that were failures and what was learned from them and/or asking about one area of improvement in themselves is also good. If the answer you get is "no projects failed", "we didn't learn anything", or "there isn't anything I need to improve" (or any similar answers), you've learned that you probably don't want to hire that person.

Always have more than one person interview (if possible), although this doesn't have to be all at the same time. The idea is that the more people involved, the more opinions you get and the more opportunity for someone to spot something that others missed. If any one is unsure of whether or not that person is a good fit for the company/position, then it should be a no.

Answer 5

To find really great developers, you have to go through a lot of bad ones. So really, what you're looking for is a system that allows you to figure out if someone is bad as quickly as possible. To me, this means actual coding problems. You will learn tons about a person's capabilities if you make them write you some code right in front of you. Some people prefer a whiteboard, some people prefer giving them an actual development environment, but however you do it, have them write you some code.

As for what to have them write, some people prefer general coding challenges like "how do you do a breadth-first search on a binary tree?" I do not. I generally like to remove as much domain knowledge as possible from the problem and have them concentrate on putting the pieces together. So I don't ask questions about crazy data structures or problems with trigonometry in them. I just ask questions that have solutions with conditionals and loops, since this is a significant portion of the code most people write anyway. Given these problems, I can find out if someone is a bad developer right away because it is painfully obvious.

Now that I've weeded out the bad developers, how to find the great ones? Well, that's actually easier than you might think. Simply the speed with which someone goes through a coding challenge they're not prepared for is one indication, but also how they handle problems like do they find the bugs in their own code. Also, every programming question has opportunities for optimization. Do they optimize their own code? Or do you have to prompt them? I've found the truly great developers generally follow this kind of process:

1. Build a quick and dirty solution
2. Test it using representative test cases
3. Fix any bugs they find
4. Optimize it
5. Test it again using the same test cases
6. Fix any bugs introduced

And finally, if there's time left, I ask them questions about why they did things the way that they did. Present them different options and ask how they might have done things differently. This will show you how much depth they have.

Good luck in your search!

Answer 6

Though I agree with the other answers, I find that a lot of "coding question" interviews tend to focus on

1. general aptitudes, and
2. ability to code up new solutions to small problems.

Though those skills are important, they're not everything. I don't spend most of my time writing new code to solve small problems; most of my time I spend analyzing, extending, debugging or maintaining existing code. I like to look at how well a candidate works in those areas.

The "coding" question I ask doesn't start off as a coding question at all. I put eight lines of C code on the board, describe its purpose and usage scenario, and then ask the candidate to criticize my code. These eight lines of (very straightforward!) code have well over a dozen bugs in them, from security holes to bad naming conventions to non-portable idioms.

This tests the candidate's ability to read and understand code they didn't write, as well as their knowledge of bad code smells and insecure coding patterns.

Once we've got a good list of bugs, then I ask "the client application calling this code has shipped to customers and you cannot update it easily, but this code that you're criticizing runs on a server which you own. Can you fix all the bugs you just identified without breaking a single client?"

Now I get to see

1. how the candidate prioritizes bug fixing -- do they go for the easy ones or the important ones?
2. what tools they have in their toolbox,
3. how they deal with an ambiguous problem.

That last one is very important to me. We deal with ambiguous situations every day and have to resolve them as best we can to serve the real needs of the customer. I therefore deliberately keep the scenario sufficiently vague; unless they ask, the candidate does not know whether there are two clients making a couple server requests a day, or thousands of clients making thousands of requests an hour and expecting submicrosecond response time. The original code is buggy as hell but very fast; a solution which wrecks performance for every client is not "without breaking a single client". Does the candidate take that into consideration, or do they just make unrealistic simplifying assumptions to fix the bugs?

Some more thoughts on interviewing here:

http://blogs.msdn.com/ericlippert/archive/tags/Interviewing/default.aspx

Answer 7

To be honest, you just can't! Great developers are good enough to work all by themselves and will be having their own businesses with well-paying customers. They are expensive, charge by the fraction of an hour and will not guarantee that their performance is as great as the developers themselves!!!

Also be aware that great developers have some very nasty character flaws, that have allowed them to become great! One flaw is their Ego. Not only are Great Developers very good at what they're doing, most of them are also very aware about how good they are.

A second flaw is stubbornness. This is a good thing for developers because this way, they will continue to follow their own design and not some half-baked idea developed by someone else. This flaw will weed out the good developers from the bad ones, since good developers will prove that they're right by making good products. Bad developers will miss deadline after deadline and generate code that ends up at the Daily WTF. In general, for this flaw to work well, you just cannot afford any failures!

And of course, good developers need to be lazy. They will write code in a way that maintenance is real easy and often not even required! The code will be easy to understand, readable and will do exactly what it needs to be without anything fancy around it. The code will follow the original specifications to the letters, even if the original specifications contain some minor flaws! (They will warn about those flaws, though.)

The best test for great developers is by giving them a very simple programming test. Then, ask them to adjust the code to do something slightly different! The faster they finish, the better. And if the second test just took a minor change in the code of the first test, even better. If the code does exactly as specified and nothing more, perfect! (And if you suggest a certain solution and they still decide to do it different then you've found your Great Developer!)

Oh, well... I realize that this makes great developers sound really bad. A bit the Prima Donna's from the IT world. To be honest, there is a lot of truth in this too. Of course, there will be a few exceptions. There will probably be a few members here who will protest against this view, but again... That's their Ego speaking, not wanting to be considered lazy and stubborn. But I mean this in the most positive ways. Laziness isn't bad when you do finish your work well in time. Stubbornness is good when you're right. And great developers will be able to finish well in advance of the deadline, following mostly their own designs. (And then they need to celebrate to tell their Ego how good they've been again!)

I know, because I am a Great Developer... :-)

Answer 8

I've done my share of developer interviewing and what I've found common among the best of them is the will to come to work for a company that will provide them the way to grow as a developer and engineer and which in its work embraces the best practices and techniques of modern software development.

Answer 9

Read the comments here: Ask HN: Where have all the coders gone?

Answer 10

Aside from all the stuff people have mentioned above (an despite a few things mentioned above), one other thing is also very important: do NOT make your company sound like it takes itself ultra-seriously, or has unreasonable expectations.

It's a joyful thing to be a great coder who can exceed expectations on the first day, swap languages and other technologies at the drop of a hat, and enjoy a little praise for hard work well done. It's quite another thing though, to read a job advert saying that to work in your little company, I must be able to exceed expectations, know 5 specific languages (requiring any specific language is almost stupid in itself), and be an expert in BSD Make version 3.1.

If you want reasonable people with strong core qualities, post reasonable ads, which focus on core qualities. Otherwise, you'll just show that you don't know how to appreciate a skilled individual, and that working there would be frustrating at best.

Answer 11

When I started working in the late 80's, interviewing for a programming gig was almost always a full-day affair. I don't really have a problem with this. I think you have to go through a few stages.

First of all, make sure their resume describes the person you want. Then your only job is to make sure they match their resume, that's easy--right? :)

Also, if they are going to be at all customer facing, examine their resume for it's communication ability. If it went past 2 pages, were there places they could have removed information? Is it written to be read and not just convey information? If the grammar isn't virtually perfect, don't expect them to be writing any customer specs (this is one of the most important documents they are ever going to write! If they can't bother to have it peer-reviewed and keep it in good shape, then they don't understand writing documents to be read.)

Phone interview a LOT of people. Probe them technically, see if they actually know what they say on their resume. If they don't seem to be lying and you like their resume, bring them in.

Don't spend a ton of time showing them around the building. Most developers don't need to be "Sold" at this stage, don't try to sell the job to them until you are sure you want them. Instead, split this into two parts. Have some good technical people probe deeply so you are sure they can actually code. This is critical, but don't dwell on it. Don't have every interview be technical, just a couple hours of it. (Lots of people we've interviewed are pretty much completely unable to code past the "Hello, World" stage. It's true).

The rest have them get to know their (potential) team. Take them to lunch--check their development practices.

This should be about 2/3 of the process and you need to find out:

Are they flexible? Are they willing to learn new practices? (Critical!)

Are they willing to admit mistakes? If not, there is no use in continuing.

Give them an impossible problem (like one of these behavioral questions) and see how they handle it. They should not get too frustrated or upset, they should spend some time on it, they should come back to with refinements that attempts to make the question solvable, and they should give up if they can't find a solution. The really nebulous/impossible problems lead to the best discussions. Have a teammate ask these too so they can get to know the person's reactions.

Personally, I always want to get an idea for how they feel about duplicated code. This doesn't go for everyone, but for me--any duplicated code is unacceptable.

Ask about a project they were really proud of. Find out why they were proud. Are they proud they could code? (Meh) Was it because a customer appreciated something they did (perhaps a GUI invention?) Was it a technical solution that other programmers failed at? It'll tell you a lot about what they try to do with their code.

Personally I've always wanted to give an interviewee some code in their favorite language and editor and say "What would you do to clean this code up". Check refactoring skills, see if they would write tests, see if they solve a problem do they comment it. When they see something ambiguous do they ask what it's actually trying to do or guess? I've never done this because it takes too long to do during an interview--but I may try some day...

When you find a good person it's pretty rare. Although you can't always do it this way, it is nice to always be hiring, but very slowly. If you interview a person a week all year long, you might find one or two a year that are obviously BRILLIANT. It'd be nice to hire them even if you aren't really looking for someone.

Ramping up quickly isn't good for you, your project or your team--Your standards get lower and the number of people you can choose from is restricted.

Answer 12

"Readers’ Forum: How do we identify the good guys?" isn't developer specific but I think it is applicable here as something to consider when trying to find good guys.