Are written tests useful skill assessments for developers?

Question

Apologies if this is a dupe, but searching around the closest topic I could find was about interviewing senior devs.

Anyhow, I recently got into a bit of a debate with a friend of mine who is employed in a similar role as myself at a different company. We're both amongst the more senior members of our team and as such, highly involved with the hiring process across the spectrum, from fresh college hires to experienced hires with 10+ years of experience. The debate in question was whether a written skills test was of any use in assessing the skills of a candidate. As in pen, paper, fill in the blank, multiple choice, pseudo/really code problem x, etc.

I argued it was a counter-productive use of time--you're better off just going through much of the same topics verbally so that you can have a conversation with the candidate in order to assess along non-technical "soft" skills as well rather than just staring at them with a piece of paper. He countered the test could be issued before hand, to which I just sort of scoffed in terms of El Google and how much do you really trust these people? I countered I'd really rather just have them code up a simple app in front of me while we discussed requirements and design at the same time.

But then a funny thing started to happen. A cousin of mine, a mid level developer in the same geographic area as myself, told me that probably 50% of the places he's interviewed at recently all used some form of written test. That just boggled my mind. Do written tests really work? Have I potentially overlooked a simple acid test that could have saved me a bunch of time?

Or are all these companies just lazy? :P What would constitute an actually useful written, pre-screening kind of test?

Answer 1

The only thing a written test really tests is the ability to do written tests :)

which is something I'm glad about given I'm re-entering the job market after a long period and am pretty good at tests.

I believe you will get more false negatives than false positives using written tests but I'm 100% convinced they are only valid if you want to hire somebody who is good at doing your written test ;)

Answer 2

I don't like written tests much myself. Mainly because I find that writing with a pen these days is difficult for me as get serious cramp due to my reliance on the keyboard. Written tests can also have the pop-quizzy element to them. For example it might ask how to use a using(){} statement which is no good if you don't know it. The same can be achieved via a try{}finally{} and dispose call which the candidate might know very well but without the knowledge of this syntactic sugar they get an unfair mark down on such a question.

I think a better scenario would be to give the candidate a laptop without an internet connection to solve a variety of problems. Although I have yet to see this happen at any interview.

Answer 3

I believe written tests should be about the communication of ideas, not necessarily how well the applicant knows his theory.

If a company insists on using a written test as part of their hiring process, I would hope that they used it as a part of a greater package:

1) Interview - Social aspects, good fit with the team

2) Written Test - see how the applicant communicates in writing/typing, since a good amount of communication is done via e-mail/IM these days

3) Sample project/code - not some simple coding test ... what has the applicant done, and can s/he show the interviewers?

Answer 4

An IQ test might be more valuable than a test that tries to quiz the applicant on trivia. A person with a high IQ can solve problems and learn, even if there are gaps in knowledge and experience.

Answer 5

@Quarrelsome: That was one of my points in the IMs that we had going back and forth. My fear was that the written test becomes too esoteric pop quiz BS types of questions used more as a "HA you're not such hot shit are ya?" type of questioning rather than a valid screening process.

Or alternatively, the test is well designed, well thought out by the original author. But that author is long gone and the test is now administered by some HR peon who has some ancient answer key that they may shakily be assessing you against.

Neither situation sounds great.

Answer 6

Written tests can be helpful when applied in the correct context. The thing they do is that they place all applicants on the same playing field for that portion of the interview. At the end of the day, you can review all of the applicants and what they wrote down and carefully review each. You can't do that with a face to face interview. There is too much personal bias based on who gave the interview and when. It also allows a not so technical person to give a technical portion of the process. You don't need a strong programmer to spend time on this section of the interview. This being said, a written test can't be used as the sole selector. It must be a small part of a many step process. Where I work, we give four seperate tests, plus two interviews (we're a psychological testing company so we have to eat our own dog food).

I think a better scenario would be to give the candidate a laptop without an internet connection to solve a variety of problems. Although I have yet to see this happen at any interview.

This is one of the tests that we give. Actually it's kinda lame. The main reason being is that it gets out what questions you're giving and then people "magically" know the right answers (yeah recruiters...). You can have the same problem with a written test, but it's much more difficult for an interviewee to remember the details of a fifty question test, rather than remember that he had to build a web program that connected to a database.

Answer 7

I far prefer sitting down with a prospective hire and doing some code review / pair programming exercises. In my opinion it gives you much more insight into the persons thinking process and problem solving abilities then a written generic test.

Answer 8

The flaw with the written test is that you don't get to experience the candidate's thought process while solving a problem, which I feel is one of the most important pieces of information to have when making a hiring decision.

I think you're much better off administering the test verbally. That way, you can also help move things along by giving the candidate hints if they get stuck on something or can't remember some trivial bit of information needed.

Answer 9

A long time ago when I was in the Air Force you had to take a test which was basically an IQ test. It had nothing to do with programming but you had to attain a certain score in order to be a programmer. Later on knowing different developers and the score they got on that test I would have to say it was a good test. The test consisted of all kinds of questions from recognizing patterns in numbers to folding/unfolding shapes.

With that said I really would hate to have to hire developers. It doesn't necessarily matter how great they are. I would want to see their work ethic and how well they produce. I'm not sure how you can measure that in a test.

Answer 10

I work at a company where we give a take-home test. We haven't ever hired someone because of a good score, but we've avoided hiring people because they've failed the test.

We give people some short written specs and ask for some short code to implement it. The problem doesn't depend on library knowledge (or even language knowledge -- we've accepted pseudo-code quite happily) -- it just shows the candidate's ability to turn business rules into a frigging case statement. The number of people who have been gainfully employed for years who can't do this is frightening.

The test is reviewed by a couple senior programmers. For a junior level person, we just want to see if they can get a correct answer. For senior level folks, we want to see something that can be maintained and is reasonably efficient. The questions that we use give us a chance to see what level someone is at.

If the candidate comes in, we discuss the test. That gives us a chance to see if they can take criticism (since no one has ever gotten what we consider a perfect score). It also gives us a nice jumping off place to discuss the work that we're doing: "Well, X is on the test because we once had a guy who couldn't do Y, which is really important here because ..."