Coding Test - allow use of web?

Question

During hiring a .NET web developer I give the candidate a coding test.

I tend to limit the candidate to MSDN installed on the test server - I think it holds everything the candidate needs to complete the task.

I admit, this is not the normal case as I don't expect the candidate to do his work without use of the web.
On the other hand I don't want the candidate to google for a complete example and copy-paste it, i want to evaluate his skills.

The question is do I need to allow free use of the web during the test?

If you think the whole coding test is wrong - I would like to here alternatives you may have for me.

Answer 1

Well, one thing you want to be aware of is that the developer you hire might not know everything that he will be thrown during the time he is working for you. If you ask him a question that he doesn't know off the top of his head you would want and expect him to research it and come back to you with proof that he understood the concepts that he just learned.

I say let them use the web - but ask them to explain in their own words how their code works. Most of my knowledge comes from online resources. However, I make sure that every line of code I write I understand.

There is a baseline knowledge that developers in a particular field should know; but you also want to figure out how quickly he can learn new things. A good test IMO is to throw a question you know he doesn't know and see how long he can figure it out using the resources he would have if he were an employee of your company.

Answer 2

There are tests, where web access can be given, and there are where it doesn't really make sense.

Case where its fine to allow web access

• When its unlikely to find even 60 percent of the code over the net
• When you will ask to explain the code after he/she completed the code
• A very specific solution using SQL query, which is unlikely to be found on the web

Case where its fine to not allow web access

• Some basic programs like, recurssion, fibonacci, factorial, string manipulation, small trick programs, etc. There is no need of computer even in some of these cases

Answer 3

I'm very sceptical about coding tests during interviews. I think that a lot of the test I have seen, represent very specific (artificial, non real-world) problems where you would use the internet to solve them.

I think it's not really important to know how to solve such problems by heart - often time it is much more important that you know how and where to search for answers.

If you want to test the persons during the interview, I think it is better to ask them some conceptual questions instead of a specific programming problem. E.g: questions about object orientation, polymorphism, design of n-tier application, etc. etc.

Or as an example from the ASP.NET world, ask the interviewed person question such as: what is ViewState, what is a postback, what is session-/application-state, etc.

Answer 4

If you want to get an idea of how a candidate will perform in a job, I think it's best to try and make the conditions of the test as close as possible to the actual working conditions.

It should be pretty easy to prevent copy-and-pasters from slipping through the cracks by asking the candidate to explain his/her code.

Answer 5

Is your goal to see what basic knowledge the candidate has and if he can code without copying solutions from the web, then don't allow internet access. If you want to see what strategies he employs to get to a solution, let him use the web if he wants to.

I personally find it more interesting if a candidate can solve problems on a larger scale than just solving a simple programming problem. So I tend to ask him about the methods he uses when programming (Unit testing? Ever worked with it? What do you think of it?). This gives me a better picture than coding in an interview situation.

Sometimes it helps if you ask the candidates beforehand to bring a one-page coding sample to take a look at their coding style. This also saves you time during the interview.

Answer 6

As you say, 'I don't expect the candidate to do his work without use of the web' why not allow it too during the test? And what if he does copy and paste? I do that too. Surely the key is to know where to look, be discerning with what you find and apply it intelligently. Do you want to hire someone with a terrific memory or someone who can develop software for you?

When I was at school, calculators were just becoming affordable. As their use was seen as unavoidable, the exams were changed. Simple number-crunching was no longer tested in the way it was before (it was important then). Rather problem-solving techniques were to be tested.

Answer 7

It's important to make sure a candidate is resourceful - you don't want your programmer sitting there when they get stuck, not moving forward; you want them to use whatever resources are at hand - be it MSDN, picking someone else's brains, using the web, etc - to get the job done. Cut-n-paste from the web does seem like cheating, but (a) if you design your task carefully then it will be unique enough for there not to be a standard answer they can copy from the web, and (b) isn't re-using existing code a key part of building software? It's not much different from using 3rd-party libraries, to avoid reinventing the wheel. On the downside, of course, you also want them to show they can develop algorithms, so the unique task needs to include some element that requires that without the solution already being on the web. Trouble is, forums are the achilles heel to all of that since they can simply ask for the solution and someone, somewhere, is going to hand over the answer unwittingly!

Answer 8

I usually allow candidates to use whatever resources they want. After they're done, I sit down with them and go through their code together, ask questions like why they chose that particular approach etc.

If a couple of minutes of Googling was enough to not just copypaste some code but to learn enough about it to be able to defend the decisions within, then he's intelligent enough!

Answer 9

Allow the candidate to use the web but tell him beforehand that if he used the web, you will have to evaluate HOW he solved the problem.

If he used the web for something simple such as finding the syntax or parameters which he forgot, don't mark him down. This is normal.

If he used the web for something like look at how a specific function is used, don't mark him down. This is normal.

If he searched for a specific code and then copy-paste it, then ask him about how the code works. If he can explain how the code works, then there's no reason to mark him down. If he can't explain it without looking at the site where he got the code, you have to mark him down.

If he used stackoverflow.com, check his profile for questions, answers and badges. From there, you can check how good a programmer he is.

Answer 10

As do many other respondents, I'd rather employ a resourceful developer who know how to use the web to the fullest to draw on other's experiences and previous work, than a developer who limits himself and his applications to the MSDN way of doing things.

I copy other peoples code all the time - daily in fact. The knack of it depends on finding the right solution quickly and integrating it into your existing work.

So let your candidate use the web and ask him how he came to his solutions. You might learn more about him from his methods than from how will he can remember previous solutions.

Answer 11

Three things I'd do.

Let applicants send in a coding example along with their cv.

Let applicants produce some real-life code (maybe even pair-program with a developer on your team) this will show you if they can actually use the tools. Internet is a tool too so they should be able to use internet.

Let applicants solve a problem in pseudo code on a blackboard during the interview. In this case you can be their "internet" by helping them.

These three approaches will show you different things. The first is a good early warning mechanism but can easily be faked (they could just download oss code from the web somewhere). The second is good to see if they can actually code but they might score badly if they're unfamiliar with the tools you use. The third will show you if they can solve theoretical problems but won't show you if they actually are good team players or if they write maintainable code.

Answer 12

I recently had a friend start talking to me on IM, he was in a coding test job interview. He had a couple SQL questions. At first i thought, hell you've got to do this yourself. I'm not going to help you cheat during an interview.

Then i thought about it again. I've been answering questions and talking to him about various technical issues for years on IM as part of his work. So when he encounters problems in the real world with the job if he gets hired, he'll do the same thing.

We don't talk about it much, but having a good network of friends to ask questions, and knowing how to search out relevant answers on the net are a big part of being an effective programmer or sysadmin. I've met people who were super smart programmers, but didn't really know how to find information online. They missed a lot, were kind of out of the loop. Knowing how to use resources should be important.

When i do interviews i often ask people what websites they read, what development tools they use, and why. It's a similar thing. Sure it's not about how they write x line of code, but it's about how they work.

No how to get around somebody just copy and pasting "answers". Well first, don't ask questions which have pat answers. Secondly when i'm interviewing i like to give people some code, ask them to refactor it, have them talk through what they are thinking. Then ask them to write some new code which implements a feature. Pair program with them. It's hard to hide inability to code when pair programming. While they are pairing, it totally makes sense to say, "let's go look up the api on the date time library."

Answer 13

It all depends what you want out of your successful candidate. I contest the view that knowing how to google makes you a good programmer because the simple fact is that the internet is full of bad examples as well as good ones. You don't really want your codebase to reflect how lucky your googler was on the day he cut and pasted all his code off the web. You want it to demonstrate sound practices, proven methodologies & elegant, efficient solutions that your team understand and are enthusiastic about. Not a jumble of styles that don't resemble each other. There's a wealth of good to be gotten from knowing how to get help from the interweb but real knowledge and ancient wisdom is being lost every day that people who don't really understand what they are doing are given jobs because they appear to solve problems with their ability to "google it".

If you really want to give your candidates access to the web then by all means do, but make the questions hard and scrutinise the results to see if they've picked the first solution they found or if they've picked the best solution to the problem.