What do you think about using the Project Euler website as a candidate profiling tool?

Question

I've read a thread around here about people using FizzBuzz to filter out completely clueless candidates for programming positions.

To be honest, I find it hard to believe many candidates won't pass that (although i CAN believe it), but one way or another, even for a very simple first filter, it seems like that bar it set too low.

I recently stumbled into Project Euler's site, and I find it very interesting. I know NOTHING about math, and I've found myself solving many of those problems anyway. Non-optimally, but all way under a minute like the site requires...

Which makes me wonder... Could this be an interesting way to select candidates? Not as the ONLY thing, obviously... But as the "show me you can code" part of the interview, instead of asking them to reverse a string or a linked list, this sounds interesting. You can give them half an hour or an hour on a real computer, and ask them to solve as many as possible (i'm not expecting more than 2-4 truly), in a real computer, with the dev environment they'll be using later. (Obviously, you look at the code they wrote too / ask them to explain what they were thinking, you don't just look at how many problems they solved)

To me, that sounds more like a test of what they'll be doing later in the real job, than programming with pen and paper... (Even thought they'll be doing CRUDs and DB logic rather than solving math, at least they're writing code and debugging it a bit)

I don't have a lot of hiring experience, as you can probably see, but i'll have to start doing that soon, am I'm not sure what's the best approach to it.

I'm really asking for opinions here. What are the downsides of this idea?
Does it suck really bad?
Is it better to make them code with a pen and without a compiler for any reason?

Any opinions are very much appreciated.

---

EDIT: Thank you all for your answers, they are very insightful. I get the objection that I'm asking them to do CS-stuff instead of Engineering stuff, so I'm still testing for a different ability than I want. You are all completely right.

Some thoughts:
- You're right, it's much better to pick specific Euler projects that let them "solve a bunch"
- I'm not particularly worried about them finding the solution on the Internet. If they can't explain their code and thought process, it won't work for them, if they can, then perfect. Being able to steal and adapt other's code is a good skill for me, saves us time.

My main problem is that I've never been able to come up with an exercise that is representative of what they'll be doing, AND that can be completed quickly. Most of the examples i've found around (Reverse a linked list) fall into this category too.

Case in point: CRUDs. Given my existing code base, I can make a very simple and relatively decent CRUD in an hour (without any bells and whistles). Now, if you throw me in front of a blank VS solution, it'll take me MUCH longer than that.

Any ideas on good exercises I can give them that:
- Will let me judge their ability on doing what they'll be doing later in "real life" - Can be done in 30-60 minutes in "tech interview conditions" (no previous code to rely on, nervousness, etc)

Answer 1

re: as many as possible. One downside is the nervousness factor: if they don't expect this, then they might get rattled, which will skew the results. Do you really want to test for "as many as possible" ?

I'd vote for picking a couple of problems and then ask them to talk it through with you, and make the point that the 'optimal' answer isn't necessary. In this way, you get to see their thought process, their communication abilities, and also if they can collaborate (if you challenge them on their ideas). It seems to me that these properties (thinking, communication, teamwork) are more important than hacking through Euler problems.

Answer 2

I would prefer to select a couple of questions at random that you yourself understand and ask them to complete them. Its not important how long they take, whats important is to understand how their logic process in solving the problem goes.

Only downside with this tho is there's likely to be somebody who has pre-solved the equations on the net, and its draconian to expect them to solve problems without access to API references and the internet.

maybe for the really simple things you could do pen and paper, but I usually prefer to run things through $LANG if I can't be 100% certain it does what i think it does. Logic errors happen and its best to be able to test them.

Answer 3

No, this is a really bad idea. You've said yourself that the job is about CRUD and DB logic; mathematical ability is mostly irrelevant and Project Euler problems (suprise, suprise) require at least a working knowledge of number theory. Some of the developers I've hired recently probably wouldn't get more than one or two of Project Euler problems straight-off (especially not in an interview!), but are good developers nonetheless. Why not test them on something related to the job? There are plenty of programming questions out there that involve databases, web interfaces and whatnot.

Either way, you should read http://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html and http://headrush.typepad.com/creating_passionate_users/2007/02/dont_ask_employ.html. I think they both get across the point quite clearly that character and intelligence is far more important than whether they're able to grok problems that are mostly about prime numbers and suchlike.

Answer 4

Bad idea.

My experience from teaching is that there is almost an inverse relation between a person's mathematical ability and his software engineering abilities (rather than data/structures algorithms).

Inexplicably, the same people who can come up with amazing data structures and algorithms and solve all the mathematically oriented problems tend to build the most boneheaded abstractions and class hierarchies I have ever seen and not understand principles like data hiding and such.

Existing screening practices (especially in larger companies) already focus way too much on the classic CS part and very little on real software engineering skills, which is why we have so much efficient but broken products around.

Answer 5

Yeah the trouble with FizzBuzz and Euler is that inevitably somebody on the internet will have the answer and an eager candidate will have these nailed anyway. I'd prefer to take a problem I've just had to overcome in my own work and discuss how the interviewee would tackle it. Unless your line of business will involve heavy mathematics I wouldn't put the candidate through the Euler mangle. Having said that though it is interesting to see how candidates react out of their comfort zone.

Answer 6

Malcolm Gladwell on the challenge of hiring in the modern world (video). In the talk he addresses the value of tricky questions/tests for job applicants. If you accept his argument, then you should not weigh project Euler questions very highly in your selection process.

Answer 7

We have used Bug Of The Month snippets from the Gimple website as tools for evaluating candidates, We give them a selection of buggy programs and by watching them work with the code we can observe if the at least know how to read other peoples code, how they approach a problem like that and how their analytical skills are. This does not necessarily reflect on how well they will be able to program but at least it gives you an insight into their thought processes

Answer 8

I think it might be better to show them some bad code that looks like the code they will be maintaining, and ask them to comment on it.

For instance, if they're going to be doing LAMP web development, show them some broken PHP with some broken embedded SQL and a painfully obvious injection attack. Then you can see if they know their PHP and SQL. [for arbitrary values of PHP and SQL].

Also, if you want to test multiple kinds of skills, make the code broken on multiple levels: show candidates a buggy implementation of a poorly designed solution to the wrong problem. See how many of these kinds of flaws your candidate spots.

That may go against another goal of keeping the interview—and hence the code—short.

You may want to start out with something easy, and if they plow through it without blinking, a more challenging code snippet. I think that the harder the problem, the less the correlation with a candidates abilities and their success at solving the problem in a short time, so don't let yourself get influenced too much by the results of the hardest problems—except if it's the difference between "Hmm... I think... well..." and "I thought this was the hard problem..."

Answer 9

As you know, some of the more difficult Euler problems require some mathematics that not all CS majors will remember. If you are careful to select problems that have a math/programming ratio reflecting what you want from your candidates, I think it's a good idea.

Of course, that assumes that you value math and problem-solving skills at all. Most of the answers here seem to hold that it's worthless to filter for those abilities. Probably only you can judge whether it's useful for the position you are trying to fill.

The Euler problems are generally easy to understand, and present an interesting but not unapproachable problem. It seems to me like those are very good attributes for interview questions designed to see how someone tackles problems.

Answer 10

How about you ask them if they are a member of any site like Project Euler, SO, and/or is a contributor to an open source project. If they say yes, you know that they are, at the very least, interested in programing and/or problem solving. You could even ask them to log into there account and see some of their contributions/solutions.

I, personally, would find it refreshing for someone to ask to see my work that I did when not under the pressure of someone standing over my shoulder watching me TYPE a solution for FizzBuzz.

Finding out that your candidate is a contributor to an open source project and/or a member of a programming community seems much more valuable than finding out if they can handle the pressure of typing (or writing on a whiteboard/paper) while the interviewer is waiting.

All that being said, I believe any programmer with the minimum math background (basic algebra) should be able to solve any of the first ten Project Euler questions in less than 30 minutes.

Finding out a candidate is a Project Euler member should be a big plus. Most of the questions past 30ish are not for the faint of heart, and although they are quite math intensive, implementing a KNOWN algorithm in an efficient manner is, in most cases, not trivial.