how to determine if someone will make a good programmer

Question

My current company is starting a development project with another afiliated company. We are a programming shop but the other company has no developers.

We will be creating an app for that company but then the company will take it in charge so they want to have two developers of their own for this, to be involved in all stages of the development.

The story is a little bit long and complicated so there is no point in further detailing.

Bottom line: we have 10 candidate people from that company (most of them never touched a programming language) and we have to select 2 to become full time programmers.

How do we identify the best 2 persons? That's the question.

Is there a test that you can give to a random person taken from the street to see if they will make a decent programmer? What should we look for? Problem solving skills, logical tests, explain something to them and tell them to explain it back, make them learn a language then ask them to write stuff ?!?!?! What?

How would you procede? I searched for similar questions but could not find something to satisfy as an answer.

P.S. The language they will have to learn is Java (EE) and we have about 1 month to decide on the persons.

EDIT: I have to mention that we have a month to select them. The project starts in a while so they will have sufficient time to learn stuff. We won't just throw them head first into the project. They will work side by side with us (starting as junior programmers - from scratch) until the project is closed. Then they will pick it up and continue on their own. We need two juniors, but we need them from those 10 persons.

Answer 1

Back in the 80s, IBM downsized a lot of its operations. At the time, they had a no-layoffs policy, so they moved the non-technical people being downsized into programming roles. The results were an unmitigated disaster. Your attempt to turn random people into programmers is likely to be another.

Answer 2

It depends what level of programming experience they'll need. If they're just maintaining your code, they shouldn't need much knowledge. In which case taking a handful and teaching them basic Java and debugging shouldn't take too long. Pick the ones that take to it best and the job is done. However I'd recommend stressing that they still lack sufficient training and that the company would be better off hiring serious programmers before attempting anything ambitious.

Answer 3

If none of them is a programmer yet or hasn't even done smaller projects on which you can trying to find out how motivated and talented a person is then the only solution is: fire two of them and hire two good programmers instead. It takes 5-10 years experience to became a good programmer anyway.

IMHO there is no simple test, i become more and more convinced that programming is a skill you can't learn, like people have or don't have musical skills.

But do an interview and find out who at least want to become a programmer, if they only want to do this to save their job you are screwed.

EDIT: And by the way even an already good programmer needs a lot of time to learn Java EE.

Answer 4

There are standard aptitude tests people take before they go to college and they try to point them in the right direction.

Google has 52,000 results for programmer aptitude test.

You can search and see if you find anything you like

Answer 5

Consider having a look at the Joel Guerrilla test: http://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html because that is essentially what you need to do.

Also I've recently seen an article describing that the crucial point whether people can become good programmers, is if they can grok pointers and recursion. If they cannot even early, they never will. So emphasize particularly on that in your interviews, because you need to find those who can.

Also ensure that they actually write code while evaluating them.

Answer 6

Pick the 2 people that most enjoy solving puzzles, like Sudoku.

You'll still have a disaster on your hands, but at least the 2 you picked will be most likely to take to programming.

Answer 7

Your best problemsolver may be a mathematician completely uninterested hence unmotivated to make software. Check their motive and don't leave people on their own (avoid "mushroom management")

Answer 8

Anyone who passes this test, Sun Certified Java Programmer, before they begin working side-by-side with your team, has proven that they can learn quickly and are fairly interested in the subject.

Having said that, the odds of finding such individuals in a random group of people are very low.

Answer 9

Instead of filling Junior Developer positions, I would get these guys to be white-box testers. The last thing you want is these guys committing production code. If they are just writing unit tests, then they are less destructive. And, if they are decent, you might get something out of these tests. If you do this, I would get your applicants to try to black-box test a piece of code that has known bugs. The two that find the most will be the best testers. As for the programming responsibilities, I don't think there is anything you can do about it. I would just minimize the damages in order to humor your customer.

Answer 10

Wow, crazy situation.

You cannot test skills so all you have is aptitude and personality to go on.

I would personally give them some small project(s) to do. Something like you would see in the beginnings of a programming class. Offer to be a mentor and encourage them to ask questions.

Here is an example using Mono (C#) on Linux to get a feel for the kind of thing I mean:

http://www.tuxradar.com/hca

Keep track of their progress and questions, review their code, and interview them about how the process went from their point of view.

Any potential programmers will be hooked. You should be able to see their passion, creativity, curiosity, and maybe even some laziness in terms of a desire to streamline or automate or improve tooling.

Look out for anybody that finds the process tedious, wants to be spoon fed, is frustrated that the demands are too high or anything else that will bite you.

PS.

I realize that the Hudzilla exercises that I reference could be run on Windows as well. You would have to provide instructions to install MonoDevelop on Windows, provide it installed, or use a different IDE. Of course, another language may make more sense in your situation. This is just an example that I think any real potential programmer would be able to follow.

Answer 11

Get them to set up a stackoverflow profile and then check what they post over the course of the month while you give them some progressively more realistic assignments.

You can be totally transparent about it, emphasizing that you want to see how well they can formulate questions (and/or answers..). It will also help take some of the training load off of your staff.

I'm assuming that for the kind of money you should be charging for this you can easily afford to pay them for those hours. After all, they're going to get tutoring from your staff, taking away lots of productive hours from your business.

Answer 12

The truth is, might as well pick randomly.

When I've interviewed candidates for programming positions, we hire way less than 20% of candidates. That means even if your 10 people all had years of Java EE experience (as opposed to no programming experience at all), I still wouldn't want to pick 2 of them.

And even if you could pick the best candidates from any number of potential applicants, and all of the candidates had education and experience you were looking for, it still wouldn't help. The industry success rate for software projects ranges from (depending on who you ask) as high as 40% to well under 10%. So the project will probably fail regardless of what you do.

I think the situation today is like post-WWII test pilots. The technology was immature, so chances of failure were high. Test pilots were dying at the rate of one per week, but you didn't dare suggest that a pilot died because of bad luck, or aircraft malfunction, or anything like that. It was always a personal failure: he just wasn't good enough. The living pilots are still living because you're better.

I see a lot of programmers today acting the same way. You need to hire the best people, with years of experience showing they can ship products! They need top technical skills! (We can make them answer strange trivia questions, that make the Right Stuff's astronaut selection tests look downright normal.) If a project crashes and burns (as most do), it can only mean the programmer wasn't good enough. It wasn't mismanagement or shifting markets or some other external factor. Never is.

The answer, then, is to look at whatever stupid meaningless tests that Microsoft and Google and Apple give, and do the same kind of thing. The results will still be no better than random, but when the project fails and your bosses ask why, you can turn around and show them that you did the same thing that the top software companies do, so it was obviously the personal failure of the two programmers, not the process and certainly not you.

Answer 13

This is a WTF.

I say give the position to the guy who wants it the most. If A is very smart but doesn't care about programming, and B is mediocre but loves to have this opportunity to change his career, B is my choice.

Answer 14

I recommend reading Teach Yourself Programming in Ten Years and A cognitive study of early learning of programming.

Answer 15

If there was a single reliable way to test what one should do for a living the world would be a better place and everyone would use that method by now. Yes yes I know there are many career tests but I haven't yet seen a single reliable one.

Your best bet would be to take a set of normal interview questions and pick the algorithmical ones and see who answers them best (like for example the google questions http://www.mytechinterviews.com/10-google-interview-questions). But even then you have 0% guarantee they will be into programming...

Answer 16

What your project needs (and probably won't get) is a person who is both good at programming and understands the business.

Forget getting the person with the most programming aptitude. Get the person who understands whatever business you're in the best.

In the few weeks that you have, they're not going to be able to learn syntax, OO principles, standard libraries, and on top of that whatever you've written. I suspect the most you can expect of them is to be testers and novice tinkerers. So better to pick the guys who al least know what the product is meant to do.

This is how I started in programming. I wrote really bad code for years for various trading desks, and but it worked well because I understood the financial side. Then I gradually read more and more and adapted my programs with new knowledge learned from hours of reading about patterns, libraries etc. If you know the business, you'll know what tools will be useful.

Answer 17

Your experienced programmers should write some thorough test suites (assuming they haven't already), and the candidates should spend the month trying to write code that passes it. Maybe they're expected to do this in off-hours, to show that they are really eager to succeed. The test writers can tag the tests with difficulty levels, and maybe cross reference some of them to sections of a reference that have info on what they'll need to understand to write the code for that test.

This way, you avoid wasting a lot of effort, and you negate the risk of having inexperienced coders make sketchy contributions to the enterprise codebase. Assuming that you're writing unit tests anyway (which, under the circumstances, seems absolutely essential), the only extra effort you invest is in rating the difficulty of the tests and in adding links to reference material. Maybe you can use some of that later if you need to bring in or test additional dubious new hires. Theoretically you could open-source your code down the road a bit and get a book deal out of it.

It's great for the candidates, too: they can progress smoothly from easier to harder tasks, and they get that glow of accomplishment when they pass new tests, boosted by the feeling of making a genuine contribution to the codebase. Well, it's not so great for the ones that don't turn out to do so well, but at least they will have had the experience of it. Maybe they will decide to really dig in and learn more, or maybe they will be more content with whatever they were doing before this roller coaster came round.

For the sake of your financial department, it might be wise to stipulate in the contract that you can cancel it unless two of the candidates manage to pass a certain percentage of the tests. You might also need some way of ensuring that they are actually doing the work themselves, though this could apply in any such circumstance. I guess the threat of legal action functions as an effective deterrent against such strategies. Also it will be pretty easy to spot "plagiarism" due to obvious similarities between their commits.

Oh yes: you are hopefully using distributed revision control: git, hg, etc.? This way you can have them each make all their commits to a completely separate repo, and then just pull in the ones that are actually useful. svn has its uses, but serving as a shooting gallery for experimental firearms is not one of them.

Answer 18

I'll take your "specification" at face value and accept you cannot change it.

I'd go for someone with an aptitude for mathematics. Give them math tests until no one can correctly answer them. That is, find their level of incompetence.

One can only hope someone trained as an accountant would do better than a shipping clerk. At least from a probability stand-point it would seem just a tiny bit better than picking at random.

I read once that a common trait of good programmers is sustained concentration. Look for that as well.

But yes, all the disclaimers above do apply. Who knows in the end. Good Luck.

Answer 19

This might seem like a joke but I am serious. Ask them about their childhood. Try to find out if they were the kind of kid that would try to "repair" broken household appliances. Or if they "repaired" things with their father. Or if they every wanted to just build something with their hands.

Maybe I can illustrate my approach better with an example:

I have a very good friend, who doesn't have allot to do with computers. But one time I went to his place and found out that he build a bookshelf for himself because he didn't find anything that fit his room. I could see that he was proud about it. He really had fun. IMHO he would make a good programmer.

Even if the person you are interviewing didn't actually build anything, the wish to do so is very important. You will be suprised how many people would never even imagine building something on their own.

Ofcourse this approach is very coarse, but it would eliminate allot of candidates in no time.

Answer 20

There was a small study done by Saeed Dehnadi, Richard Bornat on students on a programming course.

At the start of the course they subjected the students to a short aptitude test on assignment. After several weeks of lectures they gave them a programming test. They found a high correlation between students that had formed a consistent mental model of how assignment works (before being taught anything) and those that performed well in the programming test after some lessons.

The aptitude test consisted of 12 questions similar to this:

Read the following statements and tick the box next to the correct answer.

   int a = 10;
   int b = 20;

   a = b;

The new values of a and b are:

   [ ] a = 20 b = 0  
   [ ] a = 20 b = 20  
   [ ] a = 0  b = 10  
   [ ] a = 10 b = 10  
   [ ] a = 30 b = 20  
   [ ] a = 30 b = 0  
   [ ] a = 10 b = 30  
   [ ] a = 0  b = 30  
   [ ] a = 10 b = 20  
   [ ] a = 20 b = 10 

The full paper is definitely worth reading (It's only 20 pages). The full aptitude test and answer sheet can be found at the bottom of Saeed's homepage along with several other papers on similar topics. If you do use this, remember that you aren't actually looking for correct answers to the aptitude test but consistent answers.

The Answer sheet lists which answer maps to each possible mental model, and the Instructions on using the mark sheet explain how to mark to obtain a score for consistency.

Jeff has also written a blog about this paper

_{All that said, I fully agree with everyone else who has said using non-programmers as software developers is a disastrous idea. Even if this test can manage to successfully highlight some people in your group who have a natural aptitude for programming, software development is a discipline that takes years of practise to learn and decades to master (if ever). So go ahead, try this test, but unless it turns out that one of your guys goes home and spends 6 hours a night frantically coding open source software in his/her spare time my advice would be to keep non-developers off your dev team.}

Answer 21

Give them a month to learn the programming language. Then get them to write a mathematics program (Using console or forms) that solves simply problems. When I got my first start in programming, I had to do a test to output the first XX prime numbers, work out factorials, do a "lastindexof" in a string (Without using the native function), and a bunch of other problems. It took me less than an hour to complete, and I got the job.

At the end of the day, if you really need to hire one of them. Algebra + basic programming skills is what you want.

Answer 22

The approach put forward in this circumstance is very much flawed. If you turn the question around, and say, human resources are trying to find someone to fill a position who comes from a programming background, lets devise a test and see how they perform.

You are hte test candidate, and this is your approach.

Anyways, thats one point - the actual answer to your question is simple,

The people that should be chosen are the people with the most enthusiasm for the job and who can demonstrate why they want to take on the new job. Out of the 10 people some may want a career change, some may associate the programming with an eventual rise and so forth. These are the types of carrots that will most likely make these people better programmers.

Lets face fact, there are plenty of horrible programmers out there.

Answer 23

Avoiding the meta-discussion horse - that's been flogged to death and beyond....

1. Find the people who want to do the job and select...
2. ...the people who were competent at algebra+ maths and select from them...
3. ...the people who like taking things apart and learning how things work.

Those 3 indicators cover probably 80% of programmers.