How do you verify the Work and Experience level of Consultants?

Question

So, right now I am hunting for consultants for two technologies I have knowledge about but no experience in. I have put ads on Craigslist as these are two small tasks and most likely the larger firms wont want to be bothered and will charge an arm and a leg.

How do I verify that these guys are on the up and up? That the work they are sending me is THEIRS? I mean, I'd love to believe that the guy that didn't sign his email did all the flash work for Guess.com and is trolling Craigslist for gigs, I just don't think it's likely.

I also don't have much time for this, so anything I can do will have to be fast (thus the reason I am hiring consultants).

UPDATE

The problem is, this is a small company, and I AM the IT department. I mean, I know enough about Flash to ask some high level questions, but not much. The other thing is a Mac product called File Maker Pro, which I know relatively nothing about.

The point of this is to save me time so I don't have to learn the technologies and do it. Are there reputable firms that handle small things like this? I suppose that may have been a better question.

Thank you.

Answer 1

Ask them to forward you a portfolio of the projects they've worked on with a brief summary of their input on each project.

Answer 2

I would say the same way you would work with any job candidate. Phone screen them, and then (if possible) do a in-person interview. Ask them technical questions and ask them how they would go about solving your problem(s).

You will have to make a judgement based on their answers, assessing whether they really know what they are talking about or not.

Remember to take their references with a grain of salt. Remember they wouldn't give you a reference to contact that might speak badly against them so it is all very biased.

Answer 3

Get references and contact them.

Answer 4

Some ideas:

1. Ask for references and speak to them in detail about the person's responsibilities.
2. Ask for samples of their work.
3. Use Google Code Search if you suspect that their code came from elsewhere.

Answer 5

Ask them for a portfolio and make sure you contact the people who commissioned some of their previous work and have a short chat. Most people won't mind you calling to check.

Only using provided referees might mean you talk to somebody who's been instructed to give a glowing review.

Answer 6

Well, you get what you pay for, and using Craigslist for high-level technology hires might not be the best way to find the cream of the crop.

The most telling tool that I have for weeding out the liars and posers is a good, grueling phone-screen. If you don't know anything about the technology, then you've got a problem. You need someone knowledgeable, who you trust, to kick the tires on these guys.

You could always ask them for their Stack Overflow reputation score.

;)

Answer 7

If you are not confident at all of being able to weed out the charlatans yourself you need to use someone who is confident, for example:

• A trusted colleague/friend/contact
• A reputable recruitment agency
• A reputable consultancy

An agency or consultancy will have a cost involved, but hiring the wrong person will also have a cost!

Answer 8

If you are checking for technical compentency, you could see if you could find a beginner's tutorial on the technology with some questions you could... borrow. There is a large gap between the merely clueless developers (who should be able to answer that easily), and the non-developers. Also, watch out for long lists of claimed technologies (more than about 5-8?), that (almost) always means someone who doesn't know what they're doing.

Of course, if you could simply pay on delivery... :)

Answer 9

It sounds like you got sample source code from the consultants. A good way to confirm that the source code is theirs is to look through it, find some logic that seems complex or counter-intuitive and ask why it is that way. They should be able to answer that if they wrote it :-)

Answer 10

Ask what their rep score is on stackoverflow.

But seriously, you might try searching forums for flash and filemaker to find knowledgable users and/or ask on those forums for good interview questions.

Answer 11

A personal referral from someone you know would be ideal if it's possible.

As an alternative, I agree with the other suggestions to perform a thorough phone screening. If you don't feel qualified to do that, you probably know someone who is.

This is definitely the sort of thing where knowing and trusting highly qualified people is important, even if they are not specialists in the skills you seek.

Answer 12

As the other posters have noted, having them provide samples of their code is a good way to check their knowledge base. However, this isn't always possible due to NDA agreements with companies so don't reject them if they can't provide anything. Having them provide references and actually calling the references is a good way to found out information on them, but since they have already vetted their references you will likely only get positive feedback on them.

Another route that you might want to think about going is to have them come in and then give them a competency based interview. If you can find some questions about the products hat you can answer and/or code then they should be able to do the same. If you ask someone a question that requires code then they should be able to answer a basic question in a reasonable amount of time at your site. Just make sure that the question is simple enough that you can do it yourself in a reasonable amount of time.

Answer 13

I personally agree that a good phone screen is a good thing, but ideally you would find a person that you could trust that DOES know the technologies that would be willing to help you get through the phone screen.

You might have to pay for an hour or two of someone's time, but if you want to go the route of cheap labor and still get some level of quality it might be the only way.

Answer 14

Beyond asking for references and checking them, as other said, I'd recommend that you try to minimize risks.

The best way to do this is to work with them in short iterations with working deliverables at the end. By short I mean one or two weeks, and having something that you can try out and evaluate. If you can do a code review then you may check not only that what they built actually works, but how it is built.

References can be tricky. Any consultant provides their most succesful projects as a reference, but not their failures, and even so, the people working for you may not be the same, or may not be in the same mood than in previous jobs.

Moving ahead in baby steps is always a good way to assess their quality.

Answer 15

Another way to test their competency is to give them a real-life task that you have on your plate, limited to 4 hrs, and ask them to do it. If they complain about doing free work for you then you don't want them anyway(as they are likely to complain). That is the best way to get it at your level, as well as to verify that they can do the programming part of the job :-). This is how I got my current job, FWIW.

Answer 16

A technical test is not out of the question and to be expected. ask them do do something trivial (nothing more than a couple of hours worth of work) and walk you through it afterwards.

Give them a pc and net access while they are doing it as most developers would have used the net as a reference while working anyway...

As for formulating a test... maybe post a question on stack overflow for a sample test?

Answer 17

We have a multi-step process that you might be able to tailor to your situation.

1. 5 minute online test where you are given one of several questions that asks you to write a simple algorithm to solve the problem presented. FizzBuzz is a good razor for this.

2. Ask for resume and answers to five essay questions to get a feel for their personality, their work habits, communication skills, and genuine interest in the position. If they're not willing to do this, we aren't interested.

3. Ask for a code sample in advance and talk about it with them during the phone screen before asking them our gauntlet of technical questions. If they can't discuss how they might improve the code or respond to a change request for it we make up, then you can bet they didn't write it or understand it. We look for best practices, as well as their ability to discuss the code with us.

4. Technical question gauntlet during the phone screen. Ask the questions you can't really Google while you're on the phone, such as describe the ASP.NET page lifecycle, stopping them on specific parts and asking for elaboration on things such as output caching.

5. One-on-One with the Development manager (on-site)

6. 90 minute hands-on test. (on-site)

7. Lunch with the development team to make sure they will mesh with the team well, regardless of their coding abilities. This has proven to be a surprisingly valuable indicator of how consultants will work out.

8. Face-to-face with the development team. We discuss how the candidate did on the Hands-on test and then spend the rest of the hour on a handful of technical questions (not many since they proved their technical ability on the phone screen), and mostly as them softer questions about their work habits, team dyanamics, best practices.

9. The most important question you can ask any candidate is why they are leaving their present employer. If you want someone who will be able to contribute to your team long term, and really feel a personal sense of pride in their work for your company/project, their answer to this question is key. If they only bash their previous employer, pass. If they are only looking for more money but really like their project, pass. If they've peaked at the present position, consider them. Use your judgment on this one, but always ask it.

10. Get feedback from everyone who interviewed him. If at least one person is excited about them and there isn't anyone who gives a firm thumbs down, then they are probably a good choice. If everyone is luke-warm with no specific red-flags, that in itself is a big red flag!

11. Check references for people they reported to and give them a call or email.

12. Google them

Answer 18

1. As others already said: Ask for references and check them up.
2. You are not just asking one guy, do you? So use the ABC technique: Call A and ask some questions about technical details. Then call B and ask them what they think about A's answers. What would they answer. Do the same with B and C. Then you can go back to A. That should give you a good idea of who knows the details.
3. Listen to what questions they ask you! Can they work your question into a real problem statement?

Answer 19

I am sure there are good consultants on CL, but it is just too hard and risky to verify; even if they gave you reference, how do you know it's real. And code samples can mean a lot of things. They may be copying the code from other sources and yet you won't find it in code search; or you may find bits and pieces of the code on google but they are just good developers using open source framework, etc.

If they are local, I would definitely ask for an onsite visit and talk to them in person. You may not be en expert on the technology, but you should know it well enough to ask some general questions.

And I think there are many sites to hire contractors for small projects; they usually have good rating systems, do a search and try these.