Creating a good developer abstraction layer?

Question

Brad's recent question "How much of your work day is spent coding?" got me thinking about the abstraction layer you want to create for your developers. I am a big promoter of giving my developers what they need to get the job done. I hope that my developer's spend about 80% - 90% of the day coding.

One thing that helps me do that, is abstracting customer support. My developers do not spend any part of the day answering customer support questions. In fact I use two separate tools for tracking customer support requests and my bugs/features, so those questions don't clog up my bug/feature system. Then use an integration technique to push the request into the bug/feature tracking system, if it indeed is a bug or a feature we want to implement.

I was hoping to get answers on what other's think of this approach, since most user's on this site are developers. I sometimes second guess on whether I should get my developer's involved with answering emails or spending some time on the forums when they get a chance to see the frustration some customers have with the software. The benefit of this approach I hope to get is to motivate them to come up with better ways to do something or fix that bug, and that yes, we do actually have customers who use our software.

Answer 1

I have to say not taking questions all day from users would be wonderful, but not taking questions all day from other developers would be wonderful as well. It's pretty much any distractions that will slow you down from coding.

That said I do think the developers still need to interact with the customers. Maybe ask some of your developers to go through a set of trouble tickets every so often and escalate them to bugs if need be so they can get a feel for what's being asked.

Answer 2

I've found it helpful to schedule periods of the day during which developers can or cannot be bothered. That includes internal request as well as that of customers. Of course there always seem to be emergencies that just can't wait, but you have to draw the line somewhere.

Answer 3

One drawback I can see that I didn't mention in my question, is that this could create more work for the developers in other ways. For instance, during a customer interaction, a developer by mistake gives the customer contact information or the customer gets the information another way. That customer then goes around the process and calls or emails the developer directly. Then the developer has to go through the process for the customer, that usually a customer support person would do or the customer would do by emailing the support desk.

Answer 4

To step up a level, I think that there are tasks developers like to do, and don't like to do, and optimizing their happiness will help you optimize their productivity.

Most developers really don't like customer interactions, but some really do. If they're good at it and they like to do it, why not let it be part of their job? Since I joined Microsoft, every time I wrote commitments, some of them always involved interacting with customers, and some always involved interacting with other developers (brown bags, lectures, and teaching).

One of the great parts about working at a small company (like I'd done for almost all of my career previous to Microsoft) is that you get to have your fingers in every aspect of the business. At least, that was great for me... but I can see how others would be a lot less enthusiastic. :)

So the optimum "developer firewall", then, is: providing exactly as much cover from the outside world as you can afford, that your developers desire. I'm sure it's going to be different for each one.

Answer 5

It's a strange dynamic - on the one hand, a developer should never lose sight of the what the customer needs and wants. On the other hand, developers are usually not the best equipped to handle customers (a lot of us lack, um, people skills).

My previous employer hit upon what I thought was the best solution I've seen so far: they made developers "shadow" customer support people for a full day, one day out of the month. The dev could not interact directly with the customer (usually a good thing), but he/she could help the support person with the issue.

This had several benefits:

1. To the developer: he or she gets to see and hear first-hand what the customer's "pain points" are.

2. To the support person: he or she gets the benefit of a software developer within earshot, who can sometimes recognize and fix a problem much faster, or at least expedite making the development department aware of it.

3. To the customer: who benefits from having two brains and two sets of eyes on the issue.

Beyond that, there was a process of escalation that would eventually result in a bug, with a priority proportionate to the customers' pain.

Answer 6

I'd hate to make this a testimonial issue rather than anything scientific, but it goes in line with what @Wing says: Avoid developing in a vacuum.

At my workplace, at initial development we had two coders who literally put stuff together kinda blindly, but it worked nonetheless, after the first couple of client started calling; we sent them out as "support" for the people at implementation (to show the "tech" side of things).

What happened here was astonishing; just from being out there "feeling the pain" (as the company owner likes to call it), these two coders started turning around quicker code fixes, more to-the-point implementations and one of them even went out as far as becoming a sort of liaison for that customer amongst the company. It made the whole client-company interaction a bit more human for them I guess.

BTW we're a 5-man team and we all take the phone if it rings as well as pick up a travel laptop and head out to client immediately if the need arises. We all demo our software as well and consider each to be proficient enough to be able to troubleshoot basic failures if the case arises in public.

Overall the system works real good, but it requires people who are more willing to act socially than programatically for issues (i.e., for some of these things, i've seen programmers shy away quickly) and perhaps this personality type of people don't make the best of the best code, but we get stuff done nonetheless :)

Cheers,

/mp

Answer 7

That sounds like a good idea - removing the ad-hoc interruptions that support issues can provide. I guess the classic approach to support is to have layers of experts, from triage to first line support, etc, and then those people (who are "interrupt driven types") batch up requests that require developer assistance, and then the times when developers are available to be hassled/interrupted are limited.

This is kind of how I work - our business is all about ongoing support contracts, so providing quality support is very important, but constant interruptions while working on the next generation of features is no good either, but I guess you need a mix.

Answer 8

In my previous role we had the concept of golden hours, during these times the development team were not to be disturbed by anyone (e.g. 09:45 - 11:45 and 13:30 - 16:00). This reduced the number of distractions and increased the possibility that we could get through a good coding session of around 2 hours un-interrupted.

This was achieved by insisting that any time members of the support team or business development team came round to talk to a developer, they first had to talk to the team lead.

The team lead would then be responsible for judging the importance of the interruption. If the interruption was deemed to be urgent then they would allow the developer in question to be disturbed, otherwise they would take a note of the reason for the visit and pass that onto the developer outside of the golden hours or simply ask the person who needed to talk to the developer to come back later.

This concept was extremely important as most of the daily work was carried out in pairs. So an interruption of one developer more often than not was actually interrupting two developers.

We also backed this up with a bug fix rota, where each week a member of the team would be allocated to fixing any live support issues. When a live software issues was brought to the team lead, they would check the bug fix rota and direct the support team member to the appropriate developer. This seemed to benefit the team in a number of ways: -

1. Developers would get to see multiple areas of the system. As it was the responsibility of the person on the bug fix rota to at the very least identify the area of software in which the issues was occurring and preferably fix it, their exposure to the code base was continually increasing.
2. The same developer was not interrupted continually. The support team would tend to bring problems to the developers they had known the longest, or were most friendly with. This would result in the more experienced developers being interrupted more often. The golden hours coupled with the bug fix rota helped reduce the number of interruptions experienced by the more experienced team members.
3. Greater team interaction. By ensuring that the support team talked to different developers regarding live issues each week, the relationship between the two teams benefited.

Once the developer on the bug fix rota was asked to help the support team member, they would generally go to the support team area and listen to the support team member talking to the person experiencing the problem, often prompting questions to help identify the root cause of problems. Often the developer would actually take over the conversation with the customer to help speed up the process. Other approaches might involved interrogating the database or examining log files to identify the issue.

(Note: The customers for the team were not customers of the company, but were employees as the software was used to run operations at remote sites. The support team which interacted with the development team was a second level support team dedicated to supporting the software written by the development team.)

Doing this exposes the developer to the pain felt by users of the software and educates the support team in detecting recurring issues, therefore hopefully reducing the time spent detecting the cause of future issues.

The support team would only bring issues to the team lead when they were causing operations at the affected site to stop. Such as error messages appearing on the main screen or new work lists not being produced. However they would batch up smaller issues and bring them to the attention of the team lead when they had a good number of them. Such as incorrect data on reports or screen layout issues.

To finish off my rambling, I think that it is essential that developers are interrupted as infrequently as possible, there are many references out there that discuss how context switching is detrimental to developer performance. However I feel that it is equally important to keep them aware of the fact that their software is used by real people, who rely on it to get their jobs done.

Answer 9

Developers like to say, "I'm not interested in support. I just want to write new code." That is like saying, "I don't want to be a parent, I just want to have sex." The reality is that developers who do both new projects and maintenance work become better developers through exposure to the larger picture of a system's life.

I complain as much as the next developer about the interruption of production support, but in my opinion the best way to create a productive abstraction layer is to be a better developer and write applications that have less need for the abstraction.

But if it's too late, and the company has invested heavily in a maintenance-intensive app, then you don't have a choice, support has to be managed. But when a developer has project work with deadlines, support and project work fight for a developer's time.

Here's what I've seen done.

• Developers share support load. Whether there is someone designated to take in all the requests and distribute them accordingly, or each developer takes a rotating week on the support line, the work is divided and shared. This is similar to what Mauricio and Aiden have mentioned above.
  Verdict: Like Mauricio, I have a five person team and this is what we do. It works, but doesn't really solve the problem of giving project developers uninterrupted time on a project that runs many months.
• Split the team and swap for each project. The support team blocks interruptions for the project team, but the two communicate regularly. At the end of a project, the teams switch: the project team supports the changes they've developed (they're the best ones to do so at this stage of the new functionality's life) while the support team shifts into making new code. This way no one is stuck doing support all the time, but when someone's in project mode, they can focus. Of course, the success of this would be dependent on all devs on the team having strong skills in both types of work.
  Verdict: I proposed this to my team once and I got a lot of pushback. Nobody liked the idea of having to work a long stretch doing support. Your mileage may vary.
• Dedicated support and project teams. All support is handled by one team, and all project developers focus only on new development.
  Verdict: My understanding is that the project developers love this, and if the support devs like it too, then everyone's happy. But I think it leads to the project guys working in too much of a vacuum and creating apps that continue to have huge support needs. I think people need to eat their own dog food, and that doesn't happen with this approach, even though it sounds great if you can find the right people for it.

Answer 10

@AidenMontgomery - Since your answer got voted up the most, I think I am going to start looking into implementing golden hours and support rotations in some way. Thanks everybody for the feedback.