How do you manage non-user facing work in a strict scrum shop?

Question

We're a medium sized engineering shop (10-20). We are great at prioritizing and structuring work on our user facing stories and making customers happy. But the cobbler's children have no shoes. If it isn't about customers, we have 0 process.

I'm looking for systems to ensure we correctly prioritize and accomplish the non user facing work to keep a dev shop running: QA environments (pretty heavy, in our case), continuous integration systems, the packaging, and so forth.

Now, resources are always limited. We don't want to give the cobblers children 10 pair of the fanciest shoes, and specialized bike shoes to boot. We want to do the right, necessary work, with the same scrummy discipline that is applied to the rest of our development.

Tell me what system works for you: how to you prioritize and organize non-user facing work ? I want systems that are simple and integrate smoothly with scrum.

(I'm aware of a red box at the top of this text, indicating that Stack Overflow's automated question parser thinks this is a subjective question that can't be answered - I think there are likely 2 or 3 excellent answers that can be or have been proven viable - and process is integral to programming. So here is some psuedocode representing our process. Fix this algorithym).

IBacklog GetBacklogForWork(IWork requestedWork)
{
    if(requestedWork.IsUserFacing) return new PrioritizedBacklogRepository();
    // Everything else. Priority largely based on spare time and who thinks its a neat idea
    return new RandomizedPriorityRepository();
}

void HandleIncomingSuggestionsForWork(IEnumerable(IWork) ideas)
{
    foreach(work in ideas) GetBacklogForWork(work).Insert(work);
}

Answer 1

IMO something like "QA environments" is, in a sense, user-facing work.

Quality is admittedly a "non-functional" requirement (so there isn't necessarily an associated "story"). But, you may have a non-functional requirement like "the software must be tested before it's shipped". You can assign a relative priority to such a requirement ("how important is it that software be tested?"), and then execute as usual (decide how to implement that requirement, estimate how long it will take to implement, schedule the implementation, assign the implementation, etc.).

Answer 2

What we do where I work is to have a percentage, right now around 15% give or take a few percent, that is spent on internal tasks that are non-user facing work. This way the technical debt is handled and if the task backlog becomes rather large then a sprint may be spent on it instead of new functionality. The way that last one would get pitched to the user or customer is that there will be a time where just maintenance and preventive work is done so there aren't any new functions coming after the next sprint.

That's one idea that can be tweaked a bit though it isn't necessarily fully flushed out yet.

Answer 3

The way I've seen it work more or less ok is to try to do as much as possible of the non-functional/non-user-facing as PART of a related user-facing activity, or the first user-facing activity that requires it. This is the easiest to cope with, as it just reflects the needs of the development organization in order to maintain sustainable velocity moving forward.

Additional work which cannot be related will be done using a percentage as described by JB King.

The alternative of pitching it as an investment with such and such ROI to the PO is a theoretically sexy concept, but with real life POs I've seen it rarely works. Its very hard to get POs to understand the investment, not to mention actually being strong enough to delay functionality for it.

Its sometimes the difficult role of the development teams to be the guys that "slow things down" in order to keep a sustainable situation.

Dev managers sometimes feel really bad about this whole sitation, regardless of the chosen approach. My recommendation both as someone who's been in that spot, as well as an Agile coach, is that as long as you feel you are doing the right thing for the business, focusing on non-functional work that is required NOW and has a relatively quick ROI, you should feel ok about this.

Cautionary note: This is an area where self-organization is really put to the challenge. Organization needs to trust the team to do the right thing, and the team needs to earn and not abuse that trust. Its a sign of maturity for an individual or a team to know the right balance.