Resisting regression to the mean in large developer organizations.

Question

How does your organization resist regression to the mean. By this I mean how do you keep the "accepted practices" from domination, stifling innovation and original thinking. Do your architects dictate methodologies so that innovators have to hide their work? Is the path chosen which the average programmer can handle in favor of the more elegant because it is too complex?

What does your organization use to avoid this?

Answer 1

• Frequent code reviews
• We have a requirement that you read and keep up with trade journals, relevant blogs and news sites, etc. Once a month we all have to talk for fifteen minutes about something new or interesting relevant to our work. If you're working with people who really enjoy their job then they'll probably be doing this for fun anyway.
• Pair programming.

Answer 2

I guess I would need to know what you mean by elegance. I typically think of elegance, with respect to programming, as implying simplicity, understandability, and efficiency. Obviously there are trade-offs between the three; and some organizations can tolerate (or require) higher levels of complexity in order to improve efficiency. In these cases you'd better make sure you hire top-notch programmers.

Simplicity and understandability seem to go hand in hand. Refactoring, I think, is the art of taking complex, smelly, inelegant code and making it simpler and more understandable. I guess I wouldn't change things to be more complex unless there was some significant, justifiable gain in efficiency or capability. As an example, I would say that using AJAX will result in more complex application but it can result in a much better user experience. This is complexity that I would be willing to absorb. On the other hand, why develop full-fledged email capabilities in your web application when a mailto link actually does everything you need.

I guess all of that is to say that there is innovation for innovation's sake (we all like to play with new toys) and innovation to improve business value. I would try to discourage the former and encourage the latter. Give people time to explore new technologies/techniques but insist that those technologies/techniques demonstrate value before adopting them. Where you have existing solutions, refactor mercilessly to improve simplicity, understandability, and efficiency, knowing that if you refactor towards efficiency you may need to hire accordingly.

Answer 3

architectural review boards, new-technology committees, design-approval meetings,...oh wait, you said resist regresstion to the mean ;-)

in that case,

• remove disincentives for trying new things; this can be as simple as making it okay to play a little bit and buying shiny new tools to see what gets done with them
• allow experimentation and failure; this gets a bit more risky, but keep the timeframe short and impact small and you'll be ok
• remove barriers to progress like committees, review boards, and so forth; these will inevitibaly turn into fiefdoms and old-guard strongholds even if their charter is to explore new ground; as long as failure is punished and success is expected, there is no real incentive to innovate and strong disincentives to try new things
• look at new technologies from a return-on-investment perspective over a longer time frame than a single pilot project, e.g. if WPF (for example) makes the GUI 8% snappier and the users 3% happier but takes the developers 10% longer, multiply that by several projects over the next few years and the trade-off is worth it (and the developers will probably improve)

Answer 4

A few ideas, not necessarily in order, not necessarily compatible with each other, and definitely not used in my organization:

• Create small, temporary teams that manage themselves like Gore and Associates. Allow teams to change personnel. Teams that value predictability and have no risk tolerance will drop innovators and pick up predictable developers. Teams that value creatively and have a high risk tolerance will drop predictable developers and pick up innovators. In both cases, it’s better for the teams and individuals.
• Use blackbox management on your teams. Reward performance versus adherence to standard process. As long as a team communicates well and honors any agreed interfaces, let them work in the way that best suits them.
• Make teams support the software they create. This removes the, “make sure it’s simple enough for the support team to maintain” argument. If a team wants to try something technically difficult and knows they can maintain it, let them.
• Share the big picture with all teams. Constantly evaluate feedback from teams. Look for insights into problems outside a team’s problem space; sometimes it’s easier to understand a problem from the outside.
• Let teams earn risk points. If a team delivers a series of stunning successes, allow them to pursue riskier endeavors.
• Stop focusing on the bottom line. Cost/benefit analysis is most accurate in the short term and will usually suggest low risk efforts. If success were as simple as cost/benefit, companies would never fail. Realize that you might have to swing for the bleachers for long-term success – greater risk leads to greater reward.
• Describe problems without technical assumptions. "We need a way to automatically restock our partners’ inventories" is full of possibilities. Whereas "we need a .NET web app (we’re a .NET shop), probably C# (we’re best with C#), use our in-house eComm library (it’s already paid for)” is hard-pressed to innovate.
• Listen carefully when a team suggests technology you're not currently using. Just because you're not using it doesn't mean it's not the right tool.
• Embed architects in the teams versus keeping them a layer above handing down directives, libraries, and UML diagrams.
• Give developers 20% of their time at work to pursue projects that interest them in ways that interest them. Keep an inventory of 20% projects and their owners so people can browse and possibly contribute.