Release a lot of buggy features quickly... or few really stable ones?

Question

Im curious what your preferences and thoughts are on the idea of doing as little testing as possible behind the scenes and rolling our as many new features as possible, as quickly as possible, and testing on the production site, or troubleshooting them to hell until they're bulletproof, and then releasing them to the public.

Answer 1

Rolling out code to a production server with 'as little testing as possible' to get it live quicker is setting yourself up for a life of pain. What you're suggesting really is to get your users to test your system for you, that would be a beta program, but even before you get there you should have performed a good level of testing and be confident that the app works, otherwise you're not going to keep many users for long.

From a developer perspective I would only be happy releasing code that I am confident is working as planned. From a user perspective I wouldn't want to be using an app that kept falling over, no matter how early in the development cycle it is.

If it's not ready, then don't release it.

Answer 2

I think it depends on your userbase somewhat. For example, I choose to use cutting edge and less stable features on my linux box. But I think in general, and especially in web development where generating pageviews is usually a high priority, you want to go with what works for most people, and that's stability.

Answer 3

• You should test and review the code during development, before the feature is even finished.
• You should test the whole feature for functionality before moving to production.
• You should release a small number of features often, so that you get feedback on the feature. Even if the feature works perfectly, it may still not be exactly what the user wants, or you find that something can be improved when the feature is used in practice.

Answer 4

Perhaps a middle ground may be more appropriate. Your "brand" will suffer a great deal if either:

• the software you release is a steaming pile of dung (as in your former cases); or
• the software is not released in a timely fashion (as in your latter case).

In both those cases, you won't be very likely to stay in business for long.

The shop I operate in recognises the fact that software will have some bugs in it. All high severity bugs must be fixed before release and all low severity bugs must have a plan in place for fixing after release.

Software maintenance (basically bug fixing and answering customer questions) is an important part of our development process.

In addition, the "cost" of fixing a bug becomes more as the discovery of said bug moves away from the developer and towards the customer.

Fixing a bug I find during unit testing involves only me though it can affect others if my stuff is delayed.

Finding a bug during system test means other phases are definitely delayed since the code has to come back and be changed and unit tested again before once again being promoted to system test.

Finding a bug after your software is live is a whole other world of pain, involving communications with customers, multiple managerial reporting lines all wanting to leave an impression of their boot in your rear end and putting any bug fix through all phases, or risking adverse effects otherwise - a particularly nasty place in the ninth circle of hell is reserved for those developers who, in fixing their bug, introduce yet another one.

Answer 5

If you are performing a beta or alpha test with a handful of people, sure. However, if this is code that is meant to be used by the general public or your business, then no. Buggy code reflects poorly on the programmer, and I know that when something crashes or behaves unexpectedly it tends to annoy people.

Therefore, I would much rather release polished, thought out code that may not have as many bells and whistles than code that gives people a poor experience.

One footnote, however, is that you must know when enough is enough. Every programmer can spends eons going over every line of code, saying "Well, if I move this to here, I can get a .001% speed boost" or anything on the same line of thinking. Believe me, I have this problem as well, and tend to obsess. The skill to say something is "good enough" is a hard one to learn, but it is absolutely necessary, in my opinion.

Answer 6

It depends on the pain levels, and expectations, of your customers and how well your customer facing staff can manage their, ahem, 'feedback'.

If your customers are expecting to go quickly to high volume mass production, on a very tight schedule with fierce competition, with what you're delivering them (think consumer electronics like mobile phones) then they won't thank you at all for any surprise. They'll be very scared of having to recall hundreds of thousands of units for an upgrade.

Perhaps you're delivering to someone who's also doing research, a university department or similar, who may bend your delivery to fit a purpose that it's not intended for. They don't mind, may even expect, problems and are happy to find a way through. They may well be excited by the features and forgive you the bugs as long as they find you're listening to their feedback.

The most skillful customer facing staff I worked with were able to judge how long it would take the customer to notice the deficiencies in the deliveries we were providing, how long it would take us engineers to plug the gaps, and realise that by the time the customers noticed the problem we'd have a patch. The customer gets an early delivery so the contract is secure, is not too inconvenienced by the bugs, is happy with the support, all in all a happy world. It's a tricky call though; If you don't release anything until it's perfect you'll never have a customer as someone will always undercut you yet release something too early and disappoint then you're going to be replaced when the opportunity arrises. Get your judgement of the patch development time wrong and your customer will be unhappy.

In short it's something about open communication, something about bluff, deceit and deception, and a whole lot about judgement to know which to do and when.

Answer 7

It rather depends on the desires of the management than on the desires of the customers. Given a choice of 'you can have it working, or you can have it Friday', the average target-and-goal loving manager will prefer to have it Friday.

If you actually have a choice, please leave it until it works. You'll save yourself and everyone else a deal of time and trouble.

Time(do it right) < Time(do it again) + Time(correct database) + Time(explain and apologise)

(Fundamental law of software engineering.)