How Much Designing Should Go On Before Any Coding Takes Place?

Question

I'm currently in school and for my Senior Project we have to spend 1/3 terms just doing UML diagrams and other tedious documentation for our project.

This involves alot of designing and planning for future problems that have yet to occur.

For some reason this seems to be to encourage over designing. I've spent the last hour writing stuff like this.

"Connect to Server -- connecting to a server. PreCondition: No server connection exists. Postcondition--Connection now exists".

I would rather be coding than doing this nonsense. I realize this design work has its place but how much? I know this isn't bulletproof evidence for against designing alot in a tool like Enterprise Arch but here I go.

My professor that teaches these subject has designed the heck out of his project. Every thing that could possibly happen in the application has been documented. Instead of coding this himself, he's used this "immaculate documentation" to farm out the work overseas and to students during the summer.

The application that has resulted from all this design has been TERRIBLE. It's one of the worst applications I've ever seen and anyone could tell you its been overdesigned.

What does the SO experienced coder community have to say about this subject? Does designing a great deal before the project tend to make bad programs by forcing decisions made early on just because the "design docs say so"?

Thanks SO for any insight you guys might provide. If I knew this was all for a good reason, I would feel better "wasting" my time doing it. I'm more than willing to do some design work beforehand but I feel like my professor is expecting alot of engineering decisions to be made before any code is written.

EDIT: Interesting slashdot article on this subject. http://books.slashdot.org/story/09/11/16/1448204/Becoming-Agile

Answer 1

This is very subjective, but IMHO, only enough to make a reasonably good guess at what the appropriate overall system architecture should look like, but design should not stop, (or even slow appreciably) at that point, it should continue throughout development, and development refactoring should be encouraged and initiated often as continued design illuminates inconsistencies between the domain model analysis, the design, and the development model. The concepts (and the tools) your professor is teaching you about will be invaluable in the design process whether you do all your design before development, or as development progresses.

What your professsor is doing sounds like a classic case of the Waterfall Software Design Life Cycle (SDLC) process. Its major failing is not that construction phase (coding) is delayed, but that design more or less stops when coding begins (because of the impediments to change that the Waterfall methodology creates).

Some humorous takes on this can be found here.

Answer 2

In my experience this is not a good idea. It's much better to do some planning, but then write some code and iterate.

Answer 3

I think everyone is going to give you a different answer, partially because of personal opinion, and partially because they've got experience... My experience is this...

Often times, projects planned out to a 'T' are often really quite far away from original specifications... because sometimes you reach technical limitations, or because unforeseen conditions occur in which original designs just wont work.

I've worked on projects for different companies ranging from designing to a t, to a sort of "heres a general idea of what we want"... and I think what works best is some sort of happy medium between the two.

Working for a client, or building an interface another person is going to consume directly, obviously a thorough design is going to be key.

As far as UML diagrams, and real in depth planning like that?

I've worked for companies ranging from major wireless carriers to major residential and commercial real estate companies, to a pretty decently sized billing company... and I can honestly say I've not seen people ACTUALLY use UML diagrams with any sort of success.

I think its good to identify your touch points, and your work flows, but not something a piece of paper and pencil couldn't do for you.

As far as the terrible outsourced code he may have received... you get what you pay for. I dont mean to make any sort of generalizations etc, but I think even here in America you have probably 10:1 ratio for incompetent developers vs competent... and I think ratios go bigger from here. I wouldnt assume the reason for code problems was the planning

Answer 4

Formal academic trainings usually (as far as I know) place a lot of focus on design and bunch of other stuff before actual production coding start (not including prototyping).

It makes sense, you don't get it right (or almost right) to begin with, you will be screwed.

However in practice, few have the luxury of time and resource. Worse thing is, non-tech guys (especially those up there) want to see things (UI, prototype) to really feel secured that there are progress made.

To me, it's about balance, different organisation will have to draw their own line, come out with best practices. Also varies according to nature of projects/resources etc etc.

Regarding UML, here are some interesting posts on SO:
Is UML practical?
Is UML still seen as a viable way of documenting a software design?
Should I use UML when designing new code and algorithms?
If you don’t design in UML, then what do you design in?

Answer 5

In Top-down approach, popularly called the "waterfall model", things are not interesting with a long essential design phase.

If you have a prototype that works, then no matter, what the amount of time it takes, designing is better. So control your curiosity and complete the design.

Answer 6

In real life situation you probably doesn't have luxury to do elaborate design that will completely describe what you are going to build. Plus it may be counterproductive since that design will get old and broken on first day of coding (on obsolete on 2nd). Following Agile methodology is much safer with iterative building of prototypes and constant re-factoring and refinement. In this scenario you (or your team) can start coding almost immediately. There are some good books on Xp and Agile Methodology such as The Art of Agile Development that can give a better idea
Said that - not having design is not a good idea. You always need a plan. It's just with Agile plans get changed "as you go"

Answer 7

in real life, if they are paying you to design, then you design. if you are working on your own project and actually need to get it done, then you take a different iterative approach based on small successes and constant refactoring

Answer 8

There are those in the coding profession that believe design is everything, and there are those that believe it's better to code and address design issues as they arise. These are two opposite ends of the spectrum, and as is normally the case with two extremes, the answer lies somewhere in the middle.

Some amount of design based on some understanding of what the requirements are is unavoidable. Requirements tell you what to build; design tells you how to build it. In the real world (or at least in my experience), whether you do requirements/design in a formal fashion is largely based on the organization you work for, i.e., whether organization mandates that a certain software process be followed. But even if the organization doesn't mandate a formal process, the fact remains that you as the developer will do some measure of requirements/design even if it isn't formalized. At every step along the way, you are making decisions about how errors will be reported to the user, about what units of measure will be displayed, etc. All of these things are factors that would hammered out in a formal requirements/design process, but even in the absence of that, they are still decisions that must be made.

I've worked for organizations that are heavy on the process of software and those that aren't, and to be honest, I don't see much difference in the end product.

Personally, I don't think there's any substitute for quality people. That is, process is not a silver bullet. If you have marginal programmers, the best process in the world isn't going to save your project.

Answer 9

It is so last century, so academic.

What usually happens, at least in my experience is that no matter how hard you work on design you will miss something some technical limitation some link between different parts of the system. Also the starting point of the design - the requirements is always far from being set in stone.

More than that usually to validate your design you actually need to do some coding - prototypes and such. The newer approaches to development process - agile for one take it as far as saying that you only need as much design as necessary to start coding.

This is not to say that you can rush into coding without clear understanding of where you are going, but overdoing it is really bad

Answer 10

Generally you get a set of requirements and you start to design an architecture. Based on some cost estimation models you make a quotation, if you have the business (ok in your case you have it and is miserably paid, :-D) then you can go on with detailed design. The real coding can be done by code monkeys... Thinking and designing the system properly is the key to success: requirements met and costs within budget.

Designing is not in my opinion defining the pre and post conditions, that are details, but more defining packages and distributing/splitting the requirements on them in. Beside defining the big components in your system and how they are interacting. You have to think of loosely coupling them to improve testability (design for tests) among others: so the focus is clearly not on some pre and post conditions in real life large scale projects...

This is more a top-down approach I am focusing on here, which is often the way you have to go if you get a set of requirements.

Once the project runs, it is not done, some issues arise and some of them justify a redesign. The development process is almost never (excepted in simple cases) linear but an iterative process.

The architecture - if thought of well enough - will remain quite stable in general, but many iterations through the detailed design will be needed.

Answer 11

Depending on the project I either have two options I like.

For smaller projects which I want to finish quickly I like to follow the rules of http://www.extremeprogramming.org/ (agile programming) where you "spike" your ideas rather than completely working them out.

For more serious projects though I like to sketch a basic model (UML / database / sequence) before I start. This model is based on the requirements made up before. The model is mainly to give myself an idea on what classes need to be constructed and how they should work togerther. I never work out the methods though, that is for implementation time.

I have learnt that having a solid base model speeds up the project in various ways. For one you will create less useless code which have to be revised after. Second is, when working in groups it is easier to explain ideas and make sure everyone is thinking in the same direction and implementing in the same direction. Third thing is that when you take some time off or the project is larger it is nice to have an overview of what class did what :p.

Answer 12

It all depends on the balance between time and importance of the project. A high stake project will have high level stakeholders demanding a lot of design before coding can be done. Smaller projects can enjoy some freedom from over design and depending on developers they can be iterated very well between coding and design.

The key problem is not making the design first time, it is keeping the design document up to date with every decision change that happens during coding.

A divide and conquer approach can work pretty well if a big project can be broken to several smaller parts which are understood better than overall big picture. Those smaller tasks, coded well can act as building blocks or apis towards completing the bigger tasks. So basically in a top-down approach the division of a large task can be made, and then in a bottom-up fashion those small chunks can be completed and tested separately.

Answer 13

I don't want to come off as a broken record (I've been using this quote a lot lately), but this quote from John Gall is really relevant to a lot of aspects about software construction. (even though he was talking about biological systems)

wikipedia: Gall's Law

"A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system."

That is, if you're designing, design something small and achievable at first. A huge complicated up front design is destined for failure.

You can tackle huge tasks by breaking them down into smaller parts. Build the smallest possible thing you can think of that could possible work. Don't solve the whole problem at once.

One of the reasons that a big complicated design might be destined for failure is that the problem it is solving might be a wicked problem, or some other class of unsolvable problems where the subcomponents of the problem are each mutually exclusive. Those are really hard to detect up front without some kind of working experiments to act as evidence of the solvability of a subproblem.

Most of the evidence I've seen on the matter seem to predict that the bigger the upfront design, the more likely it will go over budget and probably fail to solve any problems. In short, design is neccessary, but your goal should be to keep it as small and simple as possible, and keep in mind that your first 20 drafts will probably fail due to factors you run into during the actual coding phase.

Answer 14

G'day,

It very much depends on the nature of the project.

I have worked on projects where the new system was a replacement for an existing system and the vast majority of time was spent gathering the behaviour of the existing system to establish requirements.

This phase, and then the design phase, took nearly eighteen months. The coding took about four months in calendar terms. In effort terms it was much more, but the net result was a stable system that the customer loved.

Oh, by the way, this was a replacement for an existing air traffic control system, a German air traffic control system, so for them to put it live after only one month of the three months scheduled evaluation period was quite an achievement that I'm incredibly proud of.

I think the large evaluation and design phase for this project was the best approach.

Edit: The system was in Karlsruhe at the en-route air traffic control centre run by the DFS, the German version of the FAA. The existing radar processing system was going to be replaced by Raytheon however delivery of the main system was delayed. So the DFS decided to upgrade the controller positions to those for the new system but attach them to the existing ATC processing system temporarily. Hence the name of the project Karlsruhe AdvancedDisplay System or KADS. A completely new wing was built on the back of the existing ATC centre as the whole control room was being replaced.

There was an enforced requirements gathering phase where the team was on-site working alongside the software engineers who'd built the existing system.

They documented:

• all messages between the existing radar processing system and the controller positions
• all behaviours of the existing controller positions, touch panels, keyboards, display characteristics, etc.

These requirements were then signed off by the DFS and the design phase started. This lasted about a month while several parts were prototyped and the best solutions identified. In parallel with the implementation was a test design phase where all requirements were traced through to the code and had an associated test.

Then coding and testing kicked off with several deliveries, approx. ten, done, each with an associated test and acceptance phase. A Scrum approach before it had a name in that we selected what chunks of work were to go into each "release" phase at the beginning of the phase. Final delivery came in on time and on budget.

The DFS intended to run the new system in parallel with the existing system for three months so that they could evaluate the performance and stability of the new system. Transferring completely over to the new system was an "all or nothing" proposal. They had to physically jackhammer the old radar lines out to remove the old telephone switch gear and there was no going back after that.

So for the German ATC service to be so happy with the system that they would do that was a big pat on the back for us!

BTW The number of the old software engineers who came up during the requirements phase and said that we weren't working because there was no code being written was quite funny. Definitely evidence of an old "seat of the pants" approach that was evident when you looked at some of the code. (-:

HTH

cheers,

Answer 15

Yes, that's one of the "traditional" methadologies

These days Agile iterative development processes have a lot more buzz and are "replacing" the older "waterfall model".

The contrast, we are told, is between the older, plan first, write requirements, write specs, document, then finally at the end .. code, model, with a "newer" one: Agile. In Agile, a system is incrementally developed in consultation with the users. One tries to always have a running system and unit tests. The direction can be changed before requirements are seriously misunderstood and wasted effort is made in the wrong direction.

I disagree with this explanation.

I don't disagree with Agile, just the idea that it is new. Rather, I would say that Agile is newly popular

I believe that many or most of the good, working, legacy systems were developed with incremental, evolutionary techniques, they just didn't have a prestigious buzzword back then. Certainly a team could have done all that planning and specifying, and then proceeded in Agile fashion, building a small "working" core and evolving it into the final version. (What a waste of time, though...) And many projects were developed outside of bureaucratic frameworks.

At some point, incremental prototyping became a recognized design methadology, and this was the early incarnation of Agile. I suppose with enough money and time, you could build something completely roof-down backwards, but I have to think that Agile-like patterns were used by the successful developers from day one.

The waterfall model, besides being no fun at all, has a tendency IMHO to work worse the more completely you apply it. Many projects following that model have gone on to spectacular failure. One specific problem is that it allows an organization to spend a huge amount of money with absolutely no indication of future success. The idea of running your first test after everything is done is ROTFL-time today, but think of how much a big waterfall project could cost before you even ran the first real test.

Having said all that, it won't hurt you to attempt it, and some of those documentation techniques have value outside of the waterfall model.

Answer 16

Ask your professor if they've ever had a non-academic job. Then ask them to describe the largest system they've ever written; what the largest team that they've ever been part of; whether they've ever invoiced or shipped or released a system.

If they've never had a non-academic job, or never written a large system, or invoiced or shipped or released, then realize that you're going to have to endure something awful for the sake of getting a grade and do it as gracefully as you can. Maybe all you'll learn is that you hate UML and how to get through something painful with a smile.

Answer 17

IMHO waterfall is becoming less relevant and Agile more so because of the evolution of modern programming languages. Previously, it would take forever to code even the most simple thing (check out the syntax of COBOL if you want to see how not to be lean and expressive), so it made a lot of sense to write up documents explaining what you wanted the program to do, and then go implement it. Nowadays the doc process is a lot shorter because the easiest and most natural way for a programmer to express his design is (shocker!) in the code itself.

I see this myself all the time -- I sit down with an idea in my head but not a clear view of how I want to implement it, so I sketch out potential implementations in the code itself. That is so much faster and easier than creating UML diagrams to demonstrate the architecture, and then having to do the implementation after that. Modern OO languages (and the frameworks available that sit on top) are so dynamic and expressive that we can mostly skip the middle step.

That said, it's still important to learn all that icky design stuff. After you do it the long and slow way a few times, it gets more natural and automatic to think in terms of designs in your head, so you're still going through a real design process although you may not do much of it in writing. Therefore, even though I cringed when you described months of UML diagramming, stick it and out pray that you never get a job where that's still being done.

Good luck!

Answer 18

I'm just in the middle about learning how to best implement coding projects, planning/coding to iterate or not to iterate. How long to spend on the plan etc..

I found a fantastic article online by a guy called. you can find it here. He goes through the reasoning and thought process behind a functional project plan. The functional project plan concentrates on the user's experience and interaction with the code. And getting this down first is going to be the most important part of the design process. once you have gone through this one you can do a technical project plan which will tackle the specifics of how to implement each problem the user will face when using the code. I think that OP is right when he says one of the major benefits of project plans is not in having them to code from. but in having done them once. so that when you go back to write the code you have a reference to the work you have already done. So you only have to concentrate on the work of coding.

This may all sound like common sense to you, but... Bottom line: write you'r project plan with the intention and idea that you are already doing some/most of the work. Half the work is knowing what to code.

Answer 19

It depends on the particular project and the particular customer. Some people need to be able to visualise a system to be able to understand it and what they want from it. Other times, you may be designing a system where the customer presents you with an inch thing requirements specification, for a system where the quality processes must be higher. Also, some customer require formal agreement to a list of requirements.

In the cases where the customer is unclear on requirements, an agile methodology is more suited. This way the developer can quickly implement new features and get feedback from the customer, reducing the probability of deliverying something not wanted. This contrasts to trying to establish a list of set requirements and only building what has been agreed on contractually, only to deliver something that isnt' accepted by the customer.

My company has recently hired two new developers both of whom have had many years in the software development industry, but never worked at a company where requirements where written, or design documents produced, or a set SDLC of any kind followed; the typical shop where the developer is given a piece of paper with some requirements and told to "build this". Once both were forced to sit down and iteratively complete design documents (more for their own learning than anything else) they both came to me saying how much more beneficial an upfront design has been in realising thing they would never have thought about until the application was well and truely implemented (badly). Shortly after, they started writting Unit Tests upfront (before implementation) and while for the first few weeks, found it very strange and saw it as a waste of time, each day they come to me and espouse how much more they'd learnt and beneficial unit tests are.

Rushing straight in to coding the actual implementation (as opposed to tests) is a rookie mistake that we've probably all made.

Answer 20

I think you want us to tell you that your professor is wrong. Here's some ammunition: http://www.developerdotstar.com/mag/articles/reeves%5Fdesign%5Fmain.html I have yet to discover if reeves is 100% correct but so far it is.

Answer 21

Up-front designs, e.g. UML models, are hypotheses that get tested and usually proven wrong during coding. In short, coding is not analogous to "constrction"; rather coding is designing.

Use a brief up-front architectural (strategic) design process to identify, isolate, and mitigate risks. But use disciplined refactoring and occasional annealing to evolve the system's design, bottom-up.

Answer 22

I am not surprised by many of the comments for the question.

The whole purpose in acamedic life is you learn the "thought process" at school which is very important in real life.

I have seen many kids just want to jump start coding and not write spec. Ability to write a clear spec brings you long way. It brings clarity on what has been handled and what not.

Because of all the modern IDE/Debug processes things have become more trial-error-google-trial-now it may work!-finished?

While it may be gruesome at the beginning in the end having developed the system you understand much better where your spec helped and where your coding helped and then in real life you know how to make it better.

After all you are still in school!

Good luck.

Answer 23

Probably the most relevant thing to your situation: consider that some large projects are difficult to start small. You should always start as small as possible, but often that razor won't cut fine enough; awful though it may be, your project is presumably intended to walk the class through the basic steps.

Design time spent before coding always has great potential for being well-spent -- but only if you know how to spend it well! As you have noticed, it is very easy to overengineer, make bad design systems early, and so forth. Just chalk your current project up to experience -- when you do a real-world project, you'll know what design pitfalls to avoid!

Prototyping also has great potential for being time well-spent -- but only if you have the balls to throw out the prototype and begin again from scratch, having learned from the experience. If you've ever had to wrangle sendmail, you know the pain of dealing with software that grew from a small project to a horrific monster without starting over...

Finally, note that there are a wide range of methodologies out there; for better or for worse, as a software guy, you may need to deal with any of them. Most of them can be made to work in some reasonably useful way, so even if a particular instance is dysfunctional, you should not let that poison you agains methodology in general.

Answer 24

You don't need to do any design work, unless you are hoping to produce software products that will be used and enjoyed by people other than yourself.

It's very difficult (if not impossible) to create a decent software product without understanding the goals of your potential users. You can't understand the goals of your users without research, and you can't apply research without design.

Answer 25

I'd mostly agree with the opinions promoting Agile approach to the software development. I wouldn't care to design the most complex solution with all the cases that you can imagine from the start. There is a well known saying from Donald Knuth "Premature optimization is the root of all evil", so take it seriously into account :) One thing that is most important for the software developer when starting to work on something is to exactly define what is to be built. So, before all designing, weather starting simple or going complex, it is most likely that you will get where you didn't want if you do not know what you want to build at first. It is particularly important to define the constraints of the solution.

Your question is kind of phylosofical and raises lot of other questions in the developers heads. Some of others already pointed you to looking at Agile approach to development, which on its specific way tackles these questions. So, I suggest readings on the Agile topics as well.

Cheers,
Ivan

Answer 26

Academic teachers have never done real world projects :). They're responsible for this waterfall spirit. Also UML Tools are not convenient today though Visual Studio 2010 Ultimate is coming close.

Answer 27

Plans are nothing; planning is everything.

Working on UML diagrams and other tedium can force you to confront the entire problem you're trying to solve, and to break it down into manageable chunks.

Should you follow your class design to the letter? Almost certainly not; I've never worked on a project where the requirements didn't change midway due to some unforeseen circumstance. But at least you're going into it with some idea of what needs to be done. It sounds to me like you followed your docs too rigidly, which can cause problems.