ASP.Net -- How to effectively use design patterns without over-engineering!

Question

I'd appreciate people's thoughts on a dilemma I've been struggling with ever since ASP.Net came out.

In classic ASP, the layers of code were minimal. The ASP page contained HTML and script combined. The COM components contained business logic and DAO infrastrcuture. The asp pages themselves were messy, but everything was in one place.

ASP.Net code-behind neatens up the code, that's fine. The controls allow us to be more OO on the presentation layer. These things are nice.

Here is my problem--many projects I've walked into are enterprise web apps, but not all that complex, say 10 or so web pages / user interfaces, lots of DB interaction, etc. These used to be a piece of cake to understand. Now I will often encounter 5 to 10 layers of code to create a fairly simple web page. These may include ASP, code-behind, control classes, DTO classes, ORM objects, and then a few others just thrown in for the hell of it.

In addition to the 5-10 layers to get to the DB, there are numerous custom objects created just for storing ordinary data, rather than using POCO (plain old CLD objects) like for example a collection. To understand these objects, one often has to trace back through an inheritance hiearchy including 3 or more levels of objects and interfaces.

Here is the crux: Previously, I looked at 1 asp page and say 1 or 2 smallish objects, a few SQL queries, these got the job done and were fairly simple to maintain and understand.

Now, for the same page, there may be literally 50 objects or more, spread throught hundreds of objects in the custom namespace.

I ask you, fellow code-artisans, is this progress? Are some people going a little overboard with their zany fun new design pattern toys? Is there a happy medium? Is there a way to effectively use design patterns without creating so many objects that it becomes worse spaghetti code than the old procedural paradign???

Please share your thoughts.

Answer 1

Setting aside the possibility that these levels of abstraction are required by the organization (unlikely, but it does happen), it is very common for developers - especially in non-agile, corporate/enterprise environments - to add too many layers of abstraction. It happened with classic ASP, .NET just makes it a lot easier. Most of us in this profession have a natural tendency to overcomplicate things, and it takes discipline and an active mind to overcome that.

Also, many mediocre developers wrongly believe that maximizing layers of abstraction and pattern use makes them better developers.

Finally, many mediocre developers have been taught that they should use layers, abstractions and patterns, but have not been taught well enough to know how, when or why. So they go into a project thinking "well, I know I should have some layers here..." and you can imagine what comes out.

Ultimately, the "best practice" of best practices is to code as simply as possible UNTIL you run into a real problem that is preventing you from moving forward. Many times, you should have a pattern or best practice in your toolbox that is right for the task, just like you have the right wrench for a nut. Best practices and patterns are tools - you don't get out a hammer and start swinging until you have a nail that needs to be pounded. Likewise with software development.

Answer 2

I've worked on projects that are badly done in the opposite direction (200+ MB source trees with no design patterns). Believe me, that's not the right solution either. The code gets unmaintainable, there's old code laying around that developers trip over, etc.

On the other hand, to answer your questions about design patterns:

No this is not progress. Yes, some people are going overboard. There probably is a happy medium, but it requires the design pattern wonks of the world to calm down a little bit, I think.

I try to make my solutions as simple as possible. This may involve a design pattern... or it may not; it really doesn't matter, as long as it works, and it's easy to maintain. The important thing is... design patterns are not a panacea for bad code. Just because one uses design patterns doesn't mean one's code is well-written. Unfortunately, too many developers are taught (or believe on their own) exactly that: that if they use design patterns, their code quality is higher; that's not necessarily true (not even true on a regular basis, I think).

As an aside: If you want an excellent example of overdesign, look at the source code for the CruiseControl.NET tray application. It's an excellent demonstration of design patterns run amok (honestly, Thoughtworks as a whole is an excellent demonstration of that).

Answer 3

I think it would be difficult to use the design patterns I am familiar with in Classic ASP, so in that regard we are ahead of the game.

Though I haven't used the MVC implementation in ASP.NET, it is my understanding that it addresses many of your concerns in that it is much more direct and simple.

Answer 4

The purpose of all of the tools and techniques you name is to manage complexity in a world of rapidly changing business requirements. Does "complexity and rapidly changing business requirements" characterize the world where your apps reside?

Answer 5

This is and isn't progress. How well the new tools we have are used may be the question here as some may over build and others may under build, e.g. everything is in a code behind from markup to database logic to business logic just to keep it all together.

Design patterns can be used in some places as a nice way to solve a problem in some cases. For example, in a previous job there were these devices that had some common commands that had to be supported and so I built an adapter class and a class for each type of device that worked nicely to solve the problem. Another example of a design pattern helping is to have a facade pattern hide some details of how an outside system functions so that there is a common internal list of functions to support.

Answer 6

Some people are architecture astronauts, and they will insist that these layers are all necessary and the design is absolute garbage unless you include them.

On the other end of the spectrum are people who push this all aside and just have the code all munged together: it's simple, right, so why not just have the SQL query right there in the code-behind populating to a listbox control?

You've identified the problem with the astronaut version already, it's just so overly complex to do simple tasks.

The problem with the second method is that while it works fine for small, tiny apps, those same apps have a tendency to grow into large, big apps, and then it all breaks down. You spend tons of time fixing the same bugs over and over, because that same query or piece of code has been copied/pasted as needed, and slightly modified, and spread everywhere.

There is, of course, a middle ground. Where exactly that lies is going to be different depending on who you ask.. but it's there.

Building a sane business layer is a good step to getting there. This should be a layer that presents the simplest view of the objects in your system, and contains no public-facing database-related stuff (aside from maybe a Connection object that tells the business layer how to get to the database). This consolidates all your logic into one place, so the ASP part of the project will just be wiring up the abstract "User.LoadAll()" method to the table that displays a list of users. The ASP app should have no clue if it's reading from a database, web service, or just a bunch of made up stuff.. it simply talks to the business layer.

Under the business layer, you can have queries directly, you can use ORM, you can build a deeper data access layer that does these things .. the nice thing is this decision is entirely contained by the business layer (and you can change it later, if you need to, without affecting the public API of the business layer).

Now any queries dealing with users are contained within the User object.. and if you have a bug with the way you load permissions, it can be fixed in one place.

I just did a migration like I described above (ASP with SQL in code-behind, to a distinct presentation - business - datasource (ORM) layer), and it is great. Though it does add a few layers to getting at actual information, once the business methods work, the rest of the code works. Fixing a bug is simple, and fixes it everywhere. Near the end, it quickly became the case that when you needed to get a list of reports that a user could run, someone had already written a business method for that (User.GetReports()), where before, you'd probably never find it if it had been done - if you were lucky, it was written as a function and not just inline code.

Answer 7

This is an example of a higher-level general principle, I think.

It's like the scissors we used to use in grade school, with the rounded tips and dull edges. You couldn't cut very well, but you couldn't your yourelf (or anyone else) very easily either.

Anything powerful creates the potential for greater good and greater ... otherwise.

Microsoft, much to their credit, does a really good job of providing tools with a low barrier to entry, so any-old-programmer can get something useful done relatively quickly. But because of their simplicity, sometimes they don't scale too well, and you need to move up to a more sophisticated toolset - but it pays to learn how to use new stuff properly and effectively. So your question is a good one, because that's part of the "understand it first" process.

Answer 8

For me the problem stemmed from following too many articles where the author didn't take the time to explain the context in which the solution was arrived at.

A guy writing business apps for a medium sized business who's locked into certain technologies is going to build apps differently than a guy working for a vendor who's selling it's software to other companies.

So for the inexperienced developer who wants to do the "right thing", I think over-engineering is an easy trap to fall into.

Then there's other guys who just want every project to be "perfect". I don't know what to say about those guys. I've never had enough time or budget to make any project "perfect".

Answer 9

Thanks for all the comments so far. To clarify a bit more, in classic ASP we were using VB components, which naturally limited our ability to overcomplicate them, which was good in the sense that it kept it simpler.

I liked Rex's reply "Ultimately, the "best practice" of best practices is to code as simply as possible UNTIL you run into a real problem that is preventing you from moving forward."

This sounds similar to another I heard which was “Premature optimization is the root of all evil.”

Following this, what if we just put all the bus logic in the code-behind and then moved it into chared libraries as we discovered which code would really be used by multiple pages? For example, if it's one page, you could have everything in the code-behind. If there are two pages that share one piece of logic, make a component. If two components use SQL Server, create one new component to streamline the SQL queries, etc.

I also liked Dana's distinction about building custom apps vs a single application. It's one thing to have a full-fledged API if 1000 others will use it. It could be a waste of time to put the same energy into a one off app.

"architecture astronauts" is a great term for the over-engineering tendency!

Many people express there is a middle-ground, and I think (hope) that's what we're all looking for! i.e. C# gives us the ability to do rocket-science, however maybe that level is only needed 5% of the time...I think exacerbating the problem is the fact that most interviewers these days like you to be very conevrsant in all the rocket science...

One other related question I'd like to bring up is: How about just using POCO (.Net provided) objects to store data in the ASP.Net architecture--I've seen some architectures create a new object for every single type of data interexchange...wouldn't it be simpler to just create a collection object (say list) and call it "this data", "thatdata", etc.

Answer 10

In my experience you'll be much more agile by not using custom database objects at all. In older .NET versions, you can pass around DataTable objects. In .NET 3.5, you get strong types for free using LINQ.

It's amazing how some developers spend 75% of their time retrieving data from a database. Which .NET does perfectly fine without any custom code. Use the standard library, and you'll find you have much more time to address business problems.