Getting up to speed on modern architecture

Question

Hi,

I don't have any formal qualifications in computer science, rather I taught myself classic ASP back in the days of the dotcom boom and managed to get myself a job and my career developed from there. I was a confident and, I think, pretty good programmer in ASP 3 but as others have observed one of the problems with classic ASP was that it did a very good job of hiding the nitty-gritty of http so you could become quite competent as a programmer on the basis of relatively poor understanding of the technology you were working with.

When I changed on to .NET at first I treated it like classic ASP, developing stand-alone applications as individual websites simply because I didn't know any better at the time. I moved jobs at this point and spent the next several years working on a single site whose architecture relied heavily on custom objects: in other words I gained a lot of experience working with .NET as a middle-tier development tool using a quite old-fashioned approach to OO design along the lines of the classic "car" class example that's so often used to teach OO. Breaking down programs into blocks of functionality and basing your classes and methods around that. Although we worked under an Agile approach to manage the work the whole setup was classic client/server stuff. That suited me and I gradually got to grips with .NET and started using it far more in the manner that it should be, and I began to see the power inherent in the technology and precisely why it was so much better than good old ASP 3.

In my latest job I have found myself suddenly dropped in at the deep end with two quite young, skilled and very cutting-edge programmers. They've built a site architecture which is modelling along a lot of stuff which is new to me and which, in truth I'm having a lot of trouble understanding. The application is built on a cloud computing model with multi-tenancy and the architecture is all loosely coupled using a lot of interfaces, factories and the like. They use nHibernate a lot too. Shortly after I joined, both these guys left and I'm now supposedly the senior developer on a system whose technology and architecture I don't really understand and I have no-one to ask questions of.

Except you, the internet.

Frankly I feel like I've been pitched in at the deep end and I'm sinking. I'm not sure if this is because I lack the educational background to understand this stuff, if I'm simply not mathematically minded enough for modern computing (my maths was never great - my approach to design is often to simply debug until it works, then refactor until it looks neat), or whether I've simply been presented with too much of too radical a nature at once. But the only way to find out which it is is to try and learn it.

So can anyone suggest some good places to start? Good books, tutorials or blogs? I've found a lot of internet material simply presupposes a level of understanding that I just don't have.

Your advice is much appreciated. Help a middle-aged, stuck in the mud developer get enthusastic again!

Please!

Answer 1

Patterns of Enterprise Application Architecture is an excellent book, Martin is a legend.

Answer 2

If it is about architecture I always look at Pattern & Practices if it is about the Microsoft development stack.

They have a lot of good whitepapers/books about architecture for all kinds of application types.

Answer 3

In addition to many great resources already listed in the answers and probably to be listed, a slightly different advice - use StackOverflow.

You seem to be able to write very well-thought-out, readable and dare I say it good questions.

So, whenever a certain specific architectural choice/pattern or a piece of code makes no sense, feel free to turn it into an SO question (obviously ideally after reasearching a bit yourself).

Also, regarding your point re: math:

As far as software engineering is concerned, the only math you truly need most of the time are:

• "discret math" - sets, graphs, trees etc... and their practical applications to data structures and algorithms

• A limited algebra skillset involved in analyzing O(n) complexity for the latter

• Ideally, an intuitive grasp of statistics/probability - as a generic software engineer you don't always have to be able to do advanced calculations, but a feel for "this is more likely to impact my situation based on its likelyhood of happening times its impact size" is often a good guide in design choices.

However, ONE thing that is invaluable in a good software engineer that is often what distinguishes "good at maths" person from "not good at maths" - while not strictly speaking math related - is ability to see patterns.

If the reason you're "not good at maths" is that you lack that pattern-observance ability, then you will be at a great disadvantage. That is one skill/ability that you must train in yourself above all others to succeed in architecting systems, IMHO.

Answer 4

Here are some suggestions, by no means a complete how-to or all-in-one answer;

• Make time in your working day to learn about the new stuff. Make this time in your working day because it is part of your work. It's not something you should do only in your evenings or at weekends. If you want to spend some of your own time learning it, go right ahead, but don't fail to recognise that learning is part of your job and don't get into the mindset that you have to hide your ignorance at home or that not knowing everything is fatal to your career. It's up to you to balance learning time and working time.

• Get some large sheets of paper and a box of coloured pencils (or MS Visio if you prefer). Start drawing two types of diagrams:

  • Mind maps, to diagram your understanding of the new technologies. If you don't know what mind maps are, hit Wikipedia to start with.
  • Architectural diagrams of the system you are responsible for. Whether you use UML or another widely-used format or one of your own devising is up to you.

Answer 5

I have seen cases like yours in the past as well. But with dedicated daily reading and parctise, they seem to be doing as well as any other "young and so called good programmers". If I were in your place, this is what I would do:

• Break down my trouble areas into smaller pieces.
• Will search for books/articles availablke on the subjects in the the aforementioned list.
• Will code samples by myself and experiment with them in order to gain a greater understanding.

Having said that, dont be worried abt not having a mathmatical bent of mind. I am a maths student myself and honestly, there arent many occassions when I have to use those skills. 'Most' enterprise apps dont require that kind of mathmatics skills. Beside, modenr day apis are so advanced and easy to use that you would never have to trouble yourself with writing an efficient sorting algo or anything of that sort.

Try to read as much as possible on Design Patterns. "Head first design patterns" is an excellent book to start with but the code samples are in Java(Which shouldnt reall y matter). "UML Distilled" is another good book. Thre are many other available, just google :). Also, go through your existing system in a thorough way.

all the best..

Answer 6

Frankly I feel like I've been pitched in at the deep end and I'm sinking. I'm not sure if this is because I lack the educational background to understand this stuff, if I'm simply not mathematically minded enough for modern computing (my maths was never great - my approach to design is often to simply debug until it works, then refactor until it looks neat), or whether I've simply been presented with too much of too radical a nature at once. But the only way to find out which it is is to try and learn it.

For me it's because of your lack of educational background. People learning themselves or at work often don't have the necessary background to really understand what is behind the scene of a framework. Some concepts are general to all informatic systems and we learn them at school, thus we can understand how these concept works in every languages... You seem more like a pragmatic (and efficient?) programmer, but you will make mistakes is you don't have that general background (but you should know you aren't alone in this case... particulary php dev i think don't have either this background).

How to understand relation of code with DB, ORM like Hibernate, if you don't exactly know what is a transaction (i guess you know how to use it... (pragmatic i said!) but you prolly never heard of concepts like ACID: http://en.wikipedia.org/wiki/ACID , i guess you don't know a lot about transaction isolation ).

How to use soa architecture with webservices, rpc, rmi, corba... efficiently and be able to have consistent data in your db if you don't know some concepts like the 2 phaze commit http://en.wikipedia.org/wiki/Two-phase_commit_protocol

How to write good code if you don't know many design patterns, best practices, and you don't know when to use them?

We could say many things.

The fact is just you miss a part of informatic general knowledge that engineer students all (normally?) have. When we are start working after an engineer school we are not expert in these things, but the only difference with you is that we know very important concept exist and when they should be used. Thus we just have to read about that and apply what we learn from the web or our team (because at school we barely see the basics, but on many techs).

I really think nothing is impossible for you, you just should real a lot of things about general concepts that you don't know and you heard a lot of. Without this knowledge you'll just tend to reinvent the wheel and with your pragmatism find alternative solutions that certainly works but are just a little bit dirty... For exemple your dev could work but when you introduce high load on your system, you may have concurrency problems, performances problems. This lack of knowledge is not a matter when you'r on a simple context but is really needed when you have lot of responsabilities on a complex system...

Answer 7

Sitting on the Beach - Preparation

Make a list of everything you don't understand. At the final stage, this list shall be your checklist. Clear your mind – get yourself to a fresh start, "forget" all the confusing details you already know about your architecture. Dig up every document created by the original architects. Get the documentations for every technology used in your project. Make coffee.

Floating - Managing Complexity

To float, you need to manage complexity. If you don't deal with complexity properly, you'll dive into details when there's absolutely no need, and you won't know how and where to stop, sinking right into the bottom and drowning.

"My approach to design is often to simply debug until it works, then refactor until it looks neat"

I think I used to be just like you. I developed solutions by starting from scratch and adding one piece at a time, eventually containing a complex structure inside my head. I didn't plan the architecture, I didn't separate design and implementation, I just coded, debugged, refactored. It worked: since complexity grew slowly, I had no trouble understanding the architecture that emerged.

This approach simply isn't good enough when "inheriting" a complex architecture planned by others; you cannot swallow the entire structure at once - because there are too many details, and you cannot randomly swallow little bits - because you won't understand how they all relate to each other and you'll never get to see the big picture.

Software is a puzzle of complexity management. There are very big pieces, sometimes referred to as "subsystems", which compose the big picture. Each of these is composed of smaller pieces, which in turn are also composed of smaller pieces. When you look at the code, all you see is the tiniest pieces. So forget about the code itself for now, at least until you see all the bigger pieces.

Swimming – Mapping the Architecture

The first step towards floating is seeing the big pieces. To do that, you need maps. The map to the biggest pieces is the highest-level architecture. If the original architects left you with no such map, you'll have to create it by yourself. Just like it is impossible to map a region from inside a valley, you cannot map your architecture from low-level details. You need to stand at the top of a mountain to get a 360-degrees-view of all the valleys, hills and paths. You need to map your architecture from the top.

After you have this top-level map, you should get maps for the parts composing it - just like you create an un-detailed map of an entire region, and then create separate detailed maps of sub-regions. A map should describe the different subsystems. At the very least, it should describe the responsibilities of each subsystem, its external interface, and how it interacts with other subsystems.

Diving – Managing the Details

There is this principle in diving which says you shouldn't move between depths too fast, because of the changes in pressure. This principle holds. When you move from dealing with one subsystem into dealing with one of its internal subsystems, make sure you only dive into the next level of complexity/abstraction. Let your mind handle one tier at a time.

Separate concepts, patterns, interfaces and implementations. nHibernate is an Object-Relational Mapping (ORM) solution. Thus, before you deal with the details of nHibernate itself, you need to make sure you understand the general concept of ORMs and their place in the world. Factory is a design-pattern, so before you deal with Factory you should understand what design-patterns are and what their role is.

Technologies raise and fall, but concepts remain. Once you get the concepts, it really doesn't matter much – on an architectural level – how these concepts are manifested.

The fact that your architecture is loosely-coupled is actually a good thing, because it means you can understand the role of one subsystem without the need to know much about other subsystems. The fact your architecture makes use of interfaces is also good – it means you can learn how elements interact with each other without learning how they work internally.

Waterskiing – Acquiring Distilled Knowledge

There is one book that I think is a "must-read": Code Complete by Steve McConnell. It changed my professional life.

---

I hope that this post managed to help you in some way and isn't a complete waste of your time.