There is no secret. It's sweat, not magic.
It's not about doing one thing right. It's balancing many things that must not go wrong. Sometimes, they work in sync, sometimes, they work against each other. Design is only one group of these aspects. The best design doesn't help if the project fails (e.g. because it never ships).
The first rule I'd put forward is:
1. There are no absolutes
Follows directly from the "many things to balance. D.R.Y., Y.A.G.N.I. etc. are guidelines, strictly following them cannot guarantee good design, if followed by the letter they may make your project fail.
Example: D.R.Y. One of the most fundamental principles, yet studies show that complexity of small code snippets increases by a factor of 3 or more when they get isolated, due to pre/post condition checking, error handling, and generalization to multiple related cases. So the principle needs to be weakened to "D.R.Y. (at least, not to much)" - when to and when not is the hard part.
The second rule is not a very common one:
2. An interface must be simpler than the implementation
Sounds to trivial to be catchy. Yet, there's much to say about it:
The premise of OO was to manage program sizes that could not be managed with structured programming anymore. The primary mechanism is to encapsulate complexity: we can hide complexity behind a simpler interface, and then forget about that complexity.
Interface complexity involves the documentation, error handling specifications, performance guarantees (or their absence), etc. This means that e.g. reducing the interface declaration by introducing special cases isn't a reduction in complexity - just a shuffle.
3-N Here's where I put most of the other mentions, that have been explained already very well.
Separation of Concerns, K.I.S.S, SOLID principle, D.R.Y., roughly in that order.
How to build software according to these guidelines?
Above guidelines help evaluating a piece of code. Unfortunately, there's no recipe how to get there. "Experienced" means that you have a good feel for how to structure your software, and some decisions just feel bad. Maybe all the principles are just rationnalizaitons after the fact.
The general path is to break down a system into responsibilities, until the individual pieces are managable.
There are formal processes for that, but these just work around the fact that what makes a good, isolated component is a subjective decision. But in the end, that's what we get paid for.
If you have a rough idea of the whole system, it isn't wrong to start with one of these pieces as a seed, and grow them into a "core". Top-down and bottom-up aren't antipodes.
Practice, practice, practice. Build a small program, make it run, change requirements, get it to run again. The "changing requirements" part you don't need to train a lot, we have customers for that.
Post-Project reviews - try to get used to them even for your personal projects. After it's sealed, done, evaluate what was good, what was bad. Consider the source code was thrown away - i.e. don't see that sessison as "what should be fixed?"
Conway's Law says that "A system reflects the structure of the organizaiton that built it." That applies to most complex software I've seen, and formal studies seem to confirm that. We can derive a few bits of information from that:
- If structure is important, so are the people you work with.
- Or Maybe structure isn't that important. There is not one right structure (just many wrong ones to avoid)