Whether we like it or not, many if not most of us developers either regularly work with databases or may have to work with one someday. And considering the amount of misuse and abuse in the wild, and the volume of database-related questions that come up every day, it's fair to say that there are certain concepts that developers should know - even if they don't design or work with databases today. So:

What are the important concepts that developers and other software professionals ought to know about databases?

Guidelines for Responses:

Keep your list short.
One concept per answer is best.

Be specific.
"Data modelling" may be an important skill, but what does that mean precisely?

Explain your rationale.
Why is your concept important? Don't just say "use indexes." Don't fall into "best practices." Convince your audience to go learn more.

Upvote answers you agree with.
Read other people's answers first. One high-ranked answer is a more effective statement than two low-ranked ones. If you have more to add, either add a comment or reference the original.

Don't downvote something just because it doesn't apply to you personally.
We all work in different domains. The objective here is to provide direction for database novices to gain a well-founded, well-rounded understanding of database design and database-driven development, not to compete for the title of most-important.

+3  A: 

Every developer should know that this is false: "Profiling a database operation is completely different from profiling code."

There is a clear Big-O in the traditional sense. When you do an EXPLAIN PLAN (or the equivalent) you're seeing the algorithm. Some algorithms involve nested loops and are O( n ^ 2 ). Other algorithms involve B-tree lookups and are O( n log n ).

This is very, very serious. It's central to understanding why indexes matter. It's central to understanding the speed-normalization-denormalization tradeoffs. It's central to understanding why a data warehouse uses a star-schema which is not normalized for transactional updates.

If you're unclear on the algorithm being used do the following. Stop. Explain the Query Execution plan. Adjust indexes accordingly.

Also, the corollary: More Indexes are Not Better.

Sometimes an index focused on one operation will slow other operations down. Depending on the ratio of the two operations, adding an index may have good effects, no overall impact, or be detrimental to overall performance.

I had a feeling that would be taken the wrong way. What I meant by "traditional" was that you don't really have any control over the algorithms, only the ability to influence which ones are used. Anyway, I removed that language as I don't want anything overly controversial in the main post.
@Aaron: You *do* have control over the algorithms. That's what indexes are for.
Hmm, so you can change which type of sorting algorithm is used by the DE? What data structures are used for the index? I'd prefer not to argue over this point, that's why I took it out, but I stand by the basic idea that you have a lot less control when working with database as compared to code.
@Aaron: Less control does not remove the obligation to actually understand if the query is **O**( *n* ^ 2 ) or **O**( *n* log *n* ) or just **O**( n ). Less control does not remove the obligation to actually understand what's going on and to find out how to control it.
@S.Lott: I think we are on the same side here, as I was suggesting a *greater* profiling burden for databases - "You **need** to know ... [how to] read a query plan". But my edit seems to have been rolled back, so... I guess it belongs to the community now.
+30  A: 

Good question. The following are some thoughts in no particular order:

  1. Normalization, to at least the second normal form, is essential.

  2. Referential integrity is also essential, with proper cascading delete and update considerations.

  3. Good and proper use of check constraints. Let the database do as much work as possible.

  4. Don't scatter business logic in both the database and middle tier code. Pick one or the other, preferrably in middle tier code.

  5. Decide on a consistent approach for primary keys and clustered keys.

  6. Don't over index. Choose your indexes wisely.

  7. Consistent table and column naming. Pick a standard and stick to it.

  8. Limit the number of columns in the database that will accept null values.

  9. Don't get carried away with triggers. They have their use but can complicate things in a hurry.

  10. Be careful with UDF's. They are great but can cause performance problems when you're not aware how often they might get called in a query.

  11. Get Celko's book on database design. The man is arrogant but knows his stuff.

Randy Minder
But do not read the section on heiarchical tables! It will make your brain freeze.
Charles Bretana
care to elaborate on item 4. This a topic that has always intrigued me.
I don't agree with #3. Error checking should be in the program, not the database, because that's where it's best possible to handle errors. If the database finds an error, it's likely to spit out something incomprehensible to the end user.
David Thornley
I am seeing a lot of 'designs' where 2) is simply left out. The argument is usually to 'reduce complexity'. Doesn't this just hurt?
@David: I've always preferred to put it in both places. That way you're protected against bugs as well as user error. There's no reason to make every column nullable, or to allow values outside the range 1-12 to be inserted into a `Month` column. Complex business rules are, of course, another story.
@Brad - Most of our applications at work were done well before solid programming processes were put into place. Therefore, we've got business logic scattered everywhere. Some of it's in the UI, some in the middle tier and some in the database. It's a mess. IMO, business logic belongs in the middle tier.
Randy Minder
@David - If it's an absolute certainty that database modifications will only occur in applications then you might be right. However, this is probably pretty rare. Since users will likely enter data directly into the database, it's good practice to put validation in the database as well. Besides, some types of validation are simply more efficiently done in the database.
Randy Minder
@Zedoo - Sad indeed. Anyone who would make that statement should stay well away from any database of any kind.
Randy Minder
Much too technical and specialized. Unless you work with databases you don't need to know this, it suffices to read up on it when it's time.
Point #8 is indeed important. How to get the column types right in general, is a very important thing to know.
Christian Vest Hansen
Some of these bullet points are cryptic. Explain normalization, referential integrity, UDF, etc, or at least provide a reference. This answer seems to do everything the questioner specifically warned against.
@flies - If you can't find a reference to normalization or referential integrity, using Google, then even the concept of a database is going to be WAY over your head.
Randy Minder
+3  A: 

I would like everyone, both DBAs and developer/designer/architects, to better understand how to properly model a business domain, and how to map/translate that business domain model into both a normalized database logical model, an optimized physical model, and an appropriate object oriented class model, each one of which is (can be) different, for various reasons, and understand when, why, and how they are (or should be) different from one another.

Charles Bretana
+3  A: 

I would say strong basic SQL skills. I've seen a lot of developers so far who know a little about databases but are always asking for tips about how to formulate a quite simple query. Queries are not always that easy and simple. You do have to use multiple joins (inner, left, etc.) when querying a well normalized database.


For a middle-of-the-road professional developer who uses databases a lot (writing/maintaining queries daily or almost daily), I think the expectation should be the same as any other field: You wrote one in college.

Every C++ geek wrote a string class in college. Every graphics geek wrote a raytracer in college. Every web geek wrote interactive websites (usually before we had "web frameworks") in college. Every hardware nerd (and even software nerds) built a CPU in college. Every physician dissected an entire cadaver in college, even if she's only going to take my blood pressure and tell me my cholesterol is too high today. Why would databases be any different?

Unfortunately, they do seem different, today, for some reason. People want .NET programmers to know how strings work in C, but the internals of your RDBMS shouldn't concern you too much.

It's virtually impossible to get the same level of understanding from just reading about them, or even working your way down from the top. But if you start at the bottom and understand each piece, then it's relatively easy to figure out the specifics for your database. Even things that lots of database geeks can't seem to grok, like when to use a non-relational database.

Maybe that's a bit strict, especially if you didn't study computer science in college. I'll tone it down some: You could write one today, completely, from scratch. I don't care if you know the specifics of how the PostgreSQL query optimizer works, but if you know enough to write one yourself, it probably won't be too different from what they did. And you know, it's really not that hard to write a basic one.

From the linked Joel article about C strings, doesn't the following snippet of lead to undefined behavior:char* str = "*Hello!";str[0] = strlen(str) - 1;str is a string literal and is general in read only memory. You cannot write to it:?
A professional database expert, fine, but *every developer*?
Ben Aston
Ben: Every professional developer who uses databases frequently, yeah. They're really not that hard, so if you don't know how, it means you've never taken even a little time to learn how DBs work. Every computer science major I graduated with designed a CPU and implemented an OS. A database is simpler than either of these, so if you spend any time using one, I don't see an excuse for not knowing about how they work.
+1  A: 
  • Basic SQL skills.
  • Indexing.
  • Deal with different incarnations of DATE/ TIME/ TIMESTAMP.
  • JDBC driver documentation for the platform you are using.
  • Deal with binary data types (CLOB, BLOB, etc.)
+3  A: 

Excellent question. Let's see, first no one should consider querying a datbase who does not thoroughly understand joins. That's like driving a car without knowing where the steering wheel and brakes are. You also need to know datatypes and how to choose the best one.

Another thing that developers should understand is that there are three things you should have in mind when designing a database:

  1. Data integrity - if the data can't be relied on you essentially have no data - this means do not put required logic in the application as many other sources may touch the database. Constraints, foreign keys and sometimes triggers are necessary to data integrity. Don't fail to use them because you don't like them or don't want to be bothered to understand them.

  2. Performance - it is very hard to refactor a poorly performing database and performance should be considered from the start. There are many ways to do the same query and some are known to be faster almost always, it is short-sighted not to learn and use these ways. Read some books on performance tuning before designing queries or database structures.

  3. Security - this data is the life-blood of your company, it also frequently contains personal information that can be stolen. Learn to protect your data from SQL injection attacks and fraud and identity theft.

When querying a database, it is easy to get the wrong answer. Make sure you understand your data model thoroughly. Remember often actual decisions are made based on the data your query returns. When it is wrong, the wrong business decisions are made. You can kill a company from bad queries or loose a big customer. Data has meaning, developers often seem to forget that.

Data almost never goes away, think in terms of storing data over time instead of just how to get it in today. That database that worked fine when it had a hundred thousand records, may not be so nice in ten years. Applications rarely last as long as data. This is one reason why designing for performance is critical.

Your database will probaly need fields that the application doesn't need to see. Things like GUIDs for replication, date inserted fields. etc. You also may need to store history of changes and who made them when and be able to restore bad changes from this storehouse. Think about how you intend to do this before you come ask a web site how to fix the problem where you forgot to put a where clause on an update and updated the whole table.

Never develop in a newer version of a database than the production version. Never, never, never develop directly against a production database.

If you don't have a database administrator, make sure someone is making backups and knows how to restore them and has tested restoring them.

Database code is code, there is no excuse for not keeping it in source control just like the rest of your code.

+15  A: 

First, developers need to understand that there is something to know about databases. They're not just magic devices where you put in the SQL and get out result sets, but rather very complicated pieces of software with their own logic and quirks.

Second, that there are different database setups for different purposes. You do not want a developer making historical reports off an on-line transactional database if there's a data warehouse available.

Third, developers need to understand basic SQL, including joins.

Past this, it depends on how closely the developers are involved. I've worked in jobs where I was developer and de facto DBA, where the DBAs were just down the aisle, and where the DBAs are off in their own area. (I dislike the third.) Assuming the developers are involved in database design:

They need to understand basic normalization, at least the first three normal forms. Anything beyond that, get a DBA. For those with any experience with US courtrooms (and random television shows count here), there's the mnemonic "Depend on the key, the whole key, and nothing but the key, so help you Codd."

They need to have a clue about indexes, by which I mean they should have some idea what indexes they need and how they're likely to affect performance. This means not having useless indices, but not being afraid to add them to assist queries. Anything further (like the balance) should be left for the DBA.

They need to understand the need for data integrity, and be able to point to where they're verifying the data and what they're doing if they find problems. This doesn't have to be in the database (where it will be difficult to issue a meaningful error message for the user), but has to be somewhere.

They should have the basic knowledge of how to get a plan, and how to read it in general (at least enough to tell whether the algorithms are efficient or not).

They should know vaguely what a trigger is, what a view is, and that it's possible to partition pieces of databases. They don't need any sort of details, but they need to know to ask the DBA about these things.

They should of course know not to meddle with production data, or production code, or anything like that, and they should know that all source code goes into a VCS.

I've doubtless forgotten something, but the average developer need not be a DBA, provided there is a real DBA at hand.

David Thornley
+51  A: 

The very first thing developers should know about databases is this: what are databases for? Not how do they work, nor how do you build one, nor even how do you write code to retrieve or update the data in a database. But what are they for?

Unfortunately, the answer to this one is a moving target. In the heydey of databases, the 1970s through the early 1990s, databases were for the sharing of data. If you were using a database, and you weren't sharing data you were either involved in an academic project or you were wasting resources, including yourself. Setting up a database and taming a DBMS were such monumental tasks that the payback, in terms of data exploited multiple times, had to be huge to match the investment.

Over the last 15 years, databases have come to be used for storing the persistent data associated with just one application. Building a database for MySQL, or Access, or SQL Server has become so routine that databases have become almost a routine part of an ordinary application. Sometimes, that initial limited mission gets pushed upward by mission creep, as the real value of the data becomes apparent. Unfortunately, databases that were designed with a single purpose in mind often fail dramatically when they begin to be pushed into a role that's enterprise wide and mission critical.

The second thing developers need to learn about databases is the whole data centric view of the world. The data centric world view is more different from the process centric world view than anything most developers have ever learned. Compared to this gap, the gap between structured programming and object oriented programming is relatively small.

The third thing developers need to learn, at least in an overview, is data modeling, including conceptual data modeling, logical data modeling, and physical data modeling.

Conceptual data modeling is really requirements analysis from a data centric point of view.

Logical data modeling is generally the application of a specific data model to the requirements discovered in conceptual data modeling. The relational model is used far more than any other specific model, and developers need to learn the relational model for sure. Designing a powerful and relevant relational model for a nontrivial requirement is not a trivial task. You can't build good SQL tables if you misunderstand the relational model.

Physical data modeling is generally DBMS specific, and doesn't need to be learned in much detail, unless the developer is also the database builder or the DBA. What developers do need to understand is the extent to which physical database design can be separated from logical database design, and the extent to which producing a high speed database can be accomplished just by tweaking the physical design.

The next thing developers need to learn is that while speed (performance) is important, other measures of design goodness are even more important, such as the ability to revise and extend the scope of the database down the road, or simplicity of programming.

Finally, anybody who messes with databases needs to understand that the value of data often outlasts the system that captured it.


Walter Mitty
Very well written! And the historical perspective is great for people who weren't doing database work at that time (i.e. me).
Nicely written. And I think your last point is ignored far too often by people trying to 'just get it done'.
There's a connection between what I wrote and topics such as Explain Plan, Indexing, and Data Normalization. I'd love to discuss that connection in greater depth in some sort of discussion forum. SO is not such a forum.
Walter Mitty
You know you have ADD when you stare at the above post for the last 20 minutes, know there is lots of good information, and can't bring yourself to read it because it's written like an essay instead of bullet points.
If you found reading this monster dautning, imagine what it felt like to write it! I didn't set out to write an essay. Once I got started, it just seemed to flow.Whoever added the bolding really helped the readers, IMO.
Walter Mitty
+3  A: 

About the following comment to Walter M.'s answer:

"Very well written! And the historical perspective is great for people who weren't doing database work at that time (i.e. me)".

The historical perspective is in a certain sense absolutely crucial. "Those who forget history, are doomed to repeat it.". Cfr XML repeating the hierarchical mistakes of the past, graph databases repeating the network mistakes of the past, OO systems forcing the hierarchical model upon users while everybody with even just a tenth of a brain should know that the hierarchical model is not suitable for general-purpose representation of the real world, etcetera, etcetera.

As for the question itself:

Every database developer should know that "Relational" is not equal to "SQL". Then they would understand why they are being let down so abysmally by the DBMS vendors, and why they should be telling those same vendors to come up with better stuff (e.g. DBMS's that are truly relational) if they want to go on sucking hilarious amounts of money out of their customers for such crappy software).

And every database developer should know everything about the relational algebra. Then there would no longer be a single developer left who had to post these stupid "I don't know how to do my job and want someone else to do it for me" questions on Stack Overflow anymore.

Erwin Smout
I agree that a developer needs to know where SQL and the RDM diverge. Having said that, judicious use of the RDM can be an invaluable aide to the database designer, even if the implementation is SQL.
Walter Mitty
In case you forgot, George Santayana, wrote that classic quote...

Three (things) is the magic number:

  1. Your database needs version control too.

  2. Cursors are slow and you probably don't need them.

  3. Triggers are evil*

*almost always

Three is the magic number?
Jaco Pretorius
Yes it is - it's the magic number.
+1  A: 

For some projects, and Object-Oriented model is better.

For other projects, a Relational model is better.

Mark Lutton
+9  A: 


It always depresses me to see somebody struggling to write an excessively complicated query that would have been completely straightforward with a normalized design ("Show me total sales per region.").

If you understand this at the outset and design accordingly, you'll save yourself a lot of pain later. It's easy to denormalize for performance after you've normalized; it's not so easy to normalize a database that wasn't designed that way from the start.

At the very least, you should know what 3NF is and how to get there. With most transactional databases, this is a very good balance between making queries easy to write and maintaining good performance.

+8  A: 

Basic Indexing

I'm always shocked to see a table or an entire database with no indexes, or arbitrary/useless indexes. Even if you're not designing the database and just have to write some queries, it's still vital to understand, at a minimum:

  • What's indexed in your database and what's not:
  • The difference between types of scans, how they're chosen, and how the way you write a query can influence that choice;
  • The concept of coverage (why you shouldn't just write SELECT *);
  • The difference between a clustered and non-clustered index;
  • Why more/bigger indexes are not necessarily better;
  • Why you should try to avoid wrapping filter columns in functions.

Designers should also be aware of common index anti-patterns, for example:

  • The Access anti-pattern (indexing every column, one by one)
  • The Catch-All anti-pattern (one massive index on all or most columns, apparently created under the mistaken impression that it would speed up every conceivable query involving any of those columns).

The quality of a database's indexing - and whether or not you take advantage of it with the queries you write - accounts for by far the most significant chunk of performance. 9 out of 10 questions posted on SO and other forums complaining about poor performance invariably turn out to be due to poor indexing or a non-sargable expression.

Can you elaborate on "coverage" ? I can see why SELECT * is not a good habit to get into, but I don't know the meaning of "coverage" and wonder if it alludes to another reason to avoid SELECT *.
@Edmund: An index *covers* a query if all of the output fields are *part of* the index (either as indexed columns or `INCLUDE` columns in SQL Server). If the only available index for a given query is non-covering, then all of the rows have to be retrieved, one-by-one, which is a very slow operation, and much of the time the query optimizer will decide that it isn't worth it and perform a full index/table scan instead. That's why you don't write `SELECT *` - it virtually guarantees that no index will cover the query.
@Aaronaught -- thanks! Though as a PostgreSQL user I don't need to worry about such things (yet?): indexes don't contain visibility information so table tuples always need to be scanned too. In general, though, it looks like a pretty important factor.
@Edmund: PostgreSQL may not have `INCLUDE` columns (I can't say for sure), but that doesn't mean you can't put columns you wish to cover in the actual index data. That's what we had to do back in the SQL Server 2000 days. Coverage still matters no matter which DBMS you're on.
+2  A: 

Aside from syntax and conceptual options they employ (such as joins, triggers, and stored procedures), one thing that will be critical for every developer employing a database is this:

Know how your engine is going to perform the query you are writing with specificity.

The reason I think this is so important is simply production stability. You should know how your code performs so you're not stopping all execution in your thread while you wait for a long function to complete, so why would you not want to know how your query will affect the database, your program, and perhaps even the server?

This is actually something that has hit my R&D team more times than missing semicolons or the like. The presumtion is the query will execute quickly because it does on their development system with only a few thousand rows in the tables. Even if the production database is the same size, it is more than likely going to be used a lot more, and thus suffer from other constraints like multiple users accessing it at the same time, or something going wrong with another query elsewhere, thus delaying the result of this query.

Even simple things like how joins affect performance of a query are invaluable in production. There are many features of many database engines that make things easier conceptually, but may introduce gotchas in performance if not thought of clearly.

Know your database engine execution process and plan for it.

+8  A: 

I just want to point out an observation - that is that it seems that the majority of responses assume database is interchangeable with relational databases. There are also object databases, flat file databases. It is important to asses the needs of the of the software project at hand. From a programmer perspective the database decision can be delayed until later. Data modeling on the other hand can be achieved early on and lead to much success.

I think data modeling is a key component and is a relatively old concept yet it is one that has been forgotten by many in the software industry. Data modeling, especially conceptual modeling, can reveal the functional behavior of a system and can be relied on as a road map for development.

On the other hand, the type of database required can be determined based on many different factors to include environment, user volume, and available local hardware such as harddrive space.

Do you mean like in doing entity-relationship diagrams?
Yes... did I forget to mention ERDs?:-)
@FernandoZ: +1... But you have to realize you are on SO: the home of plumbers spending their days fixing the ORM impedance mismatch so all they know, eat and think is not just relational but "SQL" :)
+4  A: 

Evolutionary Database Design.

These agile methodologies make database change process manageable, predictable and testable.

Developers should know, what it takes to refactor a production database in terms of version control, continious integration and automated testing.

Evolutionary Database Design process has administrative aspects, for example a column is to be dropped after some life time period in all databases of this codebase.

At least know, that Database Refactoring concept and methodologies exist.

Classification and process description makes it possible to implement tooling for these refactorings too.

George Polevoy
i love the refactoring concept, but regarding DB the real big issue with it is persistent data. refactoring DB often involves data migration which in reality is tough, especially if you aren't allowed any downtime of the system. also rollback isn't trivial. in my view difficulties in proper/safe rollout + rollback strategies are often showstoppers to refactor DB as lightweight as application code. itself it often makes sense to refactor stuff but you always have to outweigh cost/benefits.
manuel aldana
See also Ambler's 'Refactoring Databases' (
Jonathan Leffler
+2  A: 

I think a lot of the technical details have been covered here and I don't want to add to them. The one thing I want to say is more social than technical, don't fall for the "DBA knowing the best" trap as an application developer.

If you are having performance issues with query take ownership of the problem too. Do your own research and push for the DBAs to explain what's happening and how their solutions are addressing the problem.

Come up with your own suggestions too after you have done the research. That is, I try to find a cooperative solution to the problem rather than leaving database issues to the DBAs.

good answer. We each have our own area we contribute to every problem or solution.
+4  A: 

Avoiding SQL injection and how to secure your database

+2  A: 

Consider Denormalization as a possible angel, not the devil, and also consider NoSQL databases as an alternative to relational databases.

Also, I think the Entity-Relation model is a must-know for every developper even if you don't design databases. It'll let you understand thoroughly what's your database all about.

+3  A: 

I think every developer should understand that databases require a different paradigm.

When writing a query to get at your data, a set-based approach is needed. Many people with an interative background struggle with this. And yet, when they embrace it, they can achieve far better results, even though the solution may not be the one that first presented itself in their iterative-focussed minds.

Rob Farley
Please clarify what is meant by "set-based" approach
Rice Flour Cookies
That you should look at data as being in sets, and considering your problems as potentially solved by set arithmetic - involving ranking functions where required, subqueries, aggregates, and so on. Many developers think about what needs to be done to each row, which is iterative thinking.
Rob Farley
+1  A: 

The impedance mismatch problem, and know the common deficiencies or ORMs.

Muhammad Adel
+2  A: 

The order of columns in a non-unique index is important.

The first column should be the column that has the most variability in its content (i.e. cardinality).

This is to aid SQL Server ability to create useful statistics in how to use the index at runtime.

Mike D
+1  A: 

RDBMS Compatibility

Look if it is needed to run the application in more than one RDBMS. If yes, it might be necessary to:

  • avoid RDBMS SQL extensions
  • eliminate triggers and store procedures
  • follow strict SQL standards
  • convert field data types
  • change transaction isolation levels

Otherwise, these questions should be treated separately and different versions (or configurations) of the application would be developed.

+2  A: 

Simple respect.

  • It's not just a repository
  • You probably don't know better than the vendor or the DBAs
  • You won't support it at 3 a.m. with senior managers shouting at you
+1  A: 

From my experience with relational databases, every developer should know:

- The different data types:

Using the correct type for the correct job will make your DB design more robust, your queries faster and your life easier.

- Learn about 1xM and MxM:

This is the bread and butter for relational databases. You need to understand one-to-many and many-to-many relations and apply then when appropriate.

- "K.I.S.S." principle applies to the DB as well:

Simplicity always works best. Provided you have studied how DB work, you will avoid unnecessary complexity which will lead to maintenance and speed problems.

- Indices:

It's not enough if you know what they are. You need to understand when to used them and when not to.


  • Boolean algebra is your friend
  • Images: Don't store them on the DB. Don't ask why.
  • Test DELETE with SELECT
+1 for Images. I'd replace 'Images' with 'BLOBs' though.
Vulcan Eager
I'm not really sure about the "simplicity" part. The simplest possible database is one giant table with a bunch of `varchar(max)` columns. Relational databases should be *normalized*, not *simplified*.
Your concerns are covered earlier, in the "data types" part of my post. I was referring to the (unecessary) use of stored procedures / triggers / cursors and so on.
+1  A: 

Don't depend on the order of rows returned by an SQL query.

Vulcan Eager
...unless there's an `ORDER BY` clause in it?
And don't use `ORDER BY` unnecessarily because it adds load to the SQL server
Rice Flour Cookies
+2  A: 

Understand the tools that you use to program the database!!!

I wasted so much time trying to understand why my code was mysteriously failing.

If you're using .NET, for example, you need to know how to properly use the objects in the System.Data.SqlClient namespace. You need to know how to manage your SqlConnection objects to make sure they are opened, closed, and when necessary, disposed properly.

You need to know that when you use a SqlDataReader, it is necessary to close it separately from your SqlConnection. You need to understand how to keep connections open when appropriate to how to minimize the number of hits to the database (because they are relatively expensive in terms of computing time).

Rice Flour Cookies

Never insert data with the wrong text encoding.

Once your database becomes polluted with multiple encodings, the best you can do is apply some kind combination of heuristics and manual labor.

What is the "wrong text encoding" and how does it happen?
@vgv8, it happens when your client allows users to submit text in any encoding you want, you blindly store it. Then, when you need to perform some sort of transformation or analysis, your code breaks, because your application assumes utf-8, but some idiot added utf-16 data, and your program errors or starts spitting out gibberish.
+2  A: 

How Indexes Work

It's probably not the most important, but for sure the most underestimated topic.

The problem with indexing is that SQL tutorials usually don't mention them at all and that all the toy examples work without any index.

Even more experienced developers can write fairly good (and complex) SQL without knowing more about indexes than "An index makes the query fast".

That's because SQL databases do a very good job working as black-box:

Tell me what you need (gimme SQL), I'll take care of it.

And that works perfectly to retrieve the correct results. The author of the SQL doesn't need to know what the system is doing behind the scenes--until everything becomes sooo slooooow.....

That's when indexing becomes a topic. But that's usually very late and somebody (some company?) is already suffering from a real problem.

That's why I believe indexing is the Nr. 1 topic not to forget when working with databases. Unfortunately, it is very easy to forget it.


The arguments are borrowed from the preface of my free eBook "Use The Index, Luke". I am spending quite a lot of my time explaining how indexes work and how to use them properly.

Markus Winand