What would it take to write my own database system?
I would want to make it a server type system.
Have any successful database systems been written by one person?
+8
A:
Short answer, a lot.
Longer answer; there's a whole bunch of stuff to consider, and it requires more detail than can be gone into in one question. You'd need to deal with data storage, indexing, SQL parsing, the database catalogue, transaction management and a whole host of other stuff, including all the client handling and authentication. There's also all the stuff I've forgotten, which I expect someone else will point out.
If you really want to write a database system, you're going to want to read up many resources, starting with a good book that explains everything in detail.
Rob
2009-05-25 02:42:38
What books do you recommend?
2009-05-25 02:44:21
Frankly, I'm not in a position to recommend any; I've never embarked on the task of writing a database engine. If you're serious about this project, then surely you can do a little research and find a few books? Perhaps there's a "Sams Teach Yourself DBMS Implementation In 30 Days" you might like.
Rob
2009-05-25 02:47:39
+1 for that short answer. :-)
Thanatos
2009-05-25 02:56:08
+1. Just implementing transaction management correctly (let alone efficiently) for a multi-user DBMS is difficult enough by itself to take years out of your life. Are you sure you want to do this?
j_random_hacker
2009-05-25 03:06:41
See the answer from @altan for one suggested book.
Jonathan Leffler
2009-05-25 04:10:06
See also: http://stackoverflow.com/questions/751236/relational-databases-there-has-to-be-more-right
Jonathan Leffler
2009-05-25 04:11:17
+1
A:
If you're not specifically looking to create a relational database, you might want to check out the source code for CouchDB. It is written in Erlang and provides a schema-less approach to database storage, retrieval and replication. These are just some of the major features and considerations to take into account when deciding if you want to create your own database.
Soviut
2009-05-25 02:48:15
+1
A:
best thing would be to look at the code for other DBs, specifically Open Source DBs, and try to figure out whats going on.
From that you can work out what each part does, and then focus on the parts you're interested in, and how they've been written.
MySQL and PostgreSQL are both Open Source. Check it out.
UPDATE:
haha, yes, point taken, just looking at the code might be a bit overwhelming. But there's definitely some value in it for sure I think. Plus the person who asked the question might not have been aware of Open Source DBs. This answer might even point them to look for smaller DBs which make their source available.
andy
2009-05-25 02:52:46
Looking at code might be a good idea, but you're still going to need a pretty thorough understanding of the fundamentals, which could well be difficult to obtain from code, especially if it's been "optimised" or suchlike.
Rob
2009-05-25 02:54:15
Thats like saying "go look at *nix code to learn how to write an operating system." Now granted you *could* do that if you felt like wasting a couple of years but rather than do that, buy a couple of books and learn the appropriate way. These things aren't PHP shopping carts that you can dissect in a few hours.
Chance
2009-05-25 03:02:00
+1 I also saw this as downvoted unnecessarily. Most books will teach you how to use a RDBMS rather than how to create one. There's no better way to gain insight into the internal operation of a RDBMS than viewing the source code.
Matt Brunell
2009-05-25 03:15:14
Yea, "most" books will, but I guarantee there are a ton of books on not only how a RDBMS operates but down to specific subsets of functionality.
Chance
2009-05-25 03:59:08
+4
A:
Yes you can, if it's simple enough. For example flat-file data storage with a REST interface would certainly fit in a one-person scope.
A full-blown RDBMS that would compete with what's available for free? No.
Glenn
2009-05-25 02:57:02
A:
Depends on what you want to write, so I'd like to see the question fleshed out a bit. For example:
Do you want to handle just you and maybe some client software that you control, or do you want a full general-purpose database engine that would be used by lots of people around the world?
Must it be relational?
When you say "server type system", do you mean the system would store data in flat files living on a server, or do you mean a client/server architecture?
What are the requirements for uptime? 24x7? Or is it ok to take the system down for a few minutes per year?
I sense that you have in mind something simple, but I'm not sure what's considered thorough engineering vs over-engineering.
Phone Guy
2009-05-25 03:52:07
+2
A:
If you're serious about this, Database Systems: The Complete Book (Hector Garcia-Molina, Jeff Ullman, and Jennifer Widom) will tell you more than you need to know. Writing all those components is a pretty big commitment even if you're aiming for a small system.
altan
2009-05-25 03:55:21
I'm not sure about 'more than you need to know'; there's lots of stuff that isn't covered - but it is a good starting point (better than most).
Jonathan Leffler
2009-05-25 04:09:28