Windows Azure Storage vs. SQL Azure

Question

I saw that there was a similar question asked several months back, but it really didn't address my situation well. Here it goes...

I'm in the process of building from scratch a web-based, .NET application that has the potential to become a high-volume site (several hundred thousand page views a month to start) and am strongly considering using Windows Azure to host it. I have not built anything yet and am still researching my different options.

The application itself is, at its core, a standard CRUD application that acts upon a number of different types of entities (e.g. user, order, item, etc.). There are probably some background processes that may be running and some queuing of data (for non-realtime updates -- like getting a SO badge, for example), but most of the interactions with the user will be your typical CRUD type of actions.

Regarding Azure, I have read a number of articles about using Windows Azure Storage to store transactional data and am strongly considering doing that instead of using SQL Azure. However, I haven't seen or read a number of success stories of real people and/or real companies doing that. So I thought I'd reach out to the SO community to see if anyone has had any experience with using Windows Azure Storage, what kind of luck have you had, any gotchas I should look out for, and any best practices that you've come up with.

I've read through a lot of the Windows Azure MSDN section and the programming Windows Azure Table API document from Microsoft. I'm looking for practical advice, lessons learned, best practices, etc. Thanks in advance!

Answer 1

Windows Azure storage is just like as any NoSQL storage. It works in high-scale scenarios for us (pumping millions of records per user). However, classical CRUD approach is a bit hard to scale or to adapt to this.

I would recommend to start looking along the CQRS style of architectures. Here are some references that might get you started:

• Open Source Lokad.CQRS project for Windows Azure (patterns and guidance included)
• Efficient architectures for Azure discussion at MSDN

Answer 2

You should also check your data usage pattern before deciding to use Azure Storage or SQL storage. Since Azure Storage is providing NoSQL solutions they are more geared towards non reporting base requirements. Here reporting does not mean reports but it implies that querying capabilities of the Azure storage is limited\ non optimize for various querying scenarios. With CQRS architecture the CRUD and Reporting operations are decoupled and hence a mix n match of both Azure Storage and Azure SQL can be done.

Answer 3

Another place to find information is to step outside of the Windows Azure options and look at AWS's. The S3 & SimpleDB options have been vetted over a longer period of time and have many additional success stories out there on the web. However, S3 & SimpleDB are very similar in functionality to Windows Azure Storage Table and Blob Storage. If you're thinking about truly big data, which these structures are for, definitely check out the AWS options. If only for a reference point on existing solutions built around big data.

As for SQL Azure, it is great for lots of transactions, keeping transaction costs low, and maintaining relationships and general integrity based on relational data. However, if you are going to have massive volumes of data, just go ahead and aim for the big data structures, such as Windows Azure Table or Amazon's SimpleDB.

Answer 4

Depends on what kind of data you are talking about - generally there is a tendency to overestimate transactional data requirements. A lot of data can really be fit into a 1 GB SQL Azure (we are a SAAS provider and transactional data of almost 20 clients can fit into that much space). Also, for some strange reason, I have seen that SQL Azure space consumption seems to be somewhat lesser than the size of the database I see on-premises (might have to do with how they handle logs, not sure). And now 50 GB is the limit, which is quite frankly, HUGE.

However for this, you also need to consider what increases the space usage - storing images, videos or other large objects in the database can create considerable increase in space consumption. It is better to keep these kind of objects in Windows Azure.

So short answer - keep transactional data in SQL Azure and non-relational data in Windows Azure. Working with SQL Azure will also keep your developers more productive, since it is quite familiar in terms of programming. Treat Windows Azure similar to how you would treat windows local file storage with some added benefits (basic table structures supported).