Ressource usage when storing a vast stream of data into a database

Question

First of all, i don't have much of a clue about databases, so please forgive me if this is a stupid question.

My company is receiving a torrent of positional data via TCP. Each packet consists of a few floats and a timestamp. Our task is to store the data into a database (MS SQL Server 2005), running on a modern PC.

Each blob has about 20 bytes. There are up to 1000 senders, each sending about 25 packages per second.

Is a single, normal PC able to cope with this amount of data? If not, how to deal with this problem? Should we set up a dedicated PC for each sender?

Answer 1

So to understand your requirements:
You want to do up to 25 000 inserts a second
You want to store up to 20 bytes * 1000 senders * 25 packets * 60 seconds * 60 minutes * 24 hours = 43.2 GB / day.

A couple of things:

• This is a seriously large amount of data.
• A normal PC can't cope with this, you'll need a capable server, and very large disk array to handle this.
• What do you plan to do with your data? If you want to process it, you might want to look at data warehouses etc.

Finally this isn't your area of expertise, for this size of project, you need an expert.
Find one, even if it's for a short term consultancy.