tags:

views:

52

answers:

2

Hi, I'm a novice, both here at stack overflow as well as on sql server, so please let me know if this question is not appropriate in any way :)

Well, I'm developing a web application that will be used to analyze large amounts of data, which is stored in a SQL Server 2008 database. The interface will not allow the users to update or insert any data, so except from some updates of user data, mainly SELECT commands will be sent to the db.

Every night, the system will close down and refresh it's information from other sources. This refresh will also involve large amounts of data so the database will mostly perform INSERT and UPDATE commands during this stage.

I've created appropriate indexes to achieve good performance on SELECTS, but theses indexes causes the nightly refresh to be slow. I want the "best of two worlds", so I've googled around and found out that a common strategy is to drop/disable all indexes before writing data, and to re-create them afterwards. I've also heard that a better approach is to limit the fillfactor of the indexes, which would save me from writing the scripts for dropping and re-creating the indexes.

What do you think is the best approach to use here is my main goal is good performance? Should I go with the "fillfactor"-approach or should I get dirty and write scripts for dropping/re-creating the indexes? Any suggestions are welcome!

+2  A: 

Drop/re-create indexes nightly will help. Fill factor will only provide benefit if the level of insert/updates is causing fragmentation. You can check this by running DBCC SHOWCONTIG (careful on production and if DB is large).

Mitch Wheat
Just like to add, we have a process at one of our clients which juggles around 100GB of data per night. In our case we drop the indexes, truncate the tables, do our inserts, then add the indexes back in. We're talking hundreds of millions of records. This path sped up processing by a factor of 40. Meaning, it used to take 20 hours, now it takes 30 minutes.
Chris Lively
Also, pay attention to your recovery model. When dealing with a lot of data this way it becomes critical. You might want to experiment with the different models until you find one that keeps the performance you want with the restore capabilities you need.
Chris Lively
A: 

another option which should improve performance is to switch the recovery model of the database from full to simple during your 'system refresh' & then switch back.

You should take a full backup after you switch back to full, but it might be worth it, best is to try it against the workload & see.

Nick Kavadias
thanks for the tip Nick (and Chris), i have to admit I haven't thought about the recovery model until now. I'll give it a try!
Ozzy