views:

102

answers:

2

I'm using SQL Server 2005 Express, and I'm running into a strange issue. I have a table called "DailyPrice" that has about 24 million records (I was able to make this table thanks to all your help in this thread: http://stackoverflow.com/questions/1983611/sql-server-2005-slows-down-as-i-keep-adding-rows)

Now, I'm running a different fancy function in VB.NET where I need to get a list of entries based on a ticker symbol (just 1), and I need them organized from oldest to newest.

This query works:

SELECT     Ticker, DateStamp, ClosePrice
FROM         DailyPrice
WHERE     (Ticker = 'DD')

This one is either agonizing slow, or I get a timeout error:

SELECT     Ticker, DateStamp, ClosePrice
FROM         DailyPrice 
WHERE     (Ticker = 'DD')
ORDER BY DateStamp

The fields Ticker and DateStamp are my primary index, in case that matters. Why does this slow down when i use ORDER BY? A couple hours ago the problem query ran speedily and fine, but only once - and now I'm plagued by timeouts.

+1  A: 

UPDATE your statistics or rebuild your indexes.

This will re-index an entire database (caution if database is very large!):

exec sp_msforeachtable "dbcc dbreindex('?')" 
Mitch Wheat
Would that explain why the problem is intermittent? It went "fast" again about 2 minutes ago, and now it's slow as crap and times out again. For index rebuilding - where would I go to find info on that? Just delete my current index and recreate it?
Bill Sambrone
actually those symptoms sound more like an incorrectly cached query plan.
Mitch Wheat
...but that can also happen due to out of date statistics.
Mitch Wheat
I'm very new to SQL, I haven't set up any sort of query plan... much less know what one is :) I just googled rebuilding an index, I assume there isn't anything special I should know/worry about?
Bill Sambrone
You say Ticker and DateStamp are your primary index (i guess you mean: Primary Key?) If so, is it (Ticker, DateStamp) or (DateStamp, Ticker)? You could try to create an additional index having the columns in reverse order (so if you PK is (Ticker, DateStamp) create an additional (DateStamp, Ticker) and vice versa)
Roland Bouman
Yes, I meant primary key. I created the PK(Ticker, DateStamp) by right clicking on some index option from the Management Studio?
Bill Sambrone
Ok. (Ticker, DateStamp) is indeed the one you want for this particular query. But for reporting on one or more tickers within a specific time period, you would probably benefit more from an index having the columns the other way around, that is, (DateStamp, Tickerer). You can have both indexes simultaneously, but it can make a difference which one you choose to be PK. MS SQL PK's are clustered: the data is ordered on disk according to the PK. So for a time period, all rows are close together, allowing efficient retrieval of all rows in the same time period. Experiment and see.
Roland Bouman
Your statement "MS SQL PK's are clustered" is incorrect. The primary key and the clustered index are two seperate things. SQL server does create them that way by default....
Mitch Wheat
A: 

Query with ORDER BY will always take longer to return than the query without it. This is because SQL Server has to get all the records first and then sort them before it can return the result to the client. You may be able to speed up the query by adding indexes that match the fields you use in ORDER BY and WHERE clauses. But be careful, too many indexes can cause insert/update to be much slower and also SQL Server can decide to use the wrong index. You can specify the index as part of the table hint in FROM clause but that will make your query dependent on the presence of that index. Finally having 24+ mil records in one table will always cause issues. I would suggest you consider partitioning you table, specially if you can find a field that splits the data in several smaller sets of similar size - look at SQL Server Books online for more info on partitioning (http://msdn.microsoft.com/en-us/library/ms345146%28SQL.90%29.aspx)

DeanM