performance of table access

Question

Hi,

we are having an application which is completely written in C. for table accessing inside the code like fetching some values from atable we use pro*C and for increasing the performance of the application we also preload some tables for fetching the data.we take some input fields and fetch the output fields from the table in general.

we usually have around 30000 entries in the table and max it recahes to 0.1 million some times. but if the table entries are increased to around 10 million entries,i think it dangerously effects the performance of the application.Am i wrong some where? if it really affects the performance is there any way to keep the performance of the application stable?

my question over here is what is the possible work around if the number of rows in the table are increased to 10 million considering the way the application works with tables.?

Answer 1

If you are not sorting the table you'll get a proportional increase of search time... if you don't code anything wrong, in your example (30K vs 1M) you'll get 33X greater search times. I'm assumning you're incrementally iterating (i++ style) the table.

However, if it's somehow possible to sort the table, then you can greatly reduce search times. That is possible because an indexer algorithm that searchs sorted information will not parse every element till it gets to the sought one: it uses auxiliary tables (trees, hashes, etc), usually much faster to search, and then it pinpoints the correct sought element, or at least gets a much closer estimate of where it is in the master table.

Of course, that will come at the expense of having to sort the table, either when you insert or remove elements from it, or when you perform a search.

Answer 2

maybe you can go to 'google hash' and take a look at their implementation? although it is in C++