How to prevent Delphi ADO from loading the entire table into memory?

Question

hi

I am not a Delphi programmer, but I I got an old Delphi 7 application that I need to fix and it is using ADO.

The database table (MS Accesss) contains +100,000 rows and when I set the ADOTable.Active=true it starts to load the entire table into RAM and that takes a lot of memory and time.

How can I prevent ADO to load the entire table? I tried to set the MaxRecords but it does not help.

Basically all we do is att program startup:

// Connect to database
DataModule.MyADOConnection.Connected:=true;

DataModule.MeasurementsADOTable.MaxRecords:=1;

// Open datatables
DataModule.MeasurementsADOTable.Active:=true;

After setting Active=true it starts to load the entire measurements into RAM and it takes TIME!

We are using the MSDASQL.1 provider. Perhaps it does not support the MaxRecords property?

How do I add some limiting query into this data object to only "load TOP 1 * from Measurements" ?

Answer 1

You could use TADOQuery to limit the result set with a sql query. Or you could use TADOTable and set the CursorLocation to a Server side cursor to prevent the client loading the complete resultset in memory.

Answer 2

1. On your datamodule where "MeasurementsADOTable" currently resides, drop a TADOQuery and name it "MeasurementsADOQuery"
2. Set the Connection property of MeasurementsADOQuery to MyADOConnection (assuming this is the case based on the little code snippet provided.)
3. I'm also assuming that you are displaying a grid or otherwise using a DataSource - change the DataSource component's "DataSet" property from MeasurementsADOTable to MeasurementsADOQuery
4. Edit the actual query to be executed by setting the SQL property of MeasurementsADOQuery. (In runtime before opening: Measurements.SQL.Text := 'select top 10 * from measurements order by whatever')
5. Analyze/change all references in code from MeasurementsADOTable to MeasurementsADOQuery

Answer 3

You could use that adoTable with an Server OpenForwardOnly cursor and an TCLientDataset with PacketRecords set to nonzero value. Worked wonderfully when I had to write an app to pump data from MSSQL to Oracle on a customized way with tables with millions of records.

Answer 4

I have found ADO + Access w/Delphi to be painfully slow, for lots of things (big table reads like you're describing, but also inserts as well, etc). My answer became "Quit using ADO and Access altogether."

Never did understand why it performed so poorly, especially when earlier technologies seemed not to.

Answer 5

This article is BDE specific, but applies to ADO or most client data access libraries.

http://dn.codegear.com/article/28160

I would recommend using TADODataSet (it's "closer" to the ADO layer than TADOQuery) and selecting only the data the client needs by providing a custom search form (date range, list of specific items, etc)

Good luck