views:

1642

answers:

35

I have a Delphi 2009 program that handles a lot of data and needs to be as fast as possible and not use too much memory.

What small simple changes have you made to your Delphi code that had the biggest impact on the performance of you program by noticeably reducing execution time or memory use?


Thanks everyone for all your answers. Many great tips.

For completeness, I'll post a few important articles on Delphi optimization that I found.

Before you start optimizing Delphi code at About.com

Speed and Size: Top 10 Tricks also at About.com

Code Optimization Fundamentals and Delphi Optimization Guidelines at High Performance Delphi, relating to Delphi 7 but still very pertinent.

+3  A: 

Separating the program logic from user interface, refactoring, then optimizing the most-used, most resource-intensive elements independently.

Argalatyr
+2  A: 
  • Turn debugging OFF

  • Turn optimizations ON

  • Remove all references to units that you don't actually use

  • Look for memory leaks

Maltrap
> Turn debugging OFFThis has no effect on code size or code speed. The debug information is kept in the dcu and dcp files but is not added to the executable.
Andreas Hausladen
+10  A: 

Pre-allocating lists and arrays, rather than growing them with each iteration.

This has probably had the biggest impact for me in terms of speed.

Argalatyr
+4  A: 

When working with a tstringlist (or similar), set "sorted := false" until needed (if at all). Seems like a no-brainer...

Argalatyr
+4  A: 

Make intelligent use of SetLength() for strings and arrays. Optimise initialisation with FillChar or ZeroMemory.

Local variables created on stack (e.g. record types) are faster than heap allocated (objects and New()) variables.

Reuse objects rather than Destroy then create. But make sure management code for this is faster than memory manager!

Gerry
FastMM is fast, so your management code for object reuse needs to be really fast. This is definitely something you want to profile, but it can make a huge difference when done right.
Jim McKeeth
More to consider about caching objects vs. recreating them: http://stackoverflow.com/questions/257428/
Jim McKeeth
Hence "But make sure management code for this is faster than memory manager!" - OK if it is something that can be cleared using FillChar, or using simple assignment (use an "Initialise(params)" method rather than "Create(Params)". Otherwise probably not worth it.
Gerry
+3  A: 

Check heavily-used loops for calculations that could be (at least partially) pre-calculated or handled with a lookup table. Trig functions are a classic for this, but it applies to many others.

Argalatyr
+7  A: 

If you need to use Application.processmesssages (or similar) in a loop, try calling it only every Nth iteration.

Similarly, if updating a progressbar, don't update it every iteration. Instead, increment it by x units every x iterations, or scale the updates according to time or as a percentage of overall task length.

Argalatyr
+19  A: 

.BeginUpdate;

.EndUpdate;

;)

moobaa
Mostly when updating GUI controls. ListView.Items.BeginUpdate; ... ListView.Items.EndUpdate;
Lars Truijens
+1 - it's surprising what a difference this can make when you're adding lots of things to listviews, memos etc.
robsoft
Can also help when building up SQL using qry.SQL.Add. Have a look at what happens from the TStringList.OnChange if an ADO query has the connection assigned - there can be round trips to the OLEDB layer
Gerry
BeginUpdate and endupdate won't help anymore if you've got thousands of items. Your application will hang for seconds or minutes, unless you use the TListview or TListbox in virtual mode. That's when you'll really notice a performance boost.
Wouter van Nifterick
+1  A: 

Use a lot of assertions to debug, then turn them off in shipping code.

Jim McKeeth
+1  A: 

Turn off range and overflow checking after you have tested extensively.

Jim McKeeth
+1  A: 

If you really, really, really need to be light weight then you can shed the VCL. Take a look at the KOL & MCK. Granted if you do that then you are trading features for reduced footprint.

Jim McKeeth
+1  A: 

Use the full FastMM and study the documentation and source and see if you can tweak it to your specifications.

Jim McKeeth
A: 

Take advantage of some of the FastCode project code. Parts of it were incorporated into VCL/RTL proper (like FastMM was), but there is more out there you can use!

Note, they have a new site they are moving too, but it seems to be a bit inactive.

Jim McKeeth
A: 

Maybe take advantage of the VCL FixPack by Andreas Hausladen

The VCL Fix Pack is a Delphi unit that fixes VCL and RTL bugs at runtime by patching the original functions. If you want all IDE Fix Pack fixes in your application this unit is what you need. Adding the unit to your project (Delphi and C++Builder) automatically installs the patches that are avilable for your Delphi/C++Builder version.

Jim McKeeth
The VCL Fix Pack only fixes RTL and VCL bugs. It doesn't give you more speed.
Andreas Hausladen
Anything that kills miserable bugs makes things faster, even if only the producivity. Thank you for your programs and units like VCL Fix Pack, Andreas ;-)
Fabricio Araujo
+18  A: 

Use a Delphi Profiling tool (Some here or here) and discover your own bottle necks. Optimizing the wrong bottlenecks is a waste of time. In other words, if you apply all of these suggestions here, but ignore the fact someone put a sleep(1000) (or similar) in some very important code is a waste of your time. Fix your actual bottlenecks first.

Jim McKeeth
Yeah, this is the best solution on here so far. And the best profiler I've seen is AQTime. They're offering an old version as a free download. It's not the latest, but it still works very well. http://www.download.com/AQtime/3000-18487_4-50535.html
Mason Wheeler
AQtime may not be the best for everyone. See: http://stackoverflow.com/questions/368938/delphi-profiling-tools - but thanks for the link to the free version. I'll have to see if it works with Delphi 2009.
lkessler
It does. I've already tested it.
Mason Wheeler
@Mason: It's not a free download. The old download now redirects to a trial version for 6.11.
Fabricio Araujo
+1. Changing working code for optimization's sake *without profiling* is crazy.
cjrh
+1  A: 

You might consider using runtime packages. This could reduce your memory foot print if there are more then one program running that is written using the same packages.

Jim McKeeth
A: 

Consider hardware issues. If you really need performance then consider the type of hard drive(s) your program and your databases are running on. There are a lot of variables especially if you are running a database. RAID is not always the best answer either.

Jim McKeeth
+3  A: 
  1. Create unit tests
  2. Verify tests all pass
  3. Profile your application
  4. Refactor looking for bottlenecks and memory
  5. Repeat from Step 2 (comparing to previous pass)
Jim McKeeth
+6  A: 

Consider the careful use of threads. If you are not using threads now, then consider adding a couple. If you are, make sure you are not using too many. If you are running on a Dual or Quad core computer (which most are any more) then proper thread tuning is very important.

You could look at OmniThread Library by Gabr, but there are a number of thread libraries in development for Delphi. You could easily implement your own parallel for using anonymous types.

Jim McKeeth
AsyncCalls [http://andy.jgknet.de/blog/?page_id=100] is also worth looking into from a purely functional viewpoint.
skamradt
Yes, another good one.
Jim McKeeth
+1  A: 

For an old BDE development when I first started Delphi, I was using lots of TQuery components. Someone told me to use TTable master-detail after I explained him what I was doing, and that made the program run much faster.

Calling DisableControls can omit unnecessary UI updates.

eed3si9n
+7  A: 
  1. FastMM
  2. FastCode (lib)
  3. Use high performance data structures, like hash table (etc). Many places it is faster to make one loop which makes lookup hash table for your data. Uses quite lot of memory but it surely is fast. (this maybe is most important one, but 2 first are dead simple and need very little of effort to do)
Re 3.: Don't change anything without correctly measuring the effect it has. Using memory intensive data structures may actually slow things down, if memory access patterns does continually invalidate cache lines.
mghie
+7  A: 

Before you do anything, identify slow parts. Do not touch working code which performs fast enough.

Harriv
+4  A: 

Reduce disk operations. If there's enough memory, load the file entirely to RAM and do all operations in memory.

samir105
+1 on that. Also, even if you have the whole file in memory, process it in a way (sequentially) that cache contents are preserved as much as possible. Applies for data sizes >> processor cache size, naturally.
mghie
+12  A: 

Stop using TStringList for everything.

TStringList is not a general purpose datastructure for effective storage and handling of everything from simple to complex types. Look for alternatives. I use Delphi Container and Algorithm Library (DeCAL, formerly known as SDL). Julians EZDSL should also be a good alternative.

Vegar
+1  A: 

If you use threads, set their processor affinity. If you don't use threads yet, consider using them, or look into asynchronous I/O (completion ports) if your application does lots of I/O.

mghie
+1  A: 

Consider if a DBMS database is really the perfect choice. If you are only reading data and never changing it, then a flat fixed record file could work faster, especially if the path to the data can be easily mapped (ie, one index). A trivial binary search on a fixed record file is still extremely fast.

skamradt
+2  A: 

When identifying records, use integers if at all possible for record comparison. While a primary key of "company name" might seem logical, the time spent generating and storing a hash of this will greatly improve overall search times.

skamradt
+1  A: 
  • BeginUpdate ... EndUpdate
  • ShortString vs. String
  • Use arrays instead of TStrings and TList

But the sad answer is that tuning and optimization will give you maybe 10% improvement (and it's dangerous); re-design can give you 90%. Once you really understand the goal, you often can restate the problem (and therefore the solution) in much better terms.

Cheers

Richard Haven
A: 

Examine all loops, and look for ways to short circuit. If your looking for something specific and find it in a loop, then use the BREAK command to immediately bail...no sense looping thru the rest. If you know that you don't have a match, then use a CONTINUE as quickly as possible.

skamradt
A: 

if you have to do a string comparison, use the optimized STRCOMP or TEXTCOMP functions. For simple equality, use the optimized SAMESTR and SAMETEXT functions. Always choose the SAMESTR/STRCOMP if you know the case will always be the same.

skamradt
+2  A: 

The biggest improvement came when I started using AsyncCalls to convert single-threaded applications that used to freeze up the UI, into (sort of) multi-threaded apps.

Although AsyncCalls can do a lot more, I've found it useful for this very simple purpose. Let's say you have a subroutine blocked like this: Diable Button, Do Work, Enable Button. You move the 'Do Work' part to a local function (call it AsyncDoWork), and add four lines of code:

var  a: IAsyncCall;    
a := LocalAsyncCall(@AsyncDoWork);  
while (NOT a.Finished) do 
  application.ProcessMessages;  
a.Sync;

What this does for you is run AsyncDoWork in a separate thread, while your main thread remains available to respond to the UI (like dragging the window or clicking Abort.) When AsyncDoWork is finished the code continues. Because I moved it to a local function, all local vars are available, an the code does not need to be changed.

This is a very limited type of 'multi-threading'. Specifically, it's dual threading. You must ensure that your Async function and the UI do not both access the same VCL components or data structures. (I disable all controls except the stop button.)

I don't use this to write new programs. It's just a really quick & easy way to make old programs more responsive.

Guy Gordon
Although this question doesn't really have an answer, I'm giving you the "accepted answer" because it, more than any other, is one that look like a very simple solution to many problesm and yet I hadn't heard of before. I know about threads and was cringing at getting into them. But if AsyncCalls are this simple (yes, I know about keeping them hands off the other data), then it is something that should be kept in mind by Delphi programmers. Mghie did mention this earlier, but your description is excellent.
lkessler
+2  A: 

If you have a list, use a dynamic array of anything, even a record as follows: This needs no classes, no freeing and access to it is very fast. Even if it needs to grow you can do this - see below. Only use TList or TStringList if you need lots of size changing flexibility.

type
  TMyRec = record
    SomeString : string;
    SomeValue : double;
  end;

var
  Data : array of TMyRec;
  I : integer;

..begin
  SetLength( Data, 100 ); // defines the length and CLEARS ALL DATA
  Data[32].SomeString := 'Hello';
  ShowMessage( Data[32] );

  // Grow the list by 1 item.
  I := Length( Data );
  SetLength( Data, I+1 );

..end;

Brian.

Brian Frost
A: 

Avoid using a TTable with lookup fields when a TQuery will do.

I had a report that was extremely slow in a large database. It used a TTable with a bunch of lookup fields. I hung a network monitor on my application and found that an enormous amount of data was flowing across the lines as I traversed this TTable with lookup fields. Changing to a TQuery dramatically reduced the amount of traffic and made a huge difference in speed.

This advice is really just learning to think in client-server terms.

Kluge
A: 

Run SysInternals ProcessExplorer and FileMonitor, and watch the behavior of your app from teh OS point of view. You'll find surprises such as unexpected disk and registry activity. Where you may have thought that you were saving your settings to the registry or .ini file all in one operation, you may be performing 100 writes. You may find that a database write takes 30 writes when you thought you were doing 3. Some of this can be tuned with things like transactions and buffering. But not until you find the trouble spots first. I had such an "awakening" when I put my app through U3 certification (SanDisk U3 drives have their own certification). I never did make much money by having a U3 version of my app, but the excercise was well worth it.

Chris Thornton
A: 

Avoid thread.synchronize if possible. This stops everything and waits for the VCL thread. We changed most of our synchronizes to use thread.queue where they could be done asynchronously. The use of anonymous methods helps here as well.

Mmarquee