ansaurus

Question

What is the right tool to detect VMT or heap corruption in Delphi ?

Answer 1

+6 A:

You write that you want there to be an exception if

there is a write on a memory address that is not allocated by the application

but that happens anyway, both the hardware and the OS make sure of that.

If you mean you want to check for invalid memory writes in your application's allocated address range, then there is only so much you can do. You should use FastMM4, and use it with its most verbose and paranoid settings in debug mode of your application. This will catch a lot of invalid writes, accesses to already released memory and such, but it can't catch everything. Consider a dangling pointer that points to another writeable memory location (like the middle of a large string or array of float values) - writing to it will succeed, and it will trash other data, but there's no way for the memory manager to catch such access.

mghie 2009-12-10 10:41:20

Depends on your definition from allocated. The CPU/OS only give errors if memory is access that is not allocated from the OS. The OP talks about allocated memory as memory allocated from the heapmanager. There is a difference

Marco van de Voort 2009-12-10 12:41:55

Answer 2

+2 A:

mghie is right of course. (fastmm4 calls the flag fulldebugmode or something like that).

Note that that works usually with barriers just before and after an heap allocation that are regularly checked (on every heapmgr access?).

This has two consequences:

the place where fastmm detects the error might deviate from the spot where it happens
a total random write (not overflow of existing allocation) might not be detected.

So here are some other things to think about:

enable runtime checking
review all your compiler's warnings.
Try to compile with a different delphi version or FPC. Other compilers/rtls/heapmanagers have different layouts, and that could lead to the error being caught easier.

If that all yields nothing, try to simplify the application till it goes away. Then investigate the most recent commented/ifdefed parts.

Marco van de Voort 2009-12-10 12:46:34

Answer 3

+1 A:

The first thing I would do is add MadExcept to your application and get a stack traceback that prints out the exact calling tree, which will give you some idea what is going on here. Instead of a random exception and a binary/hex memory address, you need to see a calling tree, with the values of all parameters and local variables from the stack.

If I suspect memory corruption in a structure that is key to my application, I will often write extra code to make tracking this bug possible.

For example, in memory structures (class or record types) can be arranged to have a Magic1:Word at the beginning and a Magic2:Word at the end of each record in memory. An integrity check function can check the integrity of those structures by looking to see for each record Magic1 and Magic2 have not been changed from what they were set to in the constructor. The Destructor would change Magic1 and Magic2 to other values such as $FFFF.

I also would consider adding trace-logging to my application. Trace logging in delphi applications often starts with me declaring a TraceForm form, with a TMemo on there, and the TraceForm.Trace(msg:String) function starts out as "Memo1.Lines.Add(msg)". As my application matures, the trace logging facilities are the way I watch running applications for overall patterns in their behaviour, and misbehaviour. Then, when a "random" crash or memory corruption with "no explanation" happens, I have a trace log to go back through and see what has lead to this particular case.

Sometimes it is not memory corruption but simple basic errors (I forgot to check if X is assigned, then I go dereference it: X.DoSomething(...) that assumes X is assigned, but it isn't.

Warren P 2009-12-10 18:47:05

I found this post http://stackoverflow.com/questions/1106358/fastmm4-says-the-block-header-has-been-corrupted. I didn't know that FastMM can scan memorypool peridically or even after every operation. Very interesting!

Roland Bengtsson 2009-12-11 19:47:56

Answer 4

+4 A:

I don't have a solution but there are some clues about that particular error message.

System.TObject.InheritsFrom subtracts the constant vmtParent from the Self-pointer (the class) to get pointer to the adress of the parent class.

In Delphi 2007 vmtParent is defined:

vmtParent = -36;

So the error $FFFFFFDD (-35) sounds like the class pointer is 1 in this case.

Here is a test case to reproduce it:

procedure TForm1.FormCreate(Sender: TObject);
var
  I : integer;
  O : tobject;
begin
  I := 1;
  O := @I;
  O.InheritsFrom(TObject);
end;

I've tried it in Delphi 2010 and get 'Read of address FFFFFFD1' because the vmtParent is different between Delphi versions.

The problem is that this happens deep inside the Bold framework so you may have trouble guarding against it in your application code.

You can try this on your objects that are used in the DMAttracsTimers-code (which I assume is your application code):

Assert(Integer(Obj.ClassType)<>1,'Corrupt vmt');

Ville Krumlinde 2010-02-01 08:40:15

Another useful answer, but agree that it is not easy to catch the error in Bold as it is not reproducable.

Roland Bengtsson 2010-02-01 13:48:36

I believe you gave the most useful answer. Unfortunately the Assert you suggest only trigger when it is to late. The bad pointer is already assigned somewhere. Honestly I still have not a good realistic idea how to solve this...

Roland Bengtsson 2010-02-07 20:40:04

Thanks. It's hard to make any further guesses about this problem. You mention you have patched Bold and the possibility that you may have introduced bugs. Perhaps you can make a detailed compare of your patches to the original source and carefully look for bugs, add assertions to the changes etc just to be sure. Also if "RemoveDanglingLogonSessions" is often in your crash-stack then search your code for places where you create/destroy logon-sessions, perhaps you destroy an instance somewhere without removing it from your list of LogonSessions.

Ville Krumlinde 2010-02-08 08:30:03

Answer 5

+3 A:

It sounds like you have memory corruption of object instance data.

The VMT itself isn't getting corrupted, FWIW: the VMT is (normally) stored in the executable and the pages that map to it are read-only. Rather, as VilleK says, it looks like the first field of the instance data in your case got overwritten with a 32-bit integer with value 1. This is easy enough to verify: check the instance data of the object whose method call failed, and verify that the first dword is 00000001.

If it is indeed the VMT pointer in the instance data that is being corrupted, here's how I'd find the code that corrupts it:

Make sure there is an automated way to reproduce the issue that doesn't require user input. The issue may be only reproducible on a single machine without reboots between reproductions owing to how Windows may choose to lay out memory.
Reproduce the issue and note the address of the instance data whose memory is corrupted.
Rerun and check the second reproduction: make sure that the address of the instance data that was corrupted in the second run is the same as the address from the first run.
Now, step into a third run, put a 4-byte data breakpoint on the section of memory indicated by the previous two runs. The point is to break on every modification to this memory. At least one break should be the TObject.InitInstance call which fills in the VMT pointer; there may be others related to instance construction, such as in the memory allocator; and in the worst case, the relevant instance data may have been recycled memory from previous instances. To cut down on the amount of stepping needed, make the data breakpoint log the call stack, but not actually break. By checking the call stacks after the virtual call fails, you should be able to find the bad write.

Barry Kelly 2010-02-01 09:40:58

Unfortunately the case is not reproducable but you answer still add some information to the question. The error happens maybe 5 - 20 times a week in production. If there was a case that we could say it was reproducable it would be much easier (but noy easy...). One hint I saw in logs is that there is an exception before this error, see edit4 in question.

Roland Bengtsson 2010-02-01 13:45:22

I agree with Barry. I would add an FMagicFlag1:Int64 field to the top and FMagicFlag2:Int64 field to the bottom of every major class that could be corrupt, and in the constructor, set FMagicFlag1=cMagic1, FMagicFlag2=cMagic2, and Assert(FMagicFlag1=cMagic1)/Assert(FMagicFlag2=cMagic2) in some major method of that class. Watch for instance data corruption that way.

Warren P 2010-02-01 17:57:32

Answer 6

+1 A:

I Noticed that a timer is in the stack trace.
I have seen a lot of strange errors where the cause was the timer event is fired after the form i free'ed.
The reason is that a timer event cound be put on the message que, and noge get processed brfor the destruction of other components.
One way around that problem is disabling the timer as the first entry in the destroy of the form. After disabling the time call Application.processMessages, so any timer events is processed before destroying the components.
Another way is checking if the form is destroying in the timerevent. (csDestroying in componentstate).

BennyBechDk 2010-02-01 10:18:06

Benny has a good point here. Timers can be the source of all kinds of glitches. Try replacing the TTimer with a TJvThreadTimer that uses a background thread and TTHread.Synchronize instead of a WIN32 timer message!

Warren P 2010-02-01 17:58:58

I don't agree, see my edit6 change above.

Roland Bengtsson 2010-02-06 15:40:28

Answer 7

A:

Can you post the sourcecode of this procedure?

BoldSystem.TBoldMember.CalculateDerivedMemberWithExpression (BoldSystem.pas:4016)

So we can see what's happening on line 4016.

And also the CPU view of this function?
(just set a breakpoint on line 4016 of this procedure and run. And copy+paste the CPU view contents if you hit the breakpoint).
So we can see which CPU instruction is at address 00404E78.

André 2010-02-01 19:17:50

On line 4016 there is just a raise EBold.Create(s) after a try except. So normally it never reach that line except for this error. And I cannot reproduce it in test. But there are some progress, se my edit.

Roland Bengtsson 2010-02-02 11:24:28

Answer 8

A:

Could there be a problem with re-entrant code?

Try putting some guard code around the TTimer event handler code:

procedure TAttracsTimerDataModule.AttracsTimerTimer(ASender: TObject);
begin
  if FInTimer then
  begin
    // Let us know there is a problem or log it to a file, or something. 
    // Even throw an exception
    OutputDebugString('Timer called re-entrantly!'); 
    Exit; //======> 
  end;

  FInTimer := True;
  try

    // method contents

  finally
    FInTimer := False;
  end;
end;

N@

Nat 2010-02-02 00:06:18

Everything is of course possible, but I believe more to the enum theory. Se my Edit5 above.

Roland Bengtsson 2010-02-02 11:45:08

Answer 9

A:

I think there is another possibility: the timer is fired to check if there are "Dangling Logon Sessions". Then, a call is done on a TLogonSession object to check if it may be dropped (_GetMayDropSession), right? But what if the object is destroyed already? Maybe due to thread safety issues or just a .Free call and not a FreeAndNil call (so a variable is still <> nil) etc etc. In the mean time, other objects are created so the memory gets reused. If you try to acces the variable some time later, you can/will get random errors...

An example:

procedure TForm11.Button1Click(Sender: TObject);
var
  c: TComponent;
  i: Integer;
  p: pointer;
begin
  //create
  c := TComponent.Create(nil);
  //get size and memory
  i := c.InstanceSize;
  p := Pointer(c);
  //destroy component
  c.Free;
  //this call will succeed, object is gone, but memory still "valid"
  c.InheritsFrom(TObject);
  //overwrite memory
  FillChar(p, i, 1);
  //CRASH!
  c.InheritsFrom(TObject);
end;

Access violation at address 004619D9 in module 'Project10.exe'. Read of address 01010101.

André 2010-02-03 07:13:19

Answer 10

A:

Isn't the problem that "_GetMayDropSession" is referencing a freed session variable?

I have seen this kind of errors before, in TMS where objects were freed and referenced in an onchange etc (only in some situations it gave errors, very difficult/impossible to reproduce, is fixed now by TMS :-) ). Also with RemObjects sessions I got something similar (due to bad programming bug by myself).

I would try to add a dummy variable to the session class and check for it's value:

public variable iMagicNumber: integer;
constructor create: iMagicNumber := 1234567;
destructor destroy: iMagicNumber := -1;
"other procedures": assert(iMagicNumber = 1234567)

André 2010-02-08 07:31:19

ansaurus

tags:

views:

answers:

What is the right tool to detect VMT or heap corruption in Delphi ?

related questions