Handles being released corrupted somehow?

Question

Strange problem. Maybe someone can give some insight.

• Scenario 1. I have a TBitmap in memory that is written to while complex calculations take place to calculate the color of each pixel. Every so often (usually after every horizontal line that the bitmap is filled) the TBitmap is drawn to an image on a form (image1.Canvas.Draw(0, 0, TBitmap). Most of the time this works fine, but I noticed that if there are many slow complex calcs for each bitmap line (say taking more than 30 secs or a minute to calculate) then the main form has a momentary "flicker" that somehow erases the bitmap, so then the image.draw call only draws the latest calculated line and the first y lines are blanked out in the bitmap. I got around this by locking the bitmap prior to calculations.

• Scenario 2. This is the main hassle. I am writing to a TMemoryStream rather than a bitmap. Same deal. Calculations take place to calculate each pixel value and then each pixel value is written to the TMemoryStream with memstream.Write(bytevalue, 1) during the process. At the end of all the calculations I save the stream to a bitmap with memstream.SaveToFile('whatever.bmp') and then free the stream with memstream.Free. If the calculation is quick then the stream saves no matter what size (i am doing tests with 10000x10000 dimensions).

I can even tell the resulting file will be corrupt as the main application window/form does have a slight flicker like it is being repainted. When that happens it is as if every handle to bitmaps and TMemoryStream is killed/refreshed so the existing data is corrupted.

Any ideas? This really sucks. Especially when each single image can take an hour to create only to find that when it finishes something in the background happened and corrupted the bitmap or TMemoryStream.

Is there any way I can lock a TMemoryStream handle like I can with a bitmap? That may help. Or some declaration to tell Delphi "Don't mess with my objects, even if it looks like the app is taking too long"

Or does anyone know the back end reason inside Delphi that causes this to happen.

The TMemoryStream is created inside the procedure that does all the calculations so is a local object. With the bitmap issue the bitmap was a global variable outside the procedure and it happened, so I don't think that is the cause.

This is also under Windows 7, but I noticed the original bitmap issue under Vista.

Update 1:

Sorry for not using the comments, but there is the restirction on text size...

In reply to Remy (and anyone else reading this)...

Single threaded. For the memorystream it works fine for 5000x5000 resolution if the calculations are quick, but fails if the cals are slow.

As a basic framework the code is along the lines of

SetupMemorystream; 
for y:=0 to height do 
   for x:=0 to width do 
      DoCalcs;
      SetByteValue; 
   end; 
end; 
SaveStream;

If the DoCalcs is relatively speedy then all goes to plan. If it is slow then I get the TMemoryStream corruption and the resulting bitmap the stream is saved to is corrupt.

This was identical with using an in memory TBitmap until I discovered I could lock the bitmap which stops Delphi and/or Windows reallocating a new handle to it "when it wants to" which corrupts the data inside the bitmap.

This is too much of a coincidence to not think the same issue is not happening with the TMemoryStream and its handle.

Update 2:

One more maybe helpful bit of info.

When the TMemoryStream saves OK the resulting file (for a 5000x5000 bitmap) is 75,000,054 bytes in size.

When the saved stream is corrupt it seems to be a random value (of the size from when the handle was corrupted until the stream is saved). Example sizes have been 22 MB and 9 MB.

When I look at the resulting files is a hex editor it shows that the start of the files are correct with the header chunks, but the tail ends get truncated somehow.

This is so bizarre. Anyway I can absolutely for sure flush a TMemoryStream after a SaveToFile call and before freeing it?

Answer 1

It is hard to find such an error by just assuming what is happening. Because the operation is so lenghtly it is expensive to just repeat and wait if error will be found.

I suggest you log every step of the process. You will get an enormous log probably, but it will show you where things go wrong. Repeat the process and improve your log, this way you will slowly but surelly find the cause of the problem.

Answer 2

When you free a filestream used for write, the call to close the file is not checked for errors. So your write can fail, perhaps on flushing out the last blocks of your large file, and no exception will be raised.

I got hit by this recently : it's in QC 80148. However there is not a lot you can do about it, because the Windows CloseHandle function is unlikely to return any error either.

Answer 3

1. Before writing each byte to memory stream, set the Capacity to approximated size of bit stream so that it does not re-size memory often. This will speed thing up

2. I think you have to subtract 1 from Height and Width on for loop

Cheers

Answer 4

You can use the API call FlushFileBuffers(filestream.Handle) to flush a tFileStream, but my guess is that something else is happening to cause corruption in the first place, which could be as simple as your loop going from 0 to width rather than 1 to width or 0 to width-1...its hard to say without analyzing what your routines are doing to populate your memory stream.

Answer 5

Thanks for all the hints guys. The loops did have correct 0 to width-1 in code.

Problem was the variables for width and height were globals that got changed elsewhere in the application when the main form was resized (they usually keep track of the onscreen displayed image, but I had used the same variables inside this procedure to track the memorystream bitmap width and height. So when they got changed outside the loops screwed up and corrupted the output. What a bugger of a problem to track down. Once I localized the width and height variables everything works as expected.

I should have known it was my error and not a Delphi issue.