Is there any (performance) difference between Debug and Release?

Question

I'm using MySql Connector .NET to load an account and transfer it to the client. This operation is rather intensive, considering the child elements of the account to load.

In Debug mode, it takes, at most, 1 second to load the account. The average would be 500ms. In Release mode, it takes from 1 to 4 seconds to load the account. The average would be 1500ms.

Since there is no #if DEBUG directive or the like in my code, I'm wondering where the difference is coming from.

Is there a project build option I could change? Or does it have to do with MySql Connector .NET that would have different behaviors depending on the build mode?

EDIT: Ticks monitoring.

Debug (Average: 213000 ticks)
730000
320000
60000
50000
190000
130000
210000
180000
160000
110000
390000
270000
150000
190000
230000
210000
150000
200000
190000
140000

Release (Average: 4404500 ticks)
12940000
170000
180000
80000
80000
130000
120000
5060000
5090000
130000
50000
10430000
25160000
150000
160000
130000
17620000
10160000
100000
150000

Comparison:

Release takes 20x the time Debug takes (average comparison).

4,404,500 / 213,000 = 20

Now the first operation is indeed longer, but in general, so are all the other times for release. Any idea?

EDIT 2: I added even a broader tests that calculates the total time. For 50 account loads, it takes an average of 4 seconds in debugs, and 40 seconds in release. I'm starting to get quite desperate over this - it is a serious performance issue for my application. Does anyone have a guess on how to fix this?

Answer 1

It's possible that the difference in timings is due to a change in when assemblies required for your operation are loaded.

In release mode, the runtime may not need to immediately load an assembly that is only later required by your operation (because of various optimizations performed for release builds). Consequently, in debug mode an assembly may get loaded before you start timing your operation, and in release mode that assembly may get loaded after you start timing your operation. The time to load the assembly could be significant depending on how large the assembly is. Of course the assembly has to be loaded in both cases, and only has to be loaded once, so subsequent runs in release mode may be faster.

Try performing your operation several times within a loop and ignoring the first execution to find the average less start up overhead.

Update: Interesting that the timings in release mode vary a lot compared to those in debug mode (the std dev is 100x higher for release mode). On the lower end, the release mode timings are comparable to those in debug mode. You mention in your question that loading an account is intensive because of all of the child elements that have to load. Another difference could be the point at which the runtime decides to perform garbage collection. To test, you could try performing System.GC.Collect() after each operation (outside of your timer) and see if that changes things.

Update: If you suspect there may be a change in behavior with respect to locking, you might consider using the Windows Performance Monitor to monitor the various .NET CLR LocksAndThreads counters for your application's process(es) while you run your tests in both debug and release modes. Perhaps you are not properly releasing a lock somewhere and execution is delayed until some timeout lapses? If so, I would expect to see an increase in contention rate reported by the performance counters. I'm not sure why this would only be a problem for release builds (unless you're actually using the debugger when running debug builds).

Answer 2

Everything on the Build and Debug tabs in the application property settings can change depending on the build configuration. Some of these refer only to the compilation stage and will not affect runtime performance (Allow unsafe code, Errors and warnings, Treat warnings as errors, and XML documenation file). The others could make a difference.

I would take note of each setting that is different between the configurations, then change each one so that the configurations match, testing between each change. Then you should be able to find the source of the problem.

I would especially test Define DEBUG constant, Define TRACE constant, Conditional compilation symbols, Platform target, Optimize code, (on the Advanced screen) Check for arithmetic overflow/underflow, Generate serialization assembly, Enable unmanaged code debugging, and Enable the Visual Studio hosting process.

Answer 3

I figured it out, I allowed unsafe code in one of my dependencies build. I am still wondering why it is behaving like that, but I'll have to dig this a bit more.

Thanks for all your help!