views:

993

answers:

8

Background

While running benchmark tests this morning, my colleagues and I discovered some strange things concerning performance of C# code vs. VB.NET code.

We started out comparing C# vs. Delphi Prism calculating prime numbers, and found that Prism was about 30% faster. I figured CodeGear optimized code more when generating IL (the exe was about twice as big as C#'s and had all sorts of different IL in it.)

I decided to write a test in VB.NET as well, assuming that Microsoft's compilers would end up writing essentially the same IL for each language. However, the result there was more shocking: the code ran more than three times slower on C# than VB with the same operation!

The generated IL was different, but not extremely so, and I'm not good enough at reading it to understand the differences.

Benchmarks

I've included the code for each below. On my machine, VB finds 348513 primes in about 6.36 seconds. C# finds the same number of primes in 21.76 seconds.

Computer Specs and Notes

  • Intel Core 2 Quad 6600 @ 2.4Ghz

Every machine I've tested on there is a noticeable difference in the benchmark results between C# and VB.NET.

Both of the console applications were compiled in Release mode, but otherwise no project settings were changed from the defaults generated by Visual Studio 2008.

VB.NET code

Imports System.Diagnostics

Module Module1

    Private temp As List(Of Int32)
    Private sw As Stopwatch
    Private totalSeconds As Double

    Sub Main()
        serialCalc()
    End Sub



    Private Sub serialCalc()
        temp = New List(Of Int32)()
        sw = Stopwatch.StartNew()
        For i As Int32 = 2 To 5000000
            testIfPrimeSerial(i)
        Next
        sw.Stop()
        totalSeconds = sw.Elapsed.TotalSeconds
        Console.WriteLine(String.Format("{0} seconds elapsed.", totalSeconds))
        Console.WriteLine(String.Format("{0} primes found.", temp.Count))
        Console.ReadKey()
    End Sub

    Private Sub testIfPrimeSerial(ByVal suspectPrime As Int32)
        For i As Int32 = 2 To Math.Sqrt(suspectPrime)
            If (suspectPrime Mod i = 0) Then
                Exit Sub
            End If
        Next
        temp.Add(suspectPrime)
    End Sub

End Module

C# Code

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;

namespace FindPrimesCSharp {
    class Program {
        List<Int32> temp = new List<Int32>();
        Stopwatch sw;
        double totalSeconds;


        static void Main(string[] args) {

            new Program().serialCalc();

        }


        private void serialCalc() {
            temp = new List<Int32>();
            sw = Stopwatch.StartNew();
            for (Int32 i = 2; i <= 5000000; i++) {
                testIfPrimeSerial(i);
            }
            sw.Stop();
            totalSeconds = sw.Elapsed.TotalSeconds;
            Console.WriteLine(string.Format("{0} seconds elapsed.", totalSeconds));
            Console.WriteLine(string.Format("{0} primes found.", temp.Count));
            Console.ReadKey();
        }

        private void testIfPrimeSerial(Int32 suspectPrime) {
            for (Int32 i = 2; i <= Math.Sqrt(suspectPrime); i++) {
                if (suspectPrime % i == 0)
                    return;
            }
            temp.Add(suspectPrime);
        }

    }
}

Why is C#'s execution of Math.Sqrt() slower than VB.NET?

+58  A: 

The C# implementation is recalculating Math.Sqrt(suspectPrime) each time through the loop, while VB only calculates it at the beginning of the loop. This is just due to the nature of the control structure. In C#, for is just a fancy while loop, while in VB it's a separate construct.

Using this loop will even up the score:

        Int32 sqrt = (int)Math.Sqrt(suspectPrime)
        for (Int32 i = 2; i <= sqrt; i++) { 
            if (suspectPrime % i == 0) 
                return; 
        }
Gabe
On my machine, this optimization alone brings down the time from 16.55s to 4.75s.
Alan
Agreed... my machine went from 6.99 tp 2.59 seconds. It's probably also worth pointing out that a VB Module is the same as a `static class` in C#. In this context it probably would not affect the speed much, but it is a difference.
Matthew Whited
I use vb6 and always use a variable to avoid recalculating in each iteration, are you saying it's unnecessary in vb.net and possibly vb6 too?
kjack
kjack: I don't know about VB6, but in VB.Net the terminating value is only evaluated at the beginning of the loop (according to the section 10.9.2 of the spec), so there's no need to use a temporary variable.
Gabe
@kjack BASIC and (IIRC) FORTRAN calculate `for` bounds once only. You need to check your implementation, but to do so would be trivial (have a bounds which is a function that prints something)
Pete Kirkham
This is very surprising to me. suspectPrime is a loop invariant, so shouldn't the optimizer hoist the calculation out of the loop?
Peter Ruderman
@Peter. What if you were to use `for(..)` like a `foreach(...)`. It is possible to do `for(; iterator.MoveNext();)` currently but it wouldn't be if it only evaluated the condition once. the `For` in VB.Net is a range evaluation while the `for` in C# is a conditional loop.
Matthew Whited
thanks for the info guys
kjack
Wow! This indeed makes all the difference. I had no idea that C# treats the "for" as a "while"...lesson learned! Many thanks!
Matt Winckler
Peter Ruderman: Yes, it's a loop invariant, but the optimizer needs to know that Math.Sqrt is a pure function (no side effects and always returns the same value), which may be beyond its ken.
Gabe
+6  A: 

Are these 100% comparable? It looks like the C# version is creating a new instance of a Program, whereas the VB version isn't.

mjd79
Gotta love how you were voted down. This is a correct statement. the VB Module is a static class. So the code is different.
Matthew Whited
I don't mind getting voted down - I still have a lot more karma to go before I get the free toaster - but some comments as to *why* would be nice :)
mjd79
That was my point. They down voted you probably because they thought you were wrong (which would be fine.) Fun part is they are the ones that are wrong.
Matthew Whited
Very true; good catch. I tried it both ways and it didn't make a difference, but neglected to mention it in the question.
Matt Winckler
Creating one instance of a class would have a negligible impact on performance. Therefore, while the statement in this answer is correct, it is not a correct answer. -1
Evgeny
+2  A: 

The difference is in the loop; your C# code is computing the square root on every iteration. Changing that one line from:

for (Int32 i = 2; i <= Math.Sqrt(suspectPrime); i++) {

to:

var lim = Math.Sqrt(suspectPrime);
for (Int32 i = 2; i <= lim; i++) {

dropped the time on my machine from 26 seconds to 7.something.

tzaman
+9  A: 

I agree with the statement that the C# code is computing sqrt on every iteration and here is the proof straight out of Reflector:

VB version:

private static void testIfPrimeSerial(int suspectPrime)
{
    int VB$t_i4$L0 = (int) Math.Round(Math.Sqrt((double) suspectPrime));
    for (int i = 2; i <= VB$t_i4$L0; i++)
    {
        if ((suspectPrime % i) == 0)
        {
            return;
        }
    }
    temp.Add(suspectPrime);
}

C# version:

 private void testIfPrimeSerial(int suspectPrime)
{
    for (int i = 2; i <= Math.Sqrt((double) suspectPrime); i++)
    {
        if ((suspectPrime % i) == 0)
        {
            return;
        }
    }
    this.temp.Add(suspectPrime);
}

Which kinda points to VB generating code that performs better even if the developer is naive enough to have the call to sqrt in the loop definition.

Otávio Décio
I wonder what kind of analysis they perform to make sure there are no side effects caused by the expression.
ChaosPandion
@ChaosPandion - I agree and have some uneasiness about it. For me it has not been a problem because I instinctively never put function call in the loop control.
Otávio Décio
Their for doesn't actually say it has to evaluate the expression on every run. You just say, go from `x to y` and for every cycle assign the incremented value to variable `i`. It doesn't say, I continue untill i is higher/equal to the expression. The semantics are highly different from the C# one.
Dykam
@Dykam - But why introduce such a confusing aspect to a well defined concept?
ChaosPandion
VB's for loop is actually defined to evaluate the expressions exactly once at the beginning of the loop. Using a complex expression as the terminating condition is perfectly safe because it's guaranteed to only run once.
Gabe
@ChaosPandion, I have worked with several other languages having this construct instead of the C-way.
Dykam
@Chaos @Dykam. I'm with Dykam. I imagine the VB6 loop behaviour may well be older than the C# one (I know it goes back at least to Fortran `DO` in the 1960s). You could just as easily ask why C introduced something as complicated as its `for(...;...;...)` loop. IMHO it's better to accept as a fact of life that languages have different syntax and semantics, and make sure you understand any language feature before you use it.
MarkJ
@MarkJ - Now how could I get caught up in all the C elitism? :)
ChaosPandion
+3  A: 

Off on a tangent, if you're up and running with VS2010, you can take advantage of PLINQ and make C# (probably VB.Net as well) faster.

Change that for loop to...

var range = Enumerable.Range(2, 5000000);

range.AsParallel()
    .ForAll(i => testIfPrimeSerial(i));

I went from 7.4 -> 4.6 seconds on my machine. Moving it to release mode shaves a little more time on top of that.

48klocs
A good tangent - the parallel features were actually part of the benchmarks we were running earlier, and they are impressive indeed!
Matt Winckler
How many CPUs did you run this on?
Gabe
@Gabe: I've got a pokey dual-core laptop. I don't know what make the CPU is.
48klocs
+4  A: 

Here is the decompiled IL from the for loops. If you compare the two you will see VB.Net only does the Math.Sqrt(...) onces while C# checks it on each pass. To fix this you would need to do something like var sqrt = (int)Math.Sqrt(suspectPrime); as other have suggested.

... VB ...

.method private static void CheckPrime(int32 suspectPrime) cil managed
{
    // Code size       34 (0x22)
    .maxstack  2
    .locals init ([0] int32 i,
         [1] int32 VB$t_i4$L0)
    IL_0000:  ldc.i4.2
    IL_0001:  ldarg.0
    IL_0002:  conv.r8
    IL_0003:  call       float64 [mscorlib]System.Math::Sqrt(float64)
    IL_0008:  call       float64 [mscorlib]System.Math::Round(float64)
    IL_000d:  conv.ovf.i4
    IL_000e:  stloc.1
    IL_000f:  stloc.0
    IL_0010:  br.s       IL_001d

    IL_0012:  ldarg.0
    IL_0013:  ldloc.0
    IL_0014:  rem
    IL_0015:  ldc.i4.0
    IL_0016:  bne.un.s   IL_0019

    IL_0018:  ret

    IL_0019:  ldloc.0
    IL_001a:  ldc.i4.1
    IL_001b:  add.ovf
    IL_001c:  stloc.0
    IL_001d:  ldloc.0
    IL_001e:  ldloc.1
    IL_001f:  ble.s      IL_0012

    IL_0021:  ret
} // end of method Module1::testIfPrimeSerial

... C# ...

.method private hidebysig static void CheckPrime(int32 suspectPrime) cil managed
{
    // Code size       26 (0x1a)
    .maxstack  2
    .locals init ([0] int32 i)
    IL_0000:  ldc.i4.2
    IL_0001:  stloc.0
    IL_0002:  br.s       IL_000e

    IL_0004:  ldarg.0
    IL_0005:  ldloc.0
    IL_0006:  rem
    IL_0007:  brtrue.s   IL_000a

    IL_0009:  ret

    IL_000a:  ldloc.0
    IL_000b:  ldc.i4.1
    IL_000c:  add
    IL_000d:  stloc.0
    IL_000e:  ldloc.0
    IL_000f:  conv.r8
    IL_0010:  ldarg.0
    IL_0011:  conv.r8
    IL_0012:  call       float64 [mscorlib]System.Math::Sqrt(float64)
    IL_0017:  ble.s      IL_0004

    IL_0019:  ret
} // end of method Program::testIfPrimeSerial
Matthew Whited
+2  A: 

Generally no. They both compile to CLR (Common Language Runtime) byte-code. This is similar to a JVM (Java Virtual Machine).

elsurudo
A: 

One scenario I know for certain for that C# is slower than VB.NET is in dealing with single thread apartment COM objects. The reason for this is that the thread running the entry point to a C# application enlists into an MTA by default which causes calls into the COM object to be marshalled. In contrast, the entry point thread in a VB.NET application enlists into an STA by default and no marshalling needs to take place. This, of course, can be changed in the C# application by using the STAThread attribute. This trips up a lot of people.

Brian Gideon