Using multi-threading to speed up .NET reflection

Question

My PC have a dual core CPU and I was wondering - would it be possible to reduce .NET reflection times by two, if I start processing in a two thread.

By "processing" I have meant the following:

1.loading the assembly 2.getting all types out of it (.GetTypes()) 3.processing these types 4.querying these types for methods etc.

If yes - what would be the best (performance-wise) strategy:

1. load all of the assemblies in a one thread then process metainfo in a two concurent threads
2. load and process each assembly in its own thread

Answer 1

Loading and processing each assembly in a seperate thread is faster. But only a bit faster. For example you will get much better milage by caching MethodInfos.

I would question the need for such an optimisation though.

results:

warm up single threaded Time Elapsed 465 ms
multi threaded Time Elapsed 173 ms
single threaded Time Elapsed 456 ms

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Reflection;
using System.Diagnostics;

namespace ConsoleApplication12 {
    class Program {


        static void TimeAction(string description, Action func) {
            var watch = new Stopwatch();
            watch.Start();
            func();
            watch.Stop();
            Console.Write(description);
            Console.WriteLine(" Time Elapsed {0} ms", watch.ElapsedMilliseconds);
        }

        static void Main(string[] args) {

            var assemblies = AppDomain.CurrentDomain.GetAssemblies();

            // warm up 
            TimeAction("warm up single threaded", () =>
            {
                foreach (var assembly in assemblies) {
                    assembly.GetTypes().Select(type => type.GetMethods()).ToArray();
                }

            });

            List<Thread> threads = new List<Thread>();

            TimeAction("multi threaded", () => {
                foreach (var assembly in assemblies) {
                    Thread t = new Thread(new ThreadStart( () => 
                        assembly.GetTypes().Select(type => type.GetMethods()).ToArray()
                        ));
                    t.Start();
                    threads.Add(t);
                }

                foreach (var thread in threads) {
                    thread.Join();
                }
            });

            TimeAction("single threaded", () =>
            {
                foreach (var assembly in assemblies) {
                    assembly.GetTypes().Select(type => type.GetMethods()).ToArray();
                }

            });

            Console.ReadKey();
        }
    }
}

Answer 2

Unless you are dealing with absolutely massive libraries, reflection loading is not a terribly slow thing, in my experiences. It's possible that it would be a waste of your time.

Nonetheless, check out ThreadPool.QueueUserWorkItem, makes this stuff easy.

foreach (Assembly a in "folder/something") {
    ThreadPool.QueueUserWorkItem((object assem) => {
        // do work
    }, a);
}

Answer 3

There are a couple of things to keep in mind:

• Starting a new thread is expensive, so if the task is short lived the overhead may be prohibitive. By using the Thread pool instead of starting your own threads, the thread pool will make sure to reuse threads to the best of its abilities. However, launching a lot of tasks in a short time doesn't yield the best result with the current thread pool implementation.

• Since your specific task involves some I/O that is probably going to be the most expensive part of the operation. Forking off several threads to wait for I/O may not yield better performance. You can't really tell until you measure it.

• If you're storing the data in a shared repository or outputting to a file/the screen, you'll have to synchronize around that. This obviously reduces the concurrency.