Reflection & Application Design

Question

Hello,

Let me describe the problem and the solution i currently have and maybe you can help enlighten me on why it's a bad idea (assuming it is) and what i can do to make it a better system.

right now i have 600 "rippers" that parse files and rip them to csv or other formats. the rippers all implement a common interface and base class. At the moment when a job is queued up depending on the configuration of that job a specific ripper is called using reflection.

            foreach (var stream in streams)
            {
                try
                {
                    // load it
                    Assembly asm = Assembly.LoadFile(Path.Combine(stream.RipperPath, stream.RipperFileName), new Evidence());
                    if (asm == null)
                    {
                        MessageBox.Show(String.Format("Invalid Interface ripper loaded.\n{0}", Path.Combine(stream.RipperPath, stream.RipperFileName)));
                        return;
                    }

                    foreach (Type objType in asm.GetTypes())
                    {
                        if (!objType.IsPublic)
                            continue;

                        // make sure the type isn't Abstract
                        if (((objType.Attributes & TypeAttributes.Abstract) == TypeAttributes.Abstract))
                            continue;

                        // IRipper is the interface that all of the Rippers must implement to be loaded
                        Type objInterface = objType.GetInterface("IRipper", true);
                        if (objInterface == null)
                            continue;

                        try
                        {
                            var iri = (IRipper)Activator.CreateInstance(objType);

                            // Rippers must register with their hosts
                            iri.Host = this;
                            iri.OnStart += RipperStart;
                            iri.OnComplete += RipperComplete;
                            iri.OnProgressChanged += RipperStatusUpdate;
                            iri.dataStream = stream;
                        }
                        catch (Exception ex)
                        {
                            Console.WriteLine(ex.Message);
                        }
                    }
                }
                catch (Exception ex)
                {
                    MessageBox.Show(String.Format("Error loading interface: {0}\n{1}", Path.Combine(stream.RipperPath, stream.RipperFileName), ex.Message));
                }
            }

all rippers have implement a function called "Rip()" which the interface contracts them to.

using the current code the only problem i can see is that after loading 600 assemblies (since they're loaded when they are needed) it's going to start getting a little slow if they are not unloaded after they have been used.

What would you suggest?

Answer 1

Do you actually have 600 different rippers, or 600 streams and some smaller number of rippers? Do you really need each ripper to be in its own assembly? If you could bunch the rippers together you could have far fewer assemblies. If your stream details contained the type name as well as the executable, you could have multiple rippers and you wouldn't need to go looking through the assembly for them - just call Assembly.GetType.

I don't know how much overhead there really is per assembly, mind you - I'd expect 600 assemblies to take up a fair amount of memory, but not affect performance much beyond that.

The overall approach seems reasonable, although you might also want to look at the Managed Extensibility Framework (MEF) - it may be overkill for your case, but worth a look. The other thing I'd do is factor out the "getting a ripper" from the "processing a stream" code :)

Answer 2

You may want to take a look at the System.Addin stuff. It appears that add-ins are loaded in their own app domains, which makes it possible to unload them cleanly when they are no longer needed.