There are a huge number of ad-hoc plug-in systems for C#. Here is one at Code Project.
The general approach is that the host app publishes an assembly with interfaces. It enumerates through a folder and finds assemblies that define class that implement its interfaces and loads them and instantiates the classes.
In practice you want to do more. It's best if the host app defines two interfaces, an IHost and an IPlugIn. The IHost interface provides services that a plug-in can subscribe to. The IPlugIn gets constructed taking an IHost.
To load a plug-in, you should do more than simply get a plug-in. You should enumerate all plug-ins that are loadable. Construct them each. Ask them if they can run. Ask them to export APIs into the host. Ask them to import APIs from the host. Plug-ins should be able to ask about the existence of other plug-ins.
This way, plug-ins can extend the application by offering more APIs.
PlugIns should include events. This way plug-ins can monitor the process of plug-ins loading and unloading.
At the end of the world, you should warn plug-ins that they're going to go away. Then take them out.
This will leave you with an application that can be written in a tiny framework and implemented entirely in plug-ins if you want it to.
As an added bonus, you should also make it so that in the plug-ins folder, you resolve shortcuts to plug-ins. This lets you write your app and deliver it to someone else. They can author a plug-in in their development environment, create a shortcut to it in the app's plug-ins folder and not have to worry about deploying after each compile.