I've done this in various ways in the past. I found that C++/CLI was the best option for building a bridge between .Net and unmanaged C++ (though, as noted by other posters, COM in another option).
The ways I've done this in the past include:
- Generating C++/CLI API wrapper code from my C++ API using reflection. Of course, native C++ has no reflection system, so we rolled our own. The generated C++/CLI code was then compiled into an assembly that the C# plugin would reference.
- Generating a Dynamic Assembly from my C++ API using reflection (ie, using the Reflection.Emit stuff). The resulting assembly could be used in-process for scripting languages or you could even compile C# code against it at runtime. The assembly could even be written to disk to be used statically. The downside here is that you probably can't emit better IL than a compiler, so unless you need the dynamic generation, don't go down this path.
- Hand written C++/CLI API wrappers. If the API is not very large, writing a wrapper by hand is easy enough.
In the end you will have an assembly for C# plugins to compile against. Now you need to load the plugin assemblies at runtime. To do this you must host the CLR.
Hosting the CLR is not difficult, though I've only done it once and that was a number of years ago - perhaps times have changed. If you are not comfortable with hosting the CLR, then push as muc hof your app code as possible into DLLs and then write a small C++/CLI app that brings up your unmanaged UI. Now the CLR is hosted by the small C++/CLI app.