I'm not entirely sure if .NET works the same as plain Win32, but the way it normally works is:
Embedded resources (i.e. resource strings) in DLLs/EXEs are loaded into memory as soon as the library/app is loaded. However, if the resources are not used for a long time, they can be unloaded / paged out. Thus you don't really need to worry about running out of memory.
Having said that - HTML help isn't such a great thing to shove inside a program. If you have a lot of it, I'd really suggest storing it somewhere external to the app. That's what formats like CHM are for.
Update:
Based on your comment, I think you might want to look into satellite assemblies. This is a very common approach when you need to store large amounts of embedded resources that will be used infrequently (or not at all), i.e. for localization. They are still compiled assemblies which should hopefully fit into your plugin system.
Each plugin would effectively be two assemblies, one loaded by your main app and another declared as a reference inside the plugin. Everything is still completely self-contained, but you don't load the satellite DLL until a user actually asks for help.