If you don't like to mess with temporary files, I'd advise to use C++/CLI.
Create a C++/CLI dll project in visual studio. Create one static managed class, and define the functions as you want to use them from C#:
public ref class JpegTools
{
public:
static array<byte>^ Optimize(array<byte>^ input)
};
These functions you define can be directly called from C#, and you can implement them with all that C++ offers.
array^ corresponds to a C# byte array. You'll need to use pin_ptr<> to pin the byte array in memory, so you can pass on the data to the unmanaged Jpeg helper function of your choice. C++/CLI has ample support for marshalling managed types to native types. You can also allocate new array with gc_new, to return CLI compatible types. If you have to marshall strings from C# to C++ as part of this excercise, use Mfc/Atl's CString type.
You can statically link all the jpeg code into the dll. A C++ dll can be mixed pure native and C++/CLI code. In our C++/CLI projects, typically only the interface source files know about CLI types, all the rest work with with C++ types.
There's some overhead work to get going this way, but the upside is that your code is compile-time typechecked, and all the dealings with unmanged code and memory are dealt with on the C++ side. It actually works so well that I used C++/CLI to unit test native C++ code almost directly with NUnit.
Good luck!