You can try my example on hooking the Direct3D 9 API using C#. This utilizes EasyHook an open source .NET assembly that allows you to install hooks from managed code into unmanaged functions.
SlimDX is also used - this is an open source managed wrapper around the Direct3D libraries.
The tricky part of the hooking is determining the addresses of a COM objects' virtual functions. This is done in the above example using a small C++ helper DLL that finds the addresses from the VTable. [Update: there is a comment posted that show's how to get the function pointers from the VTable in C# also - given a IntPtr to the com object]
The example hooks the EndScene method of an IDirect3DDevice9, which is also where you would want to draw any overlays. As to displaying a form in the overlay I'm not so sure that will be so easy - I mean you can render the image easily enough, but you will have to capture inputs and manually respond/pass-thru the events to the form in question appropriately. Good luck!