There are a lot of questions here and a load of things that you need to understand.
First of all, the actual player core logic itself will almost certainly need to be in native code to prevent the possibility of a Garbage COllection (GC). During a GC, all managed threads are suspended, and when playing video or audio, that's a problem.
So right off the bat this means that if you want your app to be in managed code (which is fine), then you're going to need a hybrid solution. This means you need to understand, and for this application undrstand quite well, how native code works and how managed code interoperates with native code.
Now if there are existing FFmpeg libraries out there (I'll take your work for it, I know nothing of FFmpeg) then the first thing you have to do is determine if they have versions compiled specifically for Windows CE. If they do not, you will have to determine how difficult it will be to make that port, and the fact you're asking this question leads me to believe that doing a port of that magnitude is going to be very difficult for your current skill set.
If they do have a build for Windows CE then you need to understand what the programming interface for it is. If it happens to be C++ classes, then you can't directly call that from managed code, so you have to write, in native code, a "shim" library that exposes straight C entry points and that will make all of the C++ allocations and calls for you.
Once you have all of that done, you can then look at the task of calling either the shim or the librariy's C interface from managed code via Platform Invoke.
Basically you have a lot of work to do and there isn't any definitive article I can point you to that will walk you from one end to the other. Here are a few links to get you started though. If you fully understand every one of these, then you will probably have the proper knowledge to write a wrapper around an existing implementation (not necessarily to write the player core implementation itself).