Non-recursive mutex
Most mutexes are (or at least should be) non-recursive. A mutex is an object which can be acquired or released atomically, which allows data which is shared between multiple threads to be protected against race conditions, data corruption, and other nasty things.
A single mutex should only ever be acquired once by a single thread within the same call chain. Attempting to acquire (or hold) the same mutex twice, within the same thread context, should be considered an invalid scenario, and should be handled appropriately (usually via an ASSERT as you are breaking a fundamental contract of your code).
Recursive mutex
This should be considered a code smell, or a hack. The only way in which a recursive mutex differs from a standard mutex is that a recursive mutex can be acquired multiple times by the same thread.
The root cause of the need for a recursive mutex is lack of ownership and no clear purpose or delineation between classes. For example, your code may call into another class, which then calls back into your class. The starting class could then try to acquire the same mutex again, and because you want to avoid a crash, you implement this as a recursive mutex.
This kind of topsy-turvy class hierarchy can lead to all sorts of headaches, and a recursive mutex provides only a band-aid solution for a more fundamental architectural problem.
Regardless of the mutex type, it should always be the same thread which acquires and releases the same mutex. The general pattern that you use in code is something like this:
Thread 1
Acquire mutex A
// Modify or read shared data
Release mutex A
Thread 2
Attempt to acquire mutex A
Block as thread 1 has mutex A
When thread 1 has released mutex A, acquire it
// Modify or read shared data
Release mutex A
It gets more complicated when you have multiple mutexes that can be acquired simultaneously (say, mutex A and B). There is the risk that you'll run into a deadlock situation like this:
Thread 1
Acquire mutex A
// Access some data...
*** Context switch to thread 2 ***
Thread 2
Acquire mutex B
// Access some data
*** Context switch to thread 1 ***
Attempt to acquire mutex B
Wait for thread 2 to release mutex B
*** Context switch to thread 2 ***
Attempt to acquire mutex A
Wait for thread 1 to release mutex A
*** DEADLOCK ***
We now have a situation where each thread is waiting for the other thread to release the other lock -- this is known as the ABBA deadlock pattern.
To prevent this situation, it is important that each thread always acquires the mutexes in the same order (e.g. always A, then B).