You cannot alter the fact that Windows Mutexes are recursive. And while Posix threads are not recursive by default, you can use pthread_mutexattr_settype()
with the PTHREAD_MUTEX_RECURSIVE
flag to make one so.
Locking a mutex in Windows is actually quite an expensive operation, and best suited to inter-process synchronisation. For a mutex used only within a single process, a critical section is normally used, however these are re-entrant also. As nobugz states, you would need to use a semaphore, initialised with a maximum count of 1, to get non-recursive synchronisation.
A semaphore object is like a special counter, that can be atomically incremented and decremented across threads (or processes, if created shared). By creating one with a maximum count of 1, you get the non-recursive behaviour you require.