The MZ signature is a signature used by the MS-DOS relocatable 16-bit EXE format.
The reason a PE binary contains an MZ header is for backwards compatibility. If the executable is run on a DOS-based system it will run the MZ version (which is nearly always just stub that says you need to run the program on a Win32 system).
Of course this is not as useful nowadays as it was back when the world was transitioning from DOS to whatever would come after it.
Back then there were a few programs that would actually bind together a DOS version and a Win32 version in a single binary.
And as with most things dealing with Windows history, Raymond Chen has some interesting articles about this subject: