I see two possibilities for the hang. One is that it's lower-level than WMQ and the other is that the WMQ code may have an unhandled exception. Let's look at both of these.
Assuming TCP is hanging trying to build the socket, why would it work when WMQ is up but not down? One answer is if the WMQ listener remained up after the QMgr came down. In this case, the listener accepts the socket but has nothing to hand it off to. This is common if the listener is started with runmqlsr rather than as a QMgr listener object. It is practically unavoidable if using inetd as the listener. You didn't mention the version at the QMgr side. What version of WMQ and how is the listener started? What platform is the QMgr running on?
The second possibility I'm considering is a mismatch in configuration vs. options. WMQ will perform 1-phase commit with any client install but what you are asking it to do is coordinate with Windows as the transaction controller. In client mode, that requires the Extended Transactional Client (a.k.a. XTC). The XTC component is part of the WMQ Server installation and in fact is licensed as WMQ Server. In other words, if you pay for WMQ server on your Windows host then you are entitled to install the full WMQ server and/or the XTC component there. The XTC component is what supplies the mqic32xa.dll which provides XA transactionality over WMQ client connections.
Many times people who are unaware of the licensing implications grab the XA dlls or Java/JMS XA classes and drop them onto their WMQ client installation. If the XA classes are not installed using the WMQ Server media, this can cause unpredictable results such as what you are seeing. This is especially true if the XA support files and the WMQ Client installation are at different fix pack levels or, worse, different versions.
How was the XA support installed in your Windows server? If anything less than from the WMQ Server media, you may have an invalid installation. In any case, using the latest Fix Pack would be advisable since 6.0.2.5 is quite old at this point. The fix list for v6.0.2.9 contains several .Net-related APARs including IZ54336 which reads "AMQ9456 protocol error observed on server and 2018 hconn error on client during mqconn attempt by managed .NET application."
To properly install WMQ for your application, the proscribed method is to obtain the WMQ v7.0 Server media and install WMQ client from that. Select Extended Transactional Client for the installation. Once complete, then apply the latest Fix Pack. The .Net classes have been integrated into WMQ base product in v7 and are fully supported. Using the server media makes the XA support available when installing the client. In addition to v7 being a much better implementation of .Net, v6 is end-of-life in 12 months so using v7 now will save you from a conversion later or losing support. The v7 client is compatible with a v6 QMgr but you don't get the new v7 functionality, of course.
You can do roughly the same thing from the v6 Server media but be sure to install the latest Fix Pack to get the benefit of all the .Net fixes that have been applied. Some of the new v7 functionality has also been provided in the service stream so you get that benefit as well.
You can download the WMQ Server trial at http://bit.ly/WMQTrial
You can download the latest Fix Pack at http://bit.ly/WMQFixes