Why don't QLocalSocket/Server connections work when one process was invoked by an NSIS installer?

Question

I have an NSIS installer that installs my Qt application. At the end of the install process, the installer gives the user the option to launch the application immediately.

My application uses QLocalSocket/QLocalServer to talk to other local instances of the application. (They talk to each other basically just to ensure that there's only one instance of the app running at a time.) However, on Vista, if one of the instances was started up by the installer, then other instances cannot talk to that instance unless they were also started by the installer (or uninstaller, interestingly).

The NSIS installer launches the app with the Exec command. The client tries to connect to the server through QLocalSocket::connectToServer, which fails with the error "QLocalSocket::connectToServer: Unknown error 5".

Can anyone explain this? What's the best way to work around it?

Answer 1

Finally figured this out. The installer was running as admin (the install script said "RequestExecutionLevel admin"), and apparently it launched my app with those elevated permissions, which meant that other instances of my app running with user-level permissions couldn't connect to it. QLocalSocket/Server uses named pipes on windows, so I figure this is a windows security feature. I'm planning to work around this by using the UAC NSIS plugin, which I believe lets you run a process with user-level permissions.

Answer 2

If 5 is a windows error code, it would mean access denied. Is there a way for you to change the security on this server (You would need to access the native pipe handle)?

The finish page run option has more issues than just this, the new process gets the wrong HKCU and user profile etc.

I would recommend just disabling the run checkbox on the finish page. (This issue goes all the way back to win2000 when RunAs was added)

If you really really want this run checkbox, you can use the UAC plugin, it will allow you to start a child process as the "correct" user.