I recommend tracking down the executable in the ClickOnce cache and double-clicking on it to see if it's a ClickOnce problem or a problem in your application.
The cache is in C:\Users\username\AppData\Local\Apps\2.0\obfuscatedfolder\obfuscatedfolder and from there you'll have to look in the folders for the one with your exe in it.
If it works, then it's a bug in your app. I'd put some messageboxes or logging in your app, or put in a try-catch around the start routine and have it show the exception and prompt the user for input in order to continue.
If it doesn't work, I would install .NET 4 on the machine and enable the enhanced ClickOnce logging and then check out the log file. (Note: The app does not have to target .NET 4, it just has to be installed on the machine.) Here's an article on how to enable enhanced logging for a ClickOnce application.