What is the procedure for signing my code so that when user clicks on the installer it does not prompt unknown vendor.
My Setup is,
Application is java based. I wrap jar with launch4j Installer is based on nsis.
My build platform is xp.
One other thing when installation is finished i get a pop up saying installation was not succesfull from vista not nsis how can i get rid of it?
Have a look at the JarSigner. It works using the public key infrastructure so you'll need to get a ket signed by a CA somewhere which costs $$$. Both Ant and Maven have plugins to do this for you when you build your jar. This needs to be done to your jars before they are wrapped with launch4j and nsis.
I'm not familiar with nsis, but if the installer is java based, you may need to sign it's output jar as well.
I don't know how launch4j works, but I assume you are able to change the NSIS script. You probably have to add "CRCCheck off" if you are going to sign the .exe.
Adding RequestExecutionLevel (user or admin) to the nsis script should get rid of the Vista warning (Programs that are detected as installers will show this warning if no uninstall entry is added to the registry and the exe does not have a Vista manifest)
To get rid of the warning, you need to sign the setup.exe. Get the microsoft signtool.exe and a code signing certificate (make sure it is imported into your PC) then just create a batch file:
signtool sign /v /s my /n "My Display Company" /d "My Display Application" /du "My Support URL" /t "http://timestamp.verisign.com/scripts/timstamp.dll" path_to_setup.exe
SignTool.exe is installed by Visual Studio and by the Platform SDKs, just google a bit.