If quick startup is your goal, Java on a PC may not be your best bet. It's going to take a few seconds because that's how long it takes to load the VM from disk.
If you want your app to start more quickly it's easy to get a splash screen up, just create a module that only loads your splash screen, waits for it to fully display then uses reflection to link to your "Real" main module.
(Use reflection because otherwise it will pull in your entire program through references before it starts the main one--at least that's how it used to work).
If you're talking about run-time performance, you won't get quicker by changing languages, Java's about as fast as you can get. You MIGHT be able to get a boost by converting to C/C++ and rewriting it to suit those platforms (Less OO, stack allocations instead of heap, etc), but otherwise none of the other languages in general usage are close to Java in speed.
If you really need the quick startup, depending on what you are doing there may be some tricks. I've seen projects that try to keep a Java VM running in your toolbar and allow you to make requests (tell it to start an app). This was faster but made additional requirements of the user (Loading this additional tool)
Another possibility--if you are constantly starting up/shutting down small tasks and that's the reason the startup bothers you then you can definitely speed it up by keeping it running invisibly. Just have your Java app open a socket and listen for commands then create a little .EXE or shell script that can start your program if it's not running or send commands to that socket if it is. This would completely eliminate startups after the first run.