1) It looks like shi$%#$
UI matter. Consider changing Swing look'n'feel.
2) It loads up slowly
Can be caused by large file size and/or slow network. How much has the client to download before running the applet? Isn't the applet JAR file unnecessarily large? E.g. source files included? Too large images included? Too many unused classes and libraries included? Proguard may help a lot in minimizing the JAR file size.
This can also be caused by inefficient code. Is it written with memory and CPU efficiency in mind? Run a Profiler to see it and improve accordingly.
3) It hangs on some browsers occasionally even with the latest JRE
Browser matter. True, reliably deploying an applet is horrible. You may want to consider to use the Deployment Toolkit Script for this. The JavaScript will determine the browser used and its capabilities and take actions accordingly. It's as easy as following example taken from the aforelinked site:
<script src="http://www.java.com/js/deployJava.js"></script>
<script>
var attributes = {codebase:'http://java.sun.com/products/plugin/1.5.0/demos/jfc/Java2D',
code:'java2d.Java2DemoApplet.class',
archive:'Java2Demo.jar',
width:710, height:540} ;
var parameters = {fontSize:16} ;
var version = '1.6' ;
deployJava.runApplet(attributes, parameters, version);
</script>