Run the code through a profiler (check both speed and memory). Once you find where it is slow (usually not where you think it is) figure out what you can do to speed it up.
Another useful thing (if you are a little brave) is to use NetBeans 7.0 M2 (don't be too panicked, their non-release version are generlaly very stable) there is a plugin called "Jackpot" which searches your code for refactorings. Some of them have to do with performance... but I don't think any of them are going to make a radical change in speed.
Generally speaking, keep the code clean and easy to read and it'll be fast. When it isn't fast you will ahve an easier time speeding it up than if it is mess.
What I did one time when I was writing something that I knew had to be fast (it was code to parse classfiles) is to run the profiler each time I made a change. So for one step I thought I would reduce memroy by calling String.,intern to make sure that all of the Strings were pooled together. When I added the intern() call the memry did go down a bit but the time went up by some huge amount (String.intern is needlessly slow, or it was a few years ago). So at that point I knew that what I had just done was unnaceptably slow and I undid that change.
I don't reccomend doing that in general development, but running the code through a profiler once a day or once a week just to see how things are isn't a productivity killer.