You can access microphones through the Sound API, but it won't give you a simple loudness level. You'll just have to capture the data and make your own decision about how loud it is.
http://download.oracle.com/javase/tutorial/sound/capturing.html
Recording implies saving the data, but here you can discard the data once you've finished determining its loudness.
The root mean squared method is a good way of calculating the amplitude of a section of wave data.
In answer to your comment, yes, you'd capture a short length of data (maybe just a few milliseconds worth) and calculate the amplitude of that. Repeat this periodically depending on how often you need updates. If you want to keep track of previous loudnesses and compare them, that's up to you - at this point it's just comparing numbers. You might use the average of recent loudnesses to calculate the ambient loudness of the room, so you can detect sudden increases in noise.
I don't know how much overhead there is in turning audio capture on and off, but you may be better off keeping the TargetDataLine
open all the time, and just calculating the loudness when you need it. While the line is open you do need to keep calling read() on it though, otherwise the application will hang waiting for you to read data.