The approach here depends on whether you are doing Ajax style web pages or simple HTML, where each UI update refreshes the whole page.
The latter, a traditional page by page web site, it probably the simplest thing. For this explore JSP technologies. The idea is that you write what looks like an HTML page, but embed in it references to Java objects (or even Java code). In this case you should read up on simple frameworks such as Struts. The broad-brish idea is that you get this sequence of processing
Request arrives from Broswer, interpret it to figure out what the user wants to see
Some Java code talks to the Database gets data puts it in a Java Object
A JSP is chosen, that JSP picks items from the Java Object we just prepared
The JSP renders HTML which is sent to the Browser
In the case of Ajax, JavaScript in the Browser decides to display some data and calls a service to get it. So here, yes a "Web Service" of some kind is needed. Usually we use REST services, which return a payload in JSON format, effectively the data is transferred as JavaScript.
There are plenty of libraries for creating RESTful Web Services, for example Apache Wink.