It seems that OSGi is a hot term these days. Many benefits are invoked:
(etc)
I'm asking for a very specific use case - small to medium-sized web applications. What benefits would OSGi bring for those? It is actually worth it?
I would say as always "it depends".
Your environment
Consider an existing team with no OSGI experience(who proudly consider themselves as experienced developers who "get things done". There's a chance that they'll experience a major pain or a slow start.
MANY(more than you might think) developers are not familiar with build tools such as Ant or Maven, and when they are they only use limited features of those build tools.
Creating OSGI bundles is best accomplished with Eclipse, Ant tasks or Maven BND plugin VS a script or a manually written manifest for the jar archive.
Small applications
For small applications, OSGI introduces unnecessary complexity while you could use dynamic languages such as Jython, etc. or a plugin framework such as JPF or the SPI. You could also go directly with reflection and a simple custom classloader.
Big applications
Big applications might benefit from OSGI, especially when they are written from scratch. IMHO integrating OSGI in an existing application is more like introducing a patch for provide a modular architecture.
From my experience, after rewriting many applications, it's better to think about the modularity at the early days of a project.
Other concerns
Deployment : It is the same in any applications. If you're used to deploy Java Web Start applications, deployment is not a concern. If you're used to OSGI, deployment shouldn't be a concern.
There are always problems once in a while in any application when it comes to deploying it in production, which is natural.
Versioning : There are many ways to provide versioning in an application. But if you only use versioning as "information" vs as a tool(manage dependencies requirement), versioning is not a concern.
Reuse : When using OSGI you tend to write your code for reuse, but any well written API is designed with code reuse in mind.
Eclipse is a number one example of a successful big application written with OSGI. There are other big/nice tools which don't use OSGI and are modular.
Conlustion
In many modular frameworks, it's difficult to handle dependencies, stop/start/uninstall/install features at runtime, without restarting the application. You play with a custom classloader, shutdown and startup hooks, etc.
OSGI gives you such flexibility at a minor cost IMHO.
I'll hazard a NO, even though I'm a big fan of OSGi. Unless you're working with other OSGi bundles or you have a specific problem that you can't easily solve without this sledgehammer.
The benefit is elegant classpath separation (IMHO). If you need different versions of the same JAR/class, say because you're upgrading certain portions of an app while it's running, or because you are combining a lot of 3rd party modules, then OSGi is great.
That's not an easy thing to achieve, and it's not made easy with OSGi. It's made clean, but at a cost of another layer in the environment stack. And a lot of work to learn and maintain.
Not to mention the documentation isn't particularly beginner-friendly.
I suggest learning about it -- building Eclipse plugins is one very good way -- but not building it in to your dev plan until you know it well.