SecurityManager is just another layer of security you can apply to Tomcat and, depending your application, can be very difficult and time consuming to get right. As you've already noted, getting this right for third party applications and libraries can be even more difficult.
In my opinion, configuring this in any detail is the last thing you should be considering.
Much of this has been said already here, but if I were you, I'd follow these steps, in order:
- Run Tomcat using jsvc under an account with the least possible privelages (by convention an account named 'tomcat', 'tomcat6' etc.). If you've installed the Ubuntu package, the init script it installed already does this, so if you start tomcat via this script, you're covered.
For more on jsvc, see http://tomcat.apache.org/tomcat-6.0-doc/setup.html
- Run jsvc using chroot. Limit what's in the virtual filesystem you specify with chroot to only what Tomcat needs.
- Secure your Tomcat, database and OS configuration. The Center for Information Security has good baseline guidance for this - http://cisecurity.org/en-us/?route=downloads.multiform.
- Application security. This is most important, but 1., 2. and 3. are so easily achieved by comparison that I suggest you do those before this (or better, in parallel). Ensure all input to your application is validated before it is used. Use something like OWASP's Enterprise Security API as a starting point - http://www.owasp.org/index.php/Category:OWASP_Enterprise_Security_API. Everything else you do is a failsafe for your application security efforts. If your application is insecure, the machine it's running on may remain secure because of the other steps you've taken (low privilege account, chroot, secure config etc.) but the data that the application manages will be compromised.
- Security Manager. Once you're happy that you have a good baseline of application security, by all means consider adding a security policy to your ongoing application security efforts.
Other things you can do include running an IDS like Snort on the machine to detect some intrusion attempts, running a file watcher like swatch to detect unexpected file modifications (especially to configuration files) and running various log analysers to attempt detect any other attempts at intrusion.
There's always a tradeoff to be made with any security effort. You can never totally secure your application and there always comes a point when further effort just isn't worth it. A full blown security manager configuration, for most people, falls into this category.
"Do I need to use the java security manager to achieve this [compromising the system itself]?"
No. There are easier ways (low privilege account, chroot, secure config etc.)