Some screenshots would be much more useful. What do you man by ' didn't look quite like they should have'? Look and Feel, layout, or something else?
The only less painful advantage of the OS you can take is to set GTKLookAndFell which will make your application look almost like installed GTK theme.
Then try SWT.
If you want a step further, and make your program depend on gnome, write your app using Java Gnome bindings. This will integrate better into Gnome, maybe even allow you to use global menu (not sure about this one), normal open/save dialogs and notifications.
Anyways, explain what you really want to achieve.