VTK and OSG really solve different problems, so which you should use depends on what you are trying to accomplish! VTK is primarily intended for scientific visualization, and as such is more geared towards data visualization algorithms. It might have some ability to work in a scene graph mode, but mainly that is so that different datasets can be registered (overlaid) on top of each other. It's not really intended for representing arbitrary scenes (though it could be shoehorned into it). And you're right, it is not intended that you mix OpenGL calls with VTK. Again, it can be done, but it's not necessarily easy.
OSG on the other hand, is the spiritual successor to SGI's Performer, and is intended more for arbitrary graphical scenes. It is very dependent on the scene-graph metaphor, which I would encourage you to read about (wikipedia has a decent article). I don't have much experience with it (being more on the sci-vis side of things).
In short, if you want to view isosurfaces of volumetric data, flow field visualizations, or other visualizations derived from scientific data, go with VTK. If you are trying to implement a virtual world of some sort, go with OSG (or something similar). If you are trying to accomplish something else, let us know, and perhaps someone can recommend a more apt tool for your needs.