Although it's pegged as a 3D graphics framework for the browser, could Google O3D be used as an RIA framework similar to GWT, Flex or Rails?
Short answer:
Yes.
Long answer:
More likely it'll just be used for in-browser games. It takes a lot of work to build a decent RIA framework on top of a 3d framework, and it's unlikely such a framework (if it's ever released) will be very developer friendly.
If you want good a RIA framework look at Silverlight or Flex.
It could, but there are no user interface widgets that can draw on the O3D window. This is a classic game development problem - it is really complicated to get common UI libraries to actually render properly over top of a hardware accelerated 3d window. Even more so if you want to render windows in world space attached to objects with proper sorting. Typically, games use custom UI engines.
O3D does have a high performance Canvas for 2d drawing and it would be quite possible to write a UI library in JS that could draw on that canvas. But... that would be like re-writing Flex in JS.
The upside of that would be tight integration between 2d and 3d content - and the ability to have complex UIs both in the UI layer, and attached to objects in the 3d scene with render-to-texture tech.