When you say "relative to the size of the page", what exactly do you mean with "size of the page"?
The browser window? Then your fonts would change size when the user resizes the window - definitely not what anyone would expect, and pretty bad for usability. People don't resize windows to see a larger or smaller representation of the whole site, they enlarge them to see a larger section of the site, and make windows smaller to see a specific small section and have space for other windows next to the browser.
If you mean the size of the screen, that's even worse since it would mean huge fonts on a 30" screen. But people don't buy 30" screens so they can see huge fonts, they buy them to see multiple windows side by side.
All in all, using em or something similay is the only sensible way to make a scaleable website, since it will scale relative to the default size, which is/should be relative to what the user can comfortably read.