There are a lot of factors, for example:
- Differences in the default styles (as you mentioned)
- Different set of installed fonts
- Different system settings for text smoothing
- Different settings for text size and zoom
- Varying support for standardised features (1)
- Varying support for non-standard features (2)
- Non-standard implementation of standards (bugs)
- Different strategies for dealing with errors in the code
(1) Different browsers support different standards up to a certain version, for example CSS 2.1. The standards targeted varies, and also there is often a few features missing from making the support 100% complete.
(2) Browsers support different features that are not yet standardized (or never will be). Especially Internet Explorer has a lot of non-standard features, which of course doesn't work in other browsers.