The CSS file and the Javascript work in tandom. If you include one without the other, you won't have "everything right first", you'll have some horrible mess whereby some of the necessary steps to make sIFR work are happening, and some are not.
That seems to be what is happening here - the Javascript dynamically flags certain divs etc. and adjusts what it can, then the CSS rules kick in on these modified entities and format them appropriately. Without the CSS you're going to have mangled entities hanging potentially all over the place - which is indeed what you're seeing.
So no problems here other than the fact you've decided to invent your own incomplete deployment procedure. ;-)