I have come to the conclusion that SQL Server CE isn't a viable product. Between the versionsing issues and the impossibility of a non-administrator install it just doesn't work at my company.
Off hand I'm thinking about Jet, Oracle Berkeley, and SQLite. Given that I'm using .NET with ClickOnce, what embedded database would you recommend and why?