The first thing that popped into my head was to do a sync/merge with a desktop or server database, though that doesn't sound like it really fits your needs. So here is my second thought:
BULK INSERT table_name FROM data_file
I am not sure if it is supported in your version of SQL CE (though it appears to be supported in 3.5 SP1 from what I can tell on the MSDN pages). http://msdn.microsoft.com/en-us/library/ms188365.aspx
You could also disable any indexes on the table while inserting the data to speed things up, then enable/rebuild the index once you are done inserting.