I would disagree with Jay - check out Kim Tripp's The Clustered Index Debate continues.
Among other things, she says that having a good primary/clustered key (on an INT IDENTITY column - NOT a GUID column) will actually speed up your inserts and deletes.
So even if you use your table only for a short period of time, it would be advisable to have a TableID INT IDENTITY(1,1) PRIMARY KEY
column to get a good, fast primary key and clustered index, and as little other indices as possible (since those will slow down inserts for sure).
Depending on how often you'll be referencing your GUID column as a foreign key, you might also want to add an index there - since it'll speed up lookups for sure.
If you're bulk-loading lots of data (says: ten of thousands of rows at once), you could also think about dropping that index before the load and recreating it once the data is loaded (which is probably going to be faster than having it in place all the time) - but again: that depends on the amount of data you're loading, and how often.
Marc