Tracking changes in a SQL server 2005 database

Question

I have been tasked with developing a solution that tracks changes to a database.

For updates I need to capture:

• date of update
• old value
• new value
• field effected
• person doing change
• record id
• table record is in

For deletes:

• date of delete
• person doing delete
• The title/description/id of record deleted. The tables I am tracking changes on all have a title or description field. I'd like to capture this before the record is deleted.
• table record was in

For inserts:

• date of insert
• person doing change
• record id
• table record is in

I've thought of a few ways to do this:

• I am using stored procedures for any updates/deletes/inserts. I would create a generic "tracking" table. It would have enough fields to capture all the data. I would then add another line in each stored proc to the effect of "Insert record into tracking table".
  • downside: all updates/deletes/inserts are all jumbled in the same table
  • lots of NULLed fields
  • how do I track batch updates/deletes/inserts? <---- this might not be an issue. I don't really do any thing like this in the application.
  • how do I capture the user making the update. The database just sees one account.
  • edit a lot of existing code to edit.
• Lastly, I could create a trigger that is called after updates/deletes/inserts. Many of the same downsides as the first solution except: I would have to edit as much code. I am not sure how I would track updates. It doesn't look like there's a way using triggers to see recently updated records.

I am using asp.net, C#, sql server 2005, iis6, windows 2003. I have no budget so sadly I can't buy anything to help me with this.

Thanks for your answers!

Answer 1

I hate to side-step the issue and I know you have no budget, but the simplest solution will be to upgrade to SQL Server 2008. It has this feature built in. I thought that should at least be mentioned for anyone else who comes across this question, even if you can't use it yourself.

Answer 2

A trigger wouldn't have all the information you need for a bunch of reasons - but no user id is the clincher.

I'd say you're on the right track with a common sp to insert wherever a change is made. If you're standardizing on sp's for your interfaces then you're ahead of the game - it will be hard to sneak in a change that isn't tracked.

Look at this as the equivalent of an audit trail in an accounting application - this is the Journal - a single table with every transaction recorded. They wouldn't implement separate journals for deposits, withdrawals, adjustments, etc. and this is the same principle.

Answer 3

One way I've seen this handled (though I wouldn't recommend it, honestly) is to handle it via stored procedures, passing in the userid/username/whatever as a parameter. The stored procedures would call a logging procedure, which wrote the relevant details in a central log table.

Here's where it got a bit whacky, though...

For INSERTs/UPDATEs, the relevant row(s) were stored in the table as XML data once the INSERT/UPDATE had completed successfully. For DELETEs, the row was stored prior to the DELETE running (though, realistically, they could have gotten it from the DELETE statement's output -- at least with SQL Server 2005).

If I remember correctly, the table only had a couple columns: UserID, DateTime of the logging, Transaction Type (I/U/D), XML data containing the relevant rows, table name, and primary key value (mainly used for quick searching of what records they wanted).

Many ways to skin a cat, though...

My advice is to keep is simple. Expand it out later if/when you need to.

If you have the ability to do so, lock down users to only be able to perform actionable statements on tables via stored procedures and then handle the logging (however you want) from there.

Answer 4

we built our own and just needed the user and pc passed into each add/update stored procedure. then it's just a matter of getting the original record adn populating the variables and comparing them to the passed in variables and logging the data to our table. for deletes we just have a copy of the originating tables + a timestamp field so the record is never really deleted and can be restored anytime we need (obviously the delete routine checks for FK relationships and such).

add/update log table looks like datetime, table_name, column_name, record_id, old_value, new_value, user_id, computer

we never insert nulls so we convert them to empty strings, new entries are marked with '{new entry}' in the old_value column. record_id is made up of as many key columns to uniquely identify that single record ( field1 + '.' + field2 + ... )

Answer 5

First off, in all your tables you should have at least these columns added to the data columns DateCreated, UserCreated, DateModified, UserModified. Possibly you might want to add a "Status" or "LastAction" column so that you don't ever actually delete a row you just set it to a deleted/inserted/updated status. Next you could create a "History table" which is an exact copy of the first table. Then on any updates or deletes have the trigger copy the Deleted table entries into the History table changing the DateModified, UserModified, and Status fields at the same time.

Answer 6

I've had a setup in SQL Server where we would use views to access our data, which would handle inserts, updates and deletes with INSTEAD OF triggers.

For example: an INSTEAD OF DELETE trigger on the view would mark the records in the underlying table as deleted, and the view was filtered to not show deleted records.

In all triggers, we updated a modification date and user name. The trouble is that that logs the database user name, which is not the same as the ultimate application user name.

The view needs to be schema bound for this to work.