How to avoid a database race condition when manually incrementing PK of new row.

Question

Hi!

I have a legacy data table in SQL Server 2005 that has a PK with no identity/autoincrement and no power to implement one.

As a result, I am forced to create new records in ASP.NET manually via the ole "SELECT MAX(id) + 1 FROM table"-before-insert technique.

Obviously this creates a race condition on the ID in the event of simultaneous inserts.

What's the best way to gracefully resolve the event of a race collision? I'm looking for VB.NET or C# code ideas along the lines of detecting a collision and then re-attempting the failed insert by getting yet another max(id) + 1. Can this be done?

Thoughts? Comments? Wisdom?

Thank you!

NOTE: What if I cannot change the database in any way?

Answer 1

Create an auxiliary table with an identity column. In a transaction insert into the aux table, retrieve the value and use it to insert in your legacy table. At this point you can even delete the row inserted in the aux table, the point is just to use it as a source of incremented values.

Answer 2

The best solution is to change the database. You may not be able to change the column to be an identity column, but you should be able to make sure there's a unique constraint on the column and add a new identity column seeded with your existing PK's. Then either use the new column instead or use a trigger to make the old column mirror the new, or both.

Answer 3

The key is to do it in one statement or one transaction.

Can you do this?

INSERT (PKcol, col2, col3, ...)
SELECT (SELECT MAX(id) + 1 FROM table WITH (HOLDLOCK, UPDLOCK)), @val2, @val3, ...

Without testing, this will probably work too:

INSERT (PKcol, col2, col3, ...)
VALUES ((SELECT MAX(id) + 1 FROM table WITH (HOLDLOCK, UPDLOCK)), @val2, @val3, ...)

If you can't, another way is to do it in a trigger.

1. The trigger is part of the INSERT transaction
2. Use HOLDLOCK, UPDLOCK for the MAX. This holds the row lock until commit
3. The row being updated is locked for the duration

A second insert will wait until the first completes. The downside is that you are changing the primary key.

An auxiliary table needs to be part of a transaction.

Or change the schema as suggested...

Answer 4

What about running the whole batch (select for id and insert) in serializable transaction?

That should get you around needing to make changes in the database.

Answer 5

Not being able to change database schema is harsh.

If you insert existing PK into table you will get SqlException with a message indicating PK constraint violation. Catch this exception and retry insert a few times until you succeed. If you find that collision rate is too high, you may try max(id) + <small-random-int> instead of max(id) + 1. Note that with this approach your ids will have gaps and the id space will be exhausted sooner.

Another possible approach is to emulate autoincrementing id outside of database. For instance, create a static integer, Interlocked.Increment it every time you need next id and use returned value. The tricky part is to initialize this static counter to good value. I would do it with Interlocked.CompareExchange:

class Autoincrement {
  static int id = -1;
  public static int NextId() {
    if (id == -1) {
      // not initialized - initialize
      int lastId = <select max(id) from db>
      Interlocked.CompareExchange(id, -1, lastId);
    }
    // get next id atomically
    return Interlocked.Increment(id);
  }
}

Obviously the latter works only if all inserted ids are obtained via Autoincrement.NextId of single process.

Answer 6

Note: All you need is a source of ever-increasing integers. It doesn't have to come from the same database, or even from a database at all.

Personally, I would use SQL Express because it is free and easy.

If you have a single web server: Create a SQL Express database on the web server with a single table [ids] with a single autoincrementing field [new_id]. Insert a record into this [ids] table, get the [new_id], and pass that onto your database layer as the PK of the table in question.

If you have multiple web servers: It's a pain to setup, but you can use the same trick by setting appropriate seed/increment (i.e. increment = 3, and seed = 1/2/3 for three web servers).

Answer 7

Is the main concern concurrent access? I mean, will multiple instances of your app (or, God forbid, other apps outside your control) be performing inserts concurrently?

If not, you can probably manage the inserts through a central, synchronized module in your app, and avoid race conditions entirely.

If so, well... like Joel said, change the database. I know you can't, but the problem is as old as the hills, and it's been solved well -- at the database level. If you want to fix it yourself, you're just going to have to loop (insert, check for collisions, delete) over and over and over again. The fundamental problem is that you can't perform a transaction (I don't mean that in the SQL "TRANSACTION" sense, but in the larger data-theory sense) if you don't have support from the database.

The only further thought I have is that if you at least have control over who has access to the database (e.g., only "authorized" apps, either written or approved by you), you could implement a side-band mutex of sorts, where a "talking stick" is shared by all the apps and ownership of the mutex is required to do an insert. That would be its own hairy ball of wax, though, as you'd have to figure out policy for dead clients, where it's hosted, configuration issues, etc. And of course a "rogue" client could do inserts without the talking stick and hose the whole setup.