What should happen when a deadlock is detected

Question

Say I want to implement a database with a lock system, and I use lock avoidance and try to avoid a potential deadlock before acquiring it.

My question is:

When the session/transaction has already successfully acquired some resource A, and now it tries to acquire lock on resource B, when a deadlock is detected.

Hence, the session fails at acquiring the lock on B, should the system force the session to give up other resources the session is holding, in this case: A, and invalidate the session?

Or are there other more standard way to deal with this situation?

Answer 1

If a dead lock is detected, raise an error to indicate the programmer should alter his model.

Answer 2

SQL Server deals with deadlocks similar to in the way you're describing - it will select one of the 2 sessions as the deadlock victim and terminate / invalidate the session.

Answer 3

The usual approach to solve conflicting (deadlocked) transactions is to rollback one of the transactions and retry it later (after the other transaction has released both locks).

But if the transaction was opened in some programming language, the database can't re-run the all code from the start of the transaction; so all it can do is tell the application that there was a deadlock. The application developer has to write the code to retry the whole transaction.

Note that if the application was using something like Software Transactional Memory, the application could integrate into the database transaction and automatically rollback the application state and retry the whole transaction.