Optimistic Locking != Idempotency

The purpose of this essay is to highlight that optimistic locking and idempotency are two distinct concepts. I know this is not new, but I’ve seen enough online writings that confuse the two concepts and even claim that one can simply use optimistic locking to achieve idempotency.

Optimistic Locking

Optimistic locking is based on atomic Compare and Swap (CAS) primitives, which means to check if the state has changed since the beginning of the operation when committing.

Let’s use an example to quickly illustrate the idea of optimistic locking. In particular, we focus on the optimistic locking use case involving a database.

Image a use case of deducting payment. If a user has $100 balance and the item costs $80, then the final balance is $20. However, if the item costs $105, the deduction should fail.

The most common approach to optimistic locking, to my knowledge, is versioning. A simple implementation (pseudocode) is illustrated below:

SELECT balance, version FROM balance_tab WHERE uid=$uid;
If ($balance > $price) {
    $new_balance = $balance - $price 
    $version_new = $version + 1
} else {
    return “not enough balance”;
}
UPDATE balance_tab SET balance=$new_balance, version=$version_new WHERE uid=$uid AND version=$version

Note: The above implementation is free of the ABA problem.

Idempotency

Definition from wiki: Idempotence is the property of certain operations in mathematics and computer science, that can be applied multiple times without changing the result beyond the initial application.

Basically, in the context of software, a read operation is idempotent but an update is generally not. I will not go into details of how to implement idempotency, which is a big topic and fully deserves a separate article. (Of course, you can find plenty of materials online.)

Optimistic Locking != Idempotency

Even if your API utilises optimistic locking, it is not automatically idempotent. To prove this, we only need a single counter example. So let’s illustrate with the above example.

Imagine the first deduction request is processed but the client does not receive the response due to network error. It’s common that the client would retry the same request. The following shows how the data changes over time.

Task: deduct 10 from the account
Initial: balance = 100, version=10
Expect: balance = 90, version = 11
Request #1:
version=10  ⇒ version=11
balance=100 ⇒ balance=90
Request #2 (duplicate due to client retrying):
version=11 ⇒ version=12
balance=90 ⇒ balance=80

You can see that the result is semantically wrong, i.e. a double deduction problem.

It is clear in this example that even if we implement an API using optimistic locking, we must still make extra effort to make the API idempotent.

Conclusion

Optimistic locking and idempotency are two distinct concepts.

Optimistic locking is basically a locking mechanism that avoids data corruptions. If applied twice, it can possibly lead to semantically wrong data. Idempotency, on the other hand, is more like a property of a system and it can must be implemented with care.

Depending on the application, it could be the case that one can utilise optimistic locking to achieve idempotency, but I think that is more like an exception than a rule in general.