[Erp5-dev] CMFActivity feature: serialization_tag

[Pelletier Vincent]

Hi.

I just (r19431) added a new feature to CMFActivity: "serialization_tag".
This can be used to address activity concurency problem with storages which 
don't detect conflict errors (ie: MySQL) with MVCC-enabled Zope (2.8 and 
higher).

This change requires re-creating activity tables, since it adds a new column.

One problem which mighthappen without using this feature when having multiple 
activity execution node is the following execution plan:

Suppose activities A and B, both reading the same object that an user can 
modify:
T+0 user modifies object
T+1 node 1 executes activity A (duration: 4)
T+2 user modifies object
T+3 node 2 executed activity B (duration 1)
T+4 ... activity B commits in MySQL
T+5 ... activity A commits in MySQL

If both activities modify the same table lines, there is a possibility that 
the final MySQL content will not be coherent with final object state.
This is because MySQL locks rows it updates in the order they are found, and 
update each one immediately. So 2 requests modifying the same rows can get 
locks in an order which is not consistent with query start order.

A concrete case of this problem is when indexing objects.
Indexations use SQLDict, which regroups indexations to lower the number of 
individual SQL queries to MySQL. So 2 invocations might have significant run 
duration differencies, exacerbating above described concurency problem.
So the final state of the catalog can be incoherent with final object state.

serialization_tag solves this by basicaly forcing activities sharing the same 
tag value to be sequentialy executed.
Technicaly, it's in fact a tiny bit different: if multiple activities with the 
same serialization tag are in -1 processing_node (validation pending) state, 
they are all distributed. This is because the order they are executed in 
doesn't matter *if* they were all waiting validation at the same time (all 
will see the same object data, and any further object modification will cause 
a new activity to come up). This is done to shorten the time needed to 
validate multiple pending activities.
Also, another technical detail about tag matching: tag is matched by MySQL 
using "LIKE" string operator. So it's possible to use SQL wildcards in such 
tag.

Of course, this modification alone does not cure the indexation problem - I'll 
fix that shortly using this new feature.

-- 
Vincent Pelletier