[Erp5-dev] Delayed reindex, workflow script and acquisition

[Julien Muchembled]

Hello Bartek,


The bug you experienced is a combination of 3 known bugs, 2 of them being already fixed in trunk:

1. The first bug only exists with Zope 2.8 and a backport from Zope 2.12 was committed to ERP5/r35627
    This bug is specific to ZEO environments.

2. The second bug is also in Zope/ZODB code and is specific to ZEO environments.
    It is tested by TestInvalidationBug.testLateInvalidationFromZEO
     and a fix was committed to r39710
    We still need to discuss with Zope developers to fix ZODB.

3. The third bug affects all kinds of multi-node zopes: ZEO clusters but also single-process with several threads.
    It is tested by TestInvalidationBug.testCommitOrder
    Although this bug is specific to ERP5, a proper fix would require some changes in Zope code and we are still discussing with them. I've attached a temporary patch.

Le 11/13/10 10:20, Bartek Górny a écrit :
> On Sat, 13 Nov 2010 09:18:11 +0100
> jp at nexedi.com wrote:
>
>> Hi,
>>
>> In order to progress in a useful manner, I suggest you to do as
>> follows.
>>
>> 1- write unit test which simulates what you describe and shows the
>> problem 2- send it to the list
>>
>> This is how we did ourselves when we had similar issues (which are
>> normally fixed in recent versions).
>>
>
> This goes back to my other email - the test to show the problem would
> be like:
>
> doSomething
> doAnotherThing
> self.tic()
> check
>
> to show what happens if "another thing" is done before the activities
> go through. I don't know any other way to reproduce this problem in
> tests.

These bugs could not be tested with high-level statements like 'transaction.commit()' or 'self.tic()', especially as unit tests are single-thread.
As written above, you could look at products/ERP5/tests/testInvalidationBug.py


Regards,
Julien


>>> Hi
>>>
>>> I ran into a tricky issue caused by delayed reindexing and
>>> fast-clicking users. Here is the scenario:
>>>
>>> - there is a Document, which has some Lines
>>> - Document is a member of a Category
>>> - Lines acquire membership of a Category from their parent
>>> (Document)
>>> - he Document belongs to a Workflow
>>> - upon transition, the Workflow triggers a Script
>>> - the Script iterates over Lines doing something to which it uses
>>>    Category
>>>
>>> Now what happens is:
>>> - a user changes Category on Document and saves it
>>> - then he executes workflow transition
>>>
>>> The result is that the Script in most cases uses the new value of
>>> Category, but sometimes it uses the old value. This is because
>>> acquisition goes through portal catalog. If a users clicks through
>>> workflow transition fast enough, before the reindexing is complete
>>> (which is not uncommon), the outcome is confusion: scripts are ok,
>>> they pass all tests, user did everything right, all data is
>>> correct, only the goods went to a wrong warehouse.
>>>
>>> Now, the only way I found to prevent it is to make the Script get a
>>> category explicitly from parent (.getParent().getThisCategory()),
>>> though it is only a workaround. The question is, where is the real
>>> problem? I wrote a simple code, using legal techniques and basic
>>> API, the application passed every test, the result was bad.
>>> Therefore, there is a flaw in the core which we should nail down
>>> and fix.
>>>
>>> Bartek
>>> _______________________________________________
>>> Erp5-dev mailing list
>>> Erp5-dev at erp5.org
>>> http://mail.nexedi.com/mailman/listinfo/erp5-dev
>>>
> _______________________________________________
> Erp5-dev mailing list
> Erp5-dev at erp5.org
> http://mail.nexedi.com/mailman/listinfo/erp5-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: commit-ordering.patch
Type: text/x-diff
Size: 11864 bytes
Desc: not available
URL: <http://mail.tiolive.com/pipermail/erp5-dev/attachments/20101116/d6059023/attachment.patch>