[Erp5-dev] Delayed reindex, workflow script and acquisition

[Leonardo Rochael Almeida]

Hi,

A small correction below:

Em Ter, 2010-11-16 às 11:34 +0100, Julien Muchembled escreveu:
> Hello Bartek,
> 
> 
> The bug you experienced is a combination of 3 known bugs, 2 of them being already fixed in trunk:
> 
> 1. The first bug only exists with Zope 2.8 and a backport from Zope 2.12 was committed to ERP5/r35627
>     This bug is specific to ZEO environments.
> 
> 2. The second bug is also in Zope/ZODB code and is specific to ZEO environments.
>     It is tested by TestInvalidationBug.testLateInvalidationFromZEO
>      and a fix was committed to r39710
>     We still need to discuss with Zope developers to fix ZODB.
> 
> 3. The third bug affects all kinds of multi-node zopes: ZEO clusters but also single-process with several threads.
>     It is tested by TestInvalidationBug.testCommitOrder
>     Although this bug is specific to ERP5, a proper fix would require
> some changes in Zope code and we are still discussing with them. I've
> attached a temporary patch.

A small correction. The changes that would be necessary in Zope code
have already landed[1] (and subsequently that part of code was removed
from Zope2 core into an external package (Products.ZSQLMethods) by Zope
the developers).

[1] http://svn.zope.org/Zope/branches/2.12/src/Shared/DC/ZRDB/TM.py?rev=113622&r1=110405&r2=113622

The only thing pending before merging to trunk the fix attached by
Julien is a resolution to an internal discussion on whether it's better
to protect against data migration by having a specific ZMySQL adapter
subclass with the right .sortKey() method (which would need a code
update and restart if the sortKey() method of ZEO and FileStorage
changed), or if we're going to protect against Zope restarts by
continuing to use a generic ZMySQL adapter for CMFActivity, and
requiring a data migration to change the .sortKey() value in case of
changes in the sort order of ZEO/FileStorage (which would only happen in
case of a code migration to a different version of ZEO/FileStorage that
would have required a restart anyway).

Regards,
Leo

> Le 11/13/10 10:20, Bartek Górny a écrit :
> > On Sat, 13 Nov 2010 09:18:11 +0100
> > jp at nexedi.com wrote:
> >
> >> Hi,
> >>
> >> In order to progress in a useful manner, I suggest you to do as
> >> follows.
> >>
> >> 1- write unit test which simulates what you describe and shows the
> >> problem 2- send it to the list
> >>
> >> This is how we did ourselves when we had similar issues (which are
> >> normally fixed in recent versions).
> >>
> >
> > This goes back to my other email - the test to show the problem would
> > be like:
> >
> > doSomething
> > doAnotherThing
> > self.tic()
> > check
> >
> > to show what happens if "another thing" is done before the activities
> > go through. I don't know any other way to reproduce this problem in
> > tests.
> 
> These bugs could not be tested with high-level statements like 'transaction.commit()' or 'self.tic()', especially as unit tests are single-thread.
> As written above, you could look at products/ERP5/tests/testInvalidationBug.py
> 
> 
> Regards,
> Julien
> 
> 
> >>> Hi
> >>>
> >>> I ran into a tricky issue caused by delayed reindexing and
> >>> fast-clicking users. Here is the scenario:
> >>>
> >>> - there is a Document, which has some Lines
> >>> - Document is a member of a Category
> >>> - Lines acquire membership of a Category from their parent
> >>> (Document)
> >>> - he Document belongs to a Workflow
> >>> - upon transition, the Workflow triggers a Script
> >>> - the Script iterates over Lines doing something to which it uses
> >>>    Category
> >>>
> >>> Now what happens is:
> >>> - a user changes Category on Document and saves it
> >>> - then he executes workflow transition
> >>>
> >>> The result is that the Script in most cases uses the new value of
> >>> Category, but sometimes it uses the old value. This is because
> >>> acquisition goes through portal catalog. If a users clicks through
> >>> workflow transition fast enough, before the reindexing is complete
> >>> (which is not uncommon), the outcome is confusion: scripts are ok,
> >>> they pass all tests, user did everything right, all data is
> >>> correct, only the goods went to a wrong warehouse.
> >>>
> >>> Now, the only way I found to prevent it is to make the Script get a
> >>> category explicitly from parent (.getParent().getThisCategory()),
> >>> though it is only a workaround. The question is, where is the real
> >>> problem? I wrote a simple code, using legal techniques and basic
> >>> API, the application passed every test, the result was bad.
> >>> Therefore, there is a flaw in the core which we should nail down
> >>> and fix.
> >>>
> >>> Bartek
> >>> _______________________________________________
> >>> Erp5-dev mailing list
> >>> Erp5-dev at erp5.org
> >>> http://mail.nexedi.com/mailman/listinfo/erp5-dev
> >>>
> > _______________________________________________
> > Erp5-dev mailing list
> > Erp5-dev at erp5.org
> > http://mail.nexedi.com/mailman/listinfo/erp5-dev
> _______________________________________________
> Erp5-dev mailing list
> Erp5-dev at erp5.org
> http://mail.nexedi.com/mailman/listinfo/erp5-dev