[Erp5-dev] reindex inside a transaction?

[Bartek Gorny]

On 6 November 2010 19:50,  <jp at nexedi.com> wrote:
> Hi,
>
> Could you provide the revision of SVN you are using ? Are you using ZEO environment ?

I'm using 5.4.2, and yes I'm using ZEO with four client instances -
two of them are processing request, and two are dedicated as
processing nodes.

> Some issues of this kind have been found and fixed a couple of months ago and were released in 5.4.6 (http://svn.erp5.org/erp5/release/5.4.6/). If I remember well, part of them were related to Zope core and part of them to CMFActiivity. But it could also be the normal behaviour of MVCC.
>

For me it is hard to say, since I don't know what bugs were fixed - no
bugtracker... But the key question I'm asking is very simple: are
messages committed to SQL database while Zope transaction is in
progress? Because if they are, then things like that may happen and
data consistency is not guaranteed. Was it one of the issues which
have been fixed?

Bartek

> I can tell you more later this WE.
>
> Regards,
>
> JPS.
>
>
>
>> Hi
>>
>> I observed a very strange behaviour, which makes me rather worried.
>> Here is the use case:
>>
>> There is a document (an invoice) and a workflow. Upon validation of
>> the document the workflow script does two things:
>> - generates a pdf version
>> - changes the title of the document (because validated invoices are
>> numbered sequentially)
>>
>> Then the document is automatically reindexed. The problem is that
>> while every validated invoice has the new title when you look at the
>> object, the catalog sometimes contains the new value, and sometimes
>> the old value. Next time the doc is validated the values in ZODB and
>> in catalog become equal.
>>
>> All this is very strange, especially that it appears to happen
>> randomly and is not reproduceable. How and why it may happen?
>>
>> The only way I see is that a reindex is processed by activity tool
>> before the transaction is commited to the ZODB, so it uses old version
>> of the object. Is reindex wrote to the sql activity queue immediately,
>> or only after the transaction is committed? Because if it is written
>> immediately, then the following might happen:
>> - the workflow changes the object state, writes the object to ZODB
>> - activity tool writes the activity to the sql queue
>> - workflow triggers the after_scripts which begins to generate PDF,
>> which takes some time
>> - activity tool picks up the job and executes reindex
>> - the script returns and transaction is committed
>> and we have an inconsistency in data.
>>
>> Am I right, or there might be some other reason?
>>
>> Bartek
>>
>>
>> --
>> "CPU stepping is when you tile your front porch with extra CPU like
>> celeron 1ghz units.
>> typically you spread a thin layer of mortar the cpu's are laid and
>> then joints filled.
>> that's it cpu steps all done!"
>>
>> (dragonsprayer at www.tomshardware.co.uk)
>> _______________________________________________
>> Erp5-dev mailing list
>> Erp5-dev at erp5.org
>> http://mail.nexedi.com/mailman/listinfo/erp5-dev
>>
> _______________________________________________
> Erp5-dev mailing list
> Erp5-dev at erp5.org
> http://mail.nexedi.com/mailman/listinfo/erp5-dev
>
>



-- 
"CPU stepping is when you tile your front porch with extra CPU like
celeron 1ghz units.
typically you spread a thin layer of mortar the cpu's are laid and
then joints filled.
that's it cpu steps all done!"

(dragonsprayer at www.tomshardware.co.uk)