[Erp5-dev] reindex inside a transaction?

[bartek]

On Mon, 8 Nov 2010 10:16:23 +0100
Bartek Gorny <bartek at gorny.edu.pl> wrote:

> On 6 November 2010 19:50,  <jp at nexedi.com> wrote:
> > Hi,
> >
> > Could you provide the revision of SVN you are using ? Are you using
> > ZEO environment ?
> 
> I'm using 5.4.2, and yes I'm using ZEO with four client instances -
> two of them are processing request, and two are dedicated as
> processing nodes.
> 
> > Some issues of this kind have been found and fixed a couple of
> > months ago and were released in 5.4.6
> > (http://svn.erp5.org/erp5/release/5.4.6/). If I remember well, part
> > of them were related to Zope core and part of them to CMFActiivity.
> > But it could also be the normal behaviour of MVCC.
> >
> 
> For me it is hard to say, since I don't know what bugs were fixed - no
> bugtracker... But the key question I'm asking is very simple: are
> messages committed to SQL database while Zope transaction is in
> progress? Because if they are, then things like that may happen and
> data consistency is not guaranteed. Was it one of the issues which
> have been fixed?
> 
> Bartek
> 

Guys, are you looking at it? The issue seems serious. If you tell me
I'm wrong, I'll be more than happy to hear it. But if I'm right, then
we have a big big problem.

Bartek


> > I can tell you more later this WE.
> >
> > Regards,
> >
> > JPS.
> >
> >
> >
> >> Hi
> >>
> >> I observed a very strange behaviour, which makes me rather worried.
> >> Here is the use case:
> >>
> >> There is a document (an invoice) and a workflow. Upon validation of
> >> the document the workflow script does two things:
> >> - generates a pdf version
> >> - changes the title of the document (because validated invoices are
> >> numbered sequentially)
> >>
> >> Then the document is automatically reindexed. The problem is that
> >> while every validated invoice has the new title when you look at
> >> the object, the catalog sometimes contains the new value, and
> >> sometimes the old value. Next time the doc is validated the values
> >> in ZODB and in catalog become equal.
> >>
> >> All this is very strange, especially that it appears to happen
> >> randomly and is not reproduceable. How and why it may happen?
> >>
> >> The only way I see is that a reindex is processed by activity tool
> >> before the transaction is commited to the ZODB, so it uses old
> >> version of the object. Is reindex wrote to the sql activity queue
> >> immediately, or only after the transaction is committed? Because
> >> if it is written immediately, then the following might happen:
> >> - the workflow changes the object state, writes the object to ZODB
> >> - activity tool writes the activity to the sql queue
> >> - workflow triggers the after_scripts which begins to generate PDF,
> >> which takes some time
> >> - activity tool picks up the job and executes reindex
> >> - the script returns and transaction is committed
> >> and we have an inconsistency in data.
> >>
> >> Am I right, or there might be some other reason?
> >>
> >> Bartek
> >>
> >>
> >> --
> >> "CPU stepping is when you tile your front porch with extra CPU like
> >> celeron 1ghz units.
> >> typically you spread a thin layer of mortar the cpu's are laid and
> >> then joints filled.
> >> that's it cpu steps all done!"
> >>
> >> (dragonsprayer at www.tomshardware.co.uk)
> >> _______________________________________________
> >> Erp5-dev mailing list
> >> Erp5-dev at erp5.org
> >> http://mail.nexedi.com/mailman/listinfo/erp5-dev
> >>
> > _______________________________________________
> > Erp5-dev mailing list
> > Erp5-dev at erp5.org
> > http://mail.nexedi.com/mailman/listinfo/erp5-dev
> >
> >
> 
> 
>