[Erp5-dev] reindex inside a transaction?

[jp at nexedi.com]

Hi,

I am on trip for 10 days without much time to reply. But, I am nearly sure
that similar issues have been  solved in our production systems some months ago.
Part of the issue related to Zope, part to CMFActivity.I will provide more
detail when I get back to office.

Initially, we hoped that  Zope 2.12 would solve the issue. However, although
it solves part of it, it does not solve it completely.

It would be very useful that you try 5.4.6 with Zope 2.8
  http://svn.erp5.org/erp5/release/5.4.6/
and possibily with Zope 2.12
  http://svn.erp5.org/erp5/trunk/buildout/

If you do not hear from me on tuesday, do not hesitate to post an email.

Regards,

JPS.



> On Mon, 8 Nov 2010 10:16:23 +0100
> Bartek Gorny <bartek at gorny.edu.pl> wrote:
> 
> > On 6 November 2010 19:50,  <jp at nexedi.com> wrote:
> > > Hi,
> > >
> > > Could you provide the revision of SVN you are using ? Are you using
> > > ZEO environment ?
> > 
> > I'm using 5.4.2, and yes I'm using ZEO with four client instances -
> > two of them are processing request, and two are dedicated as
> > processing nodes.
> > 
> > > Some issues of this kind have been found and fixed a couple of
> > > months ago and were released in 5.4.6
> > > (http://svn.erp5.org/erp5/release/5.4.6/). If I remember well, part
> > > of them were related to Zope core and part of them to CMFActiivity.
> > > But it could also be the normal behaviour of MVCC.
> > >
> > 
> > For me it is hard to say, since I don't know what bugs were fixed - no
> > bugtracker... But the key question I'm asking is very simple: are
> > messages committed to SQL database while Zope transaction is in
> > progress? Because if they are, then things like that may happen and
> > data consistency is not guaranteed. Was it one of the issues which
> > have been fixed?
> > 
> > Bartek
> > 
> 
> Guys, are you looking at it? The issue seems serious. If you tell me
> I'm wrong, I'll be more than happy to hear it. But if I'm right, then
> we have a big big problem.
> 
> Bartek
> 
> 
> > > I can tell you more later this WE.
> > >
> > > Regards,
> > >
> > > JPS.
> > >
> > >
> > >
> > >> Hi
> > >>
> > >> I observed a very strange behaviour, which makes me rather worried.
> > >> Here is the use case:
> > >>
> > >> There is a document (an invoice) and a workflow. Upon validation of
> > >> the document the workflow script does two things:
> > >> - generates a pdf version
> > >> - changes the title of the document (because validated invoices are
> > >> numbered sequentially)
> > >>
> > >> Then the document is automatically reindexed. The problem is that
> > >> while every validated invoice has the new title when you look at
> > >> the object, the catalog sometimes contains the new value, and
> > >> sometimes the old value. Next time the doc is validated the values
> > >> in ZODB and in catalog become equal.
> > >>
> > >> All this is very strange, especially that it appears to happen
> > >> randomly and is not reproduceable. How and why it may happen?
> > >>
> > >> The only way I see is that a reindex is processed by activity tool
> > >> before the transaction is commited to the ZODB, so it uses old
> > >> version of the object. Is reindex wrote to the sql activity queue
> > >> immediately, or only after the transaction is committed? Because
> > >> if it is written immediately, then the following might happen:
> > >> - the workflow changes the object state, writes the object to ZODB
> > >> - activity tool writes the activity to the sql queue
> > >> - workflow triggers the after_scripts which begins to generate PDF,
> > >> which takes some time
> > >> - activity tool picks up the job and executes reindex
> > >> - the script returns and transaction is committed
> > >> and we have an inconsistency in data.
> > >>
> > >> Am I right, or there might be some other reason?
> > >>
> > >> Bartek
> > >>
> > >>
> > >> --
> > >> "CPU stepping is when you tile your front porch with extra CPU like
> > >> celeron 1ghz units.
> > >> typically you spread a thin layer of mortar the cpu's are laid and
> > >> then joints filled.
> > >> that's it cpu steps all done!"
> > >>
> > >> (dragonsprayer at www.tomshardware.co.uk)
> > >> _______________________________________________
> > >> Erp5-dev mailing list
> > >> Erp5-dev at erp5.org
> > >> http://mail.nexedi.com/mailman/listinfo/erp5-dev
> > >>
> > > _______________________________________________
> > > Erp5-dev mailing list
> > > Erp5-dev at erp5.org
> > > http://mail.nexedi.com/mailman/listinfo/erp5-dev
> > >
> > >
> > 
> > 
> > 
> 
> _______________________________________________
> Erp5-dev mailing list
> Erp5-dev at erp5.org
> http://mail.nexedi.com/mailman/listinfo/erp5-dev
>