[Erp5-dev] getSimulationState - strage behaviour on ZEO cluster

Robin Sébastien seb at nexedi.com
Mon Apr 12 15:10:42 CEST 2010 

Hi,


>
> I believe you, but it doesn't really matter if you have this issue or
> not, and how many instances you are running. It happened once, so it
> may happen again, and if we don't know why and when it happens this
> becomes very serious. I too would very much like to understand the
> problem. I don't know what to do to reproduce it.

Sure, I fully agree, we always take into account any issue very seriously
until we understand it. Any "weird" behavior is usually nothing else than
a bug. We always succeeded to find issues we saw like this (even if it can
takes many days or week) :
- think about all possible source of the problem, by looking deeply in
   all the code, included low level code
- do several hypothesis on the source of the problem
- find ways to validate or invalidate hypothesis. It is possible to
   add some logs in some particular places.
- think about ways to reproduce it and try to reproduce. This is not
   always possible, but depending on hypothesis, it is possible to
   change conditions in order to make problems happening more often
   (or not, if the hypothesis is wrong).
- until the issue is solved, we make sure that the data is consistent.
   This is very important. If we know possible consequences of a particular
   bug, then we should make sure to take all actions in order avoir problems
   on data. For instance, it is possible to use the alarm "check_catalog" available
   in the erp5_administration business template. This alarm compare the content
   of the sql catalog with the content of zodb objects and report any
   difference.

In your case, it could be interesting to know if all zeo clients are
well registered and if all ZODB invalidations are well taken into
account by all zeo clients.


Regards.

   Sébastien.

>
>>
>> Here is a few things to check :
>> - do you have anything in logs when you have the problem ?
>
> No.
>
>> - which version of zope is used (we had many startup problems with
>>    older version of zope, with zope 2.8.11 it works well) ?
>
> 2.8.11
>
>> - we had similar problems when a dict object was used instead of a
>>    PersistentMapping. In such case, the content of the dictionnary
>>    can change with the same instance, I mean you do refresh, you
>>    have content A, then you do again refresh, you have content B,
>>    and you refresh again, you can get again content A. We even had
>>    this issue when we tried to change the content of workflow history
>>    by a script and by mistake we used dict instead of PersistentMapping.
>>    The answer will be probably no, but I just ask in case, did you
>>    tried to replace the workflow history by a dictionnary ?
>
> No. This part of the site is plain-vanilla ERP5.
>
> Bartek
>
>>
>>
>> Regards.
>>
>>    Sébastien.
>>
>>
>>
>>
>> On 11/04/10 22:41, Patrick Prodöhl wrote:
>>> Hi Bartek,
>>>
>>> I have seen this kind of behaviour only in combination of ERP5 and ZEO.
>>> Anyway, I've adapted this procedure for all my ZEO setups (i.e. running
>>> Plone, etc.).
>>>
>>> Last but not least, if you have a cronjob doing an automatic restart of the
>>> single instances (i.e. once per week), you should put a "sleep 120" between
>>> each "zopectl restart" line to avoid the parallel startup of two instances
>>> at the (nearly) same time.
>>>
>>> HTH,
>>> - Patrick
>>>
>>>
>>> Am 11.04.10 22:31 schrieb "Bartek Gorny" unter<bartek at gorny.edu.pl>:
>>>
>>>> On 9 April 2010 16:47, Patrick Prodöhl<patrick.prodoehl at logica.com>    wrote:
>>>>> Hi Bartek,
>>>>>
>>>>> ">    invoices returns different results depending on the Zope instance!"
>>>>>
>>>>> I don't know if this is still the case, but some time ago when working with
>>>>> ERP5 and ZEO, you had to ensure that each zeo-instance was started
>>>>> completely before starting the next instance. Spend a look into the instance
>>>>> log (zopectl logtail) and wait for the first instance to be started
>>>>> completely before initiating the next instance.
>>>>> Maybe this solves your issue: Would be great to get some feedback if it
>>>>> worked for you or not...
>>>>
>>>> Seems to work - I restarted everything this way, and the info is
>>>> correct in every instance. Though, it may be only because of the
>>>> restart. Now I can only wait and see if it happens again - and pray it
>>>> doesn't... Anyway, it this is really the case, then the whole ZEO
>>>> thing is not as reliable as it should be. I wonder whether this is a
>>>> problem with Zope/ZEO as such, or is it only ERP5/ZEO issue?
>>>>
>>>> Bartek
>>>>
>>>>> - Patrick
>>>>>
>>>>>
>>>>>
>>>>> Am 09.04.10 16:29 schrieb "Bartek Gorny" unter<bartek at gorny.edu.pl>:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> I have a small installation of ERP5 running on ZEO - three Zope
>>>>>> instances working with the same ZEO server. Usually it works fine, but
>>>>>> now I observed a very strange behavior: getSimulationState on some
>>>>>> invoices returns different results depending on the Zope instance!
>>>>>>
>>>>>> When I check an invoice on each instance, it looks the same - it has
>>>>>> the same data. It has the same workflow history too, and the history
>>>>>> says the invoice has been stopped. But /getSimulationState returns
>>>>>> "stopped" on one instance and "draft" on two other instances. And
>>>>>> reindexing the invoice changes its state in the catalog, depending on
>>>>>> which instance the reindex is run.
>>>>>>
>>>>>> It seems very strange, since all the instances are sharing the same
>>>>>> database, so they are supposed to access exactly the same data. Any
>>>>>> clue where to look for the problem?
>>>>>>
>>>>>> Bartek
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Patrick Prodöhl | Consultant
>>>>> Nockherstraße 2-4, 81541 München | Germany
>>>>> M: +49 178 9277 942
>>>>> patrick.prodoehl at logica.com<mailto:patrick.prodoehl at logica.com>    |
>>>>> www.logica.de<http://www.logica.de/>
>>>>> Logica Deutschland GmbH&    Co. KG
>>>>> Zettachring 4, 70567 Stuttgart; Amtsgericht Stuttgart HRA 722072
>>>>> Persönlich haftender Gesellschafter: Logica Deutschland Verwaltungs GmbH
>>>>> Geschäftsführer: Torsten Straß (Vors.) | Steven Blythe | Eric Guyot | Olaf
>>>>> Scholz | Oliver Starzonek | Dr. Alexander Wurdack
>>>>> Handelsregister: AG Stuttgart HRB 724084
>>>>>
>>>>>
>>>>>
>>>>> Please help Logica to respect the environment by not printing this email  /
>>>>> Pour contribuer comme Logica au respect de l'environnement, merci de ne pas
>>>>> imprimer ce mail /  Bitte drucken Sie diese Nachricht nicht aus und helfen
>>>>> Sie so Logica dabei, die Umwelt zu schützen. /  Por favor ajude a Logica a
>>>>> respeitar o ambiente nao imprimindo este correio electronico.
>>>>>
>>>>>
>>>>>
>>>>> This e-mail and any attachment is for authorised use by the intended
>>>>> recipient(s) only. It may contain proprietary material, confidential
>>>>> information and/or be subject to legal privilege. It should not be copied,
>>>>> disclosed to, retained or used by, any other party. If you are not an
>>>>> intended recipient then please promptly delete this e-mail and any attachment
>>>>> and all copies and inform the sender. Thank you.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Erp5-dev mailing list
>>>>> Erp5-dev at erp5.org
>>>>> http://mail.nexedi.com/mailman/listinfo/erp5-dev
>>>>>
>>>>
>>>>
>>>
>>>
>>> Please help Logica to respect the environment by not printing this email  / Pour contribuer comme Logica au respect de l'environnement, merci de ne pas imprimer ce mail /  Bitte drucken Sie diese Nachricht nicht aus und helfen Sie so Logica dabei, die Umwelt zu schützen. /  Por favor ajude a Logica a respeitar o ambiente nao imprimindo este correio electronico.
>>>
>>>
>>>
>>> This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
>>>
>>>
>>> _______________________________________________
>>> Erp5-dev mailing list
>>> Erp5-dev at erp5.org
>>> http://mail.nexedi.com/mailman/listinfo/erp5-dev
>>
>> _______________________________________________
>> Erp5-dev mailing list
>> Erp5-dev at erp5.org
>> http://mail.nexedi.com/mailman/listinfo/erp5-dev
>>
>
>
>

More information about the Erp5-dev mailing list