[Erp5-dev] some missing methods

[Yoshinori Okuji]

On Tuesday 19 April 2005 11:16 pm, Sebastien Robin wrote:
> I have a question, may be it is a bit stupid, but I would like to
> (without list), take the output of the method on each object without
> calling mysql, then join the output and then call mysql ? we should have
> something like this (case 2):
> insert into catalog values (a,b,c);
> insert into catalog values (d,e,f);
> insert into catalog values (g,h,i);
>
> In both cases, we call mysql only one time, and in the second case we
> don't have to write a z_xxx_list zsql method.
>
> Isn't it possible to do the second case ? Is there an import perfomance
> difference ?

As you guess, the performance changes a lot.

This is an example. First, I make a dump of Nexedi ERP5 without extended 
inserts:

[yo at localhost test]$ mysqldump -u root nexedi_erp5 > standard.sql

Then, insert it into another database:

[yo at localhost test]$ time mysql test < standard.sql
0.79user 0.44system 3:12.21elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+373minor)pagefaults 0swaps

With extended inserts:

[yo at localhost test]$ mysqldump -u root -e nexedi_erp5 > extended.sql

Insert it into a database:

[yo at localhost test]$ time mysql test < extended.sql
0.20user 0.01system 0:07.77elapsed 2%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+901minor)pagefaults 0swaps

So using extended inserts is 25 times faster in this test. I heard that Coramy 
ERP5 was indexed only for 7 minutes with extended inserts, while reindexing 
it without extended inserts took several hours, according to Jean-Paul.

I think this difference is mainly because of rebuilding of indexes. If each 
row is inserted one by one, MySQL must reindex the rows every time. If a 
number of rows are inserted at one shot, MySQL does reindex much fewer times.

That's why we are interested in reindexing a document with all sub-objects, 
even when only one sub-object is modified. There is a possibility that this 
can be faster than reindexing some objects, although I suspect that this does 
not perform well, due to the overhead of the evaluation of DTMLs.

YO
-- 
Yoshinori Okuji, Nexedi Research Director
Nexedi: Consulting and Development of Free / Open Source Software
http://www.nexedi.com
ERP5: Free / Open Source ERP Software for small and medium companies
http://www.erp5.org
Storever: OpenBrick, WiFi infrastructure, notebooks and servers
http://www.storever.com