[Erp5-dev] hot reindexing

[Yoshinori Okuji]

Hello,

To support hot reindexing, many things in ZSQLCatalog were modified. 
FYI, hot reindexing is a feature that you can reindex all objects in 
the whole site on the fly without disturbing users. This will be 
documented in near future in details.

This mail specifies how to migrate to this new ZSQLCatalog. Now 
ZSQLCatalog can contain multiple catalogs at a time, and you can choose 
which catalog should be used by default. Catalogs are instances of 
SQLCatalog, so I call catalogs in ZSQLCatalog "SQL Catalogs".

In the previous implementation, ZSQLCatalog stored methods in itself. 
The current implementation moved these methods to SQL Catalogs and 
ZSQLCatalog contains SQL Catalogs instead.

For example, you can make a SQL Catalog named "erp5_mysql" in 
ZSQLCatalog, and you can cut and paste all methods to this "erp5_mysql" 
SQL Catalog. Then, you can choose the default SQL Catalog in the tab 
Properties in ZSQLCatalog.

When you view objects and modify parameters through ZSQLCatalog (or 
ERP5Catalog), ZSQLCatalog works as a proxy to the default SQL Catalog 
actually. So, say, if you modify filters in ZSQLCatalog, the result 
would be reflected to the default SQL Catalog. And, this is the same as 
modifying filters in the default SQL Catalog directly, indeed.


For other related changes:

You need to recreate tables for portal_activities, if you use 
CMFActivity. I'd recommend making a different database specific to 
portal_activities at this time. This is useful for hot reindexing, 
because activities are independent of catalogged objects (this 
description is not enough, so I will write more info afterwards). If 
you make a new database for portal_activities, please make a connection 
named "cmf_activity_sql_connection". This is compatible with the skins 
inside CMFActivity.

The way to set up initial properties for SQL Catalogs has been changed. 
Rather than writing the pairs of key/value in Python code, an XML file 
named "properties.xml" is used. This file is generated automatically 
from the ZMI (see Advanced). Because of this change, I think it is 
broken to set up CPS, since I didn't make properties.xml for CPS. 
(Sorry, I don't have CPS in my box.)

The table "catalog" is divided into two, because many columns were not 
used in ERP5. So columns used in ERP5 are left in "catalog" and the 
others were moved to a new table "compatibility". Probably ERP5 does 
not need "compatibility" at all, but it might be useful in the future, 
if ERP5 uses non-ERP5 objects inside an ERP5 Site (CPS Documents, for 
example).


How to use hot reindexing:

You can use the ZMI to start hot reindexing. You need to make a Python 
Script named "Catalog_getReindexablePathList" which returns a list of 
the paths of folders to be reindexed (e.g. ('portal_templates', 
'portal_simulation', 'accounting', )). This script can be in 
portal_catalog or portal_skins. You need to have at least two seperate 
databases (you can make any number of databases in MySQL). That's it. 
It should still have bugs, but it will work somehow.

YO
-- 
Yoshinori Okuji, Nexedi Research Director
Nexedi: Consulting and Development of Free / Open Source Software
http://www.nexedi.com
ERP5: Free / Open Source ERP Software for small and medium companies
http://www.erp5.org
Storever: OpenBrick, WiFi infrastructure, notebooks and servers
http://www.storever.com