[Erp5-dev] ZODB Components now in master
Arnaud Fontaine
arnaud.fontaine at nexedi.com
Wed Mar 7 11:50:31 CET 2012
Hello,
I have just merged zodb-component branch which implements bt5
Extensions, Documents and live tests into the ZODB, called Components
and accessible through Component Tool (portal_components), rather than
on the filesystem. Later on, ERP5 products will be progressively moved
to the ZODB as well.
One of the main advantage from a deployment point of view is that you no
longer have to manually synchronize bt5 Extensions, Documents and live
tests accross all ZEO clients, likewise ZODB Property Sheets.
Also, as mentioned by Jean-Paul in his previous email about ZODB
Components, you can easily define new versions (similar to git branch)
and their priorities (similar to portal skins), and switch immediately
to one version or another of Components without restarting Zope. For
example, versions could be useful to override ERP5 Components with your
project ones or to create snapshots before upgrade.
Migrating bt5 Extensions, Documents and live tests to ZODB
==========================================================
1) Only users who have Developer Role can edit Components and Component
Tool. Therefore, you must first edit your zope configuration file to
add users to Developer Role (separated by spaces). This can *only* be
done through Zope configuration file for security reasons (similar to
ClassTool with ALLOW_CLASS_TOOL file in ERP5Type product).
Here is an example of configuration:
%import Products.ERP5Type
<ERP5Type erp5>
developers zope
</ERP5Type>
2) Upgrade erp5_core and erp5_property_sheets Business Templates.
3) Once you have restarted Zope, go to the Business Template to be
migrated and add version(s) if necessary in 'Registered Version
Priority Selection' field. The versions format is 'VERSION |
PRIORITY' where PRIORITY is optional and equals to 0.0 if missing. By
default, 'erp5 | 0.0' is defined upon site creation, thus you must
not add any additional version for migrating ERP5 Components.
4) Then, you can migrate Components thanks to 'Migrate Components From
Filesystem' action defined on Business Template. Please note that
this will update Extensions (template_extension_id_list), Documents
(template_document_id_list), and Tests (template_test_id_list) in the
Business Template to match migrated Components IDs in
portal_components.
Upon export, you will get two files for each Component: one is the
metadata (ending with '.xml') and the other one is the source code
itself (ending with '.py'). Therefore, you can keep Git history while
not having source code crippled as with XML.
5) You may have to modify some imports in the migrated Components
(especially absolute imports for tests for example) to use Components
modules instead.
Notes
=====
* Like Portal Type as classes, all Components are lazily loaded and are
available in erp5.component.{document,extension,test} packages if and
only if they have been validated before. You can import them just like
usual Python modules (implemented through modules hooks as defined in
PEP 302) and also import a specific version of a Component:
'erp5.component.XXX.VERSION_version.COMPONENT'.
* When you edit an already validated Component, checkConsistency will
check whether the reference, version and source code are valid and if
not, the Component will be in modified state (e.g. when the Component
is loaded, only the validated values will be used), and will switch to
validated state automatically once reported errors have been fixed.
* Everytime a Component switches to validated or invalidated state, all
components are reset. For now this is a global reset but it may be
improved in the future if necessary.
* At any time, you can change the versions (and priorities) order by
editing version_priority_list ERP5Site property, which will
automatically trigger a reset of all Component packages.
* You can run live tests like you used to do with ClassTool through 'Run
Live Tests' action on Component Tool.
* erp5_core_component Business Template does not contain anything for
now but will be useful when the Products will be migrated to the ZODB
in the future.
Besides of that, you can just add Extensions and use ZODB Document
Component as Portal Type type_class as you used to do (available
validated ZODB Components will always be used over filesystem modules).
Editing Components
==================
You can manage Components through a web browser (portal_components) and
install erp5_ace_editor bt5 if you wish to use Ace Javascript editor to
edit components instead of a plain textarea. Soon, there will be a
setting in Preference to choose which source code editor to use (either
textarea, ace editor...), this has already been implemented but not
merged yet.
You can also edit Components through webdav, once set up in your Zope
configuration, for example:
<webdav-source-server>
address IP:PORT
force-connection-close off
</webdav-source-server>
Then, you can mount it using davfs2 package (or with your favourite
editor as long as it supports webdav). For example, with davfs2:
mount -o gid=GID,uid=UID -t davfs http://IP:PORT/ MOUNTPOINT
Test results compared to master
===============================
Here is a diff with annotations of the tests summary between master and
zodb-component branch (only test failures which happen on zodb-component
branch):
- testAccountingReports (9 failures)
+ testAccountingReports (11 failures)
+ testAccountStatementMultipleSection (testAccountingReports.TestAccountingReports)
+ testOtherPartiesReport (testAccountingReports.TestAccountingReports)
=> Not reproducible (test environment?).
- testCacheTool (1 failures)
+ testCacheTool (3 failures)
+ test_01_CacheFactoryOnePlugin (testCacheTool.TestCacheTool), Test cache factory containing only one cache plugin.
+ test_02_CacheFactoryMultiPlugins (testCacheTool.TestCacheTool), Test a cache factory containing multiple cache plugins.
=> Random failure happening on master as well (timing issue).
- testDmsWithFlare (2 failures, 2 skips)
+ testDmsWithFlare (3 failures, 2 skips)
+ test_09_SearchableText (testDmsWithFlare.TestDocumentWithFlare)
=> Not reproducible (test environment?).
+ testERP5BankingAccountingDate (1 errors)
+ test_AccountingDate (testERP5BankingAccountingDate.TestERP5BankingAccountingDate)
=> Not reproducible (test environment?).
+ testERP5Catalog (48 errors, 1 skips)
+ test_48bis_ERP5Site_hotReindexAllCheckCachedValues (testERP5Catalog.TestERP5Catalog)
+ test_50_LocalRolesArgument (testERP5Catalog.TestERP5Catalog), test local_roles= argument
+ test_50_LocalRolesArgument (testERP5Catalog.TestERP5Catalog), test local_roles= argument
+ test_51_SearchWithKeyWords (testERP5Catalog.TestERP5Catalog)
+ test_52_QueryAndTableAlias (testERP5Catalog.TestERP5Catalog)
+ test_53_DateFormat (testERP5Catalog.TestERP5Catalog)
+ test_54_FixIntUid (testERP5Catalog.TestERP5Catalog)
+ test_55_FloatFormat (testERP5Catalog.TestERP5Catalog)
+ test_56_CreateUidDuringClearCatalog (testERP5Catalog.TestERP5Catalog)
+ test_60_ViewableOwnerIndexing (testERP5Catalog.TestERP5Catalog)
+ test_BackwardCompatibilityWithOldMethods (testERP5Catalog.TestERP5Catalog)
+ test_CatalogUidDuplicates (testERP5Catalog.TestERP5Catalog)
+ test_EscapingLoginInSescurityQuery (testERP5Catalog.TestERP5Catalog)
+ test_ExactMatchSearch (testERP5Catalog.TestERP5Catalog)
+ test_IndexationContextIndependence (testERP5Catalog.TestERP5Catalog)
+ test_KeywordSearch (testERP5Catalog.TestERP5Catalog)
+ test_MonoValueAssigneeIndexing (testERP5Catalog.TestERP5Catalog)
+ test_ObjectReindexationConcurency (testERP5Catalog.TestERP5Catalog)
+ test_ParameterSelectDict (testERP5Catalog.TestERP5Catalog)
+ test_PercentCharacter (testERP5Catalog.TestERP5Catalog)
+ test_PersonDocumentWithMonovaluedLocalRole (testERP5Catalog.TestERP5Catalog), Test when user is added, which has local roles on own Person Document
+ test_QueryDictFromRequest (testERP5Catalog.TestERP5Catalog), use a dict from REQUEST as a keyword parameter.
+ test_RealOwnerIndexing (testERP5Catalog.TestERP5Catalog)
+ test_SearchFolderWithMultipleSpaces (testERP5Catalog.TestERP5Catalog)
+ test_SearchFolderWithParenthesis (testERP5Catalog.TestERP5Catalog)
+ test_SearchFolderWithRelatedDynamicRelatedKey (testERP5Catalog.TestERP5Catalog)
+ test_SearchFolderWithRelatedDynamicStrictRelatedKey (testERP5Catalog.TestERP5Catalog)
+ test_SearchFolderWithSingleQuote (testERP5Catalog.TestERP5Catalog)
+ test_SearchOnOwner (testERP5Catalog.TestERP5Catalog)
+ test_SearchedStringIsNotStripped (testERP5Catalog.TestERP5Catalog)
+ test_SelectDictWithDynamicRelatedKey (testERP5Catalog.TestERP5Catalog)
+ test_SubDocumentsSecurityIndexing (testERP5Catalog.TestERP5Catalog)
+ test_SubDocumentsWithAcquireLocalRoleSecurityIndexing (testERP5Catalog.TestERP5Catalog)
+ test_UserOrGroupLocalRoleIndexing (testERP5Catalog.TestERP5Catalog)
+ test_UserOrGroupRoleIndexing (testERP5Catalog.TestERP5Catalog)
+ test_WildcardMatchesUnsetValue (testERP5Catalog.TestERP5Catalog)
+ test_check_security_table_content (testERP5Catalog.TestERP5Catalog)
+ test_complex_query (testERP5Catalog.TestERP5Catalog)
+ test_countResultsUsesFromExpression (testERP5Catalog.TestERP5Catalog)
+ test_distinct_select_expression (testERP5Catalog.TestERP5Catalog)
+ test_getParentUid (testERP5Catalog.TestERP5Catalog)
+ test_getRecordForUid (testERP5Catalog.TestERP5Catalog)
+ test_ignore_empty_string (testERP5Catalog.TestERP5Catalog)
+ test_ignore_empty_string_related_key (testERP5Catalog.TestERP5Catalog)
+ test_multipleRelatedKeyDoMultipleJoins (testERP5Catalog.TestERP5Catalog)
+ test_path (testERP5Catalog.TestERP5Catalog)
+ test_queriesEndingWithSemicolon (testERP5Catalog.TestERP5Catalog)
+ test_sortOnRelatedKeyWithUnsetRelation (testERP5Catalog.TestERP5Catalog)
=> Random failure to be investigated but also happening on master. I
haven't had time to investigate this issue, but this is coming most
likely from test_48bis_ERP5Site_hotReindexAllCheckCachedValues which
then makes the following tests fail.
Shortcomings to be addressed soon
=================================
One shortcoming I intend to fix after the merge is to be able to get
proper tracebacks (for now, you get something similar to PythonScript as
tracebacks). Besides of that, there should not be anything else.
If you have any suggestions or comments, please let me know...
Regards,
--
Arnaud Fontaine
More information about the Erp5-dev
mailing list