[Erp5-dev] ZODB Components now in master

[Jean-Paul Smets]

Hi,

Thanks for your hard work. Please publish some doc in erp5.com site or blog.

Regards,

JPS.




On 07/03/2012 11:50, Arnaud Fontaine wrote:
> Hello,
>
> I  have   just  merged   zodb-component  branch  which   implements  bt5
> Extensions, Documents  and live tests  into the ZODB,  called Components
> and accessible  through Component Tool (portal_components),  rather than
> on the filesystem.  Later on,  ERP5 products will be progressively moved
> to the ZODB as well.
>
> One of the main advantage from a deployment point of view is that you no
> longer have to  manually synchronize bt5 Extensions,  Documents and live
> tests accross all ZEO clients, likewise ZODB Property Sheets.
>
> Also,  as  mentioned by  Jean-Paul  in  his  previous email  about  ZODB
> Components, you can  easily define new versions (similar  to git branch)
> and their priorities  (similar to portal skins),  and switch immediately
> to one  version or  another of Components  without restarting  Zope. For
> example, versions could be useful  to override ERP5 Components with your
> project ones or to create snapshots before upgrade.
>
>
> Migrating bt5 Extensions, Documents and live tests to ZODB
> ==========================================================
>
> 1) Only users who have Developer  Role can edit Components and Component
>     Tool. Therefore, you must first  edit your zope configuration file to
>     add users to Developer Role (separated by spaces). This can *only* be
>     done through Zope configuration file for security reasons (similar to
>     ClassTool with ALLOW_CLASS_TOOL file in ERP5Type product).
>
>     Here is an example of configuration:
>
> %import Products.ERP5Type
> <ERP5Type erp5>
>    developers zope
> </ERP5Type>
>
> 2) Upgrade erp5_core and erp5_property_sheets Business Templates.
>
> 3)  Once you  have restarted  Zope, go  to the  Business Template  to be
>     migrated  and  add version(s)  if  necessary  in 'Registered  Version
>     Priority  Selection'  field.   The  versions  format  is  'VERSION  |
>     PRIORITY' where PRIORITY is optional and equals to 0.0 if missing. By
>     default, 'erp5  | 0.0' is defined  upon site creation, thus  you must
>     not add any additional version for migrating ERP5 Components.
>
> 4) Then, you  can migrate Components thanks to  'Migrate Components From
>     Filesystem' action  defined on  Business Template.  Please  note that
>     this will  update Extensions  (template_extension_id_list), Documents
>     (template_document_id_list), and Tests (template_test_id_list) in the
>     Business   Template    to   match   migrated   Components    IDs   in
>     portal_components.
>
>     Upon export,  you will get two  files for each Component:  one is the
>     metadata (ending  with '.xml') and the  other one is the  source code
>     itself (ending with '.py'). Therefore, you can keep Git history while
>     not having source code crippled as with XML.
>
> 5)  You may  have  to modify  some imports  in  the migrated  Components
>     (especially absolute imports for tests for example) to use Components
>     modules instead.
>
> Notes
> =====
>
> * Like Portal Type as classes, all  Components are lazily loaded and are
>    available in erp5.component.{document,extension,test}  packages if and
>    only if they have been validated before. You can import them just like
>    usual Python modules (implemented through  modules hooks as defined in
>    PEP  302)  and  also  import   a  specific  version  of  a  Component:
>    'erp5.component.XXX.VERSION_version.COMPONENT'.
>
> * When you  edit an  already validated Component,  checkConsistency will
>    check whether the reference, version and  source code are valid and if
>    not, the Component will be in  modified state (e.g. when the Component
>    is loaded, only the validated values will be used), and will switch to
>    validated state automatically once reported errors have been fixed.
>
> * Everytime a Component switches to  validated or invalidated state, all
>    components are  reset. For now  this is a global  reset but it  may be
>    improved in the future if necessary.
>
> * At any  time, you can  change the  versions (and priorities)  order by
>    editing   version_priority_list   ERP5Site    property,   which   will
>    automatically trigger a reset of all Component packages.
>
> * You can run live tests like you used to do with ClassTool through 'Run
>    Live Tests' action on Component Tool.
>
> * erp5_core_component Business  Template does  not contain  anything for
>    now but will be useful when the  Products will be migrated to the ZODB
>    in the future.
>
> Besides  of that,  you can  just add  Extensions and  use ZODB  Document
> Component  as  Portal Type  type_class  as  you  used to  do  (available
> validated ZODB Components will always be used over filesystem modules).
>
> Editing Components
> ==================
>
> You can manage Components through  a web browser (portal_components) and
> install erp5_ace_editor bt5 if you wish  to use Ace Javascript editor to
> edit components  instead of  a plain  textarea.  Soon,  there will  be a
> setting in Preference to choose which  source code editor to use (either
> textarea,  ace editor...),  this has  already been  implemented but  not
> merged yet.
>
> You can  also edit Components through  webdav, once set up  in your Zope
> configuration, for example:
>
> <webdav-source-server>
>    address IP:PORT
>    force-connection-close off
> </webdav-source-server>
>
> Then, you  can mount  it using  davfs2 package  (or with  your favourite
> editor as long as it supports webdav). For example, with davfs2:
>
> mount -o gid=GID,uid=UID -t davfs http://IP:PORT/ MOUNTPOINT
>
> Test results compared to master
> ===============================
>
> Here is a diff with annotations  of the tests summary between master and
> zodb-component branch (only test failures which happen on zodb-component
> branch):
>
> -  testAccountingReports                              (9 failures)
> +  testAccountingReports                              (11 failures)
> +   testAccountStatementMultipleSection (testAccountingReports.TestAccountingReports)
> +   testOtherPartiesReport (testAccountingReports.TestAccountingReports)
>
> =>  Not reproducible (test environment?).
>
> -  testCacheTool                                      (1 failures)
> +  testCacheTool                                      (3 failures)
> +   test_01_CacheFactoryOnePlugin (testCacheTool.TestCacheTool), Test cache factory containing only one cache plugin.
> +   test_02_CacheFactoryMultiPlugins (testCacheTool.TestCacheTool), Test a cache factory containing multiple cache plugins.
>
> =>  Random failure happening on master as well (timing issue).
>
> -  testDmsWithFlare                                   (2 failures, 2 skips)
> +  testDmsWithFlare                                   (3 failures, 2 skips)
> +   test_09_SearchableText (testDmsWithFlare.TestDocumentWithFlare)
>
> =>  Not reproducible (test environment?).
>
> +  testERP5BankingAccountingDate                      (1 errors)
> +   test_AccountingDate (testERP5BankingAccountingDate.TestERP5BankingAccountingDate)
>
> =>  Not reproducible (test environment?).
>
> +  testERP5Catalog                                    (48 errors, 1 skips)
> +   test_48bis_ERP5Site_hotReindexAllCheckCachedValues (testERP5Catalog.TestERP5Catalog)
> +   test_50_LocalRolesArgument (testERP5Catalog.TestERP5Catalog), test local_roles= argument
> +   test_50_LocalRolesArgument (testERP5Catalog.TestERP5Catalog), test local_roles= argument
> +   test_51_SearchWithKeyWords (testERP5Catalog.TestERP5Catalog)
> +   test_52_QueryAndTableAlias (testERP5Catalog.TestERP5Catalog)
> +   test_53_DateFormat (testERP5Catalog.TestERP5Catalog)
> +   test_54_FixIntUid (testERP5Catalog.TestERP5Catalog)
> +   test_55_FloatFormat (testERP5Catalog.TestERP5Catalog)
> +   test_56_CreateUidDuringClearCatalog (testERP5Catalog.TestERP5Catalog)
> +   test_60_ViewableOwnerIndexing (testERP5Catalog.TestERP5Catalog)
> +   test_BackwardCompatibilityWithOldMethods (testERP5Catalog.TestERP5Catalog)
> +   test_CatalogUidDuplicates (testERP5Catalog.TestERP5Catalog)
> +   test_EscapingLoginInSescurityQuery (testERP5Catalog.TestERP5Catalog)
> +   test_ExactMatchSearch (testERP5Catalog.TestERP5Catalog)
> +   test_IndexationContextIndependence (testERP5Catalog.TestERP5Catalog)
> +   test_KeywordSearch (testERP5Catalog.TestERP5Catalog)
> +   test_MonoValueAssigneeIndexing (testERP5Catalog.TestERP5Catalog)
> +   test_ObjectReindexationConcurency (testERP5Catalog.TestERP5Catalog)
> +   test_ParameterSelectDict (testERP5Catalog.TestERP5Catalog)
> +   test_PercentCharacter (testERP5Catalog.TestERP5Catalog)
> +   test_PersonDocumentWithMonovaluedLocalRole (testERP5Catalog.TestERP5Catalog), Test when user is added, which has local roles on own Person Document
> +   test_QueryDictFromRequest (testERP5Catalog.TestERP5Catalog), use a dict from REQUEST as a keyword parameter.
> +   test_RealOwnerIndexing (testERP5Catalog.TestERP5Catalog)
> +   test_SearchFolderWithMultipleSpaces (testERP5Catalog.TestERP5Catalog)
> +   test_SearchFolderWithParenthesis (testERP5Catalog.TestERP5Catalog)
> +   test_SearchFolderWithRelatedDynamicRelatedKey (testERP5Catalog.TestERP5Catalog)
> +   test_SearchFolderWithRelatedDynamicStrictRelatedKey (testERP5Catalog.TestERP5Catalog)
> +   test_SearchFolderWithSingleQuote (testERP5Catalog.TestERP5Catalog)
> +   test_SearchOnOwner (testERP5Catalog.TestERP5Catalog)
> +   test_SearchedStringIsNotStripped (testERP5Catalog.TestERP5Catalog)
> +   test_SelectDictWithDynamicRelatedKey (testERP5Catalog.TestERP5Catalog)
> +   test_SubDocumentsSecurityIndexing (testERP5Catalog.TestERP5Catalog)
> +   test_SubDocumentsWithAcquireLocalRoleSecurityIndexing (testERP5Catalog.TestERP5Catalog)
> +   test_UserOrGroupLocalRoleIndexing (testERP5Catalog.TestERP5Catalog)
> +   test_UserOrGroupRoleIndexing (testERP5Catalog.TestERP5Catalog)
> +   test_WildcardMatchesUnsetValue (testERP5Catalog.TestERP5Catalog)
> +   test_check_security_table_content (testERP5Catalog.TestERP5Catalog)
> +   test_complex_query (testERP5Catalog.TestERP5Catalog)
> +   test_countResultsUsesFromExpression (testERP5Catalog.TestERP5Catalog)
> +   test_distinct_select_expression (testERP5Catalog.TestERP5Catalog)
> +   test_getParentUid (testERP5Catalog.TestERP5Catalog)
> +   test_getRecordForUid (testERP5Catalog.TestERP5Catalog)
> +   test_ignore_empty_string (testERP5Catalog.TestERP5Catalog)
> +   test_ignore_empty_string_related_key (testERP5Catalog.TestERP5Catalog)
> +   test_multipleRelatedKeyDoMultipleJoins (testERP5Catalog.TestERP5Catalog)
> +   test_path (testERP5Catalog.TestERP5Catalog)
> +   test_queriesEndingWithSemicolon (testERP5Catalog.TestERP5Catalog)
> +   test_sortOnRelatedKeyWithUnsetRelation (testERP5Catalog.TestERP5Catalog)
>
> =>  Random  failure to be  investigated but  also happening on  master. I
>     haven't had time  to investigate this issue, but this  is coming most
>     likely from  test_48bis_ERP5Site_hotReindexAllCheckCachedValues which
>     then makes the following tests fail.
>
> Shortcomings to be addressed soon
> =================================
>
> One shortcoming  I intend to fix  after the merge  is to be able  to get
> proper tracebacks (for now, you get something similar to PythonScript as
> tracebacks). Besides of that, there should not be anything else.
>
>
> If you have any suggestions or comments, please let me know...
>
> Regards,