Metadata-Version: 1.1
Name: Products.csvreplicata
Version: 1.1.11
Summary: CSV import/export for Archetypes or other contents (via plugins for the later). Created by Makina Corpus.
Home-page: https://svn.plone.org/svn/collective/Products.csvreplicata
Author: Eric BREHAULT
Author-email: eric.brehault@makina-corpus.org
License: GPL
Description: Introduction
        ============
        
        Allows to export and import AT objects contents and hierarchy.
        
        It supports also other contents or objects via plugins (ZCA).
        This part  mainly targeted for a developper audiance, so see interfaces.py / adapters.py.
        
        
        We have plans to integrate the export/import code with transmogrifier_, [ also_ ] but we have no ETA For now.
        
        .. _transmogrifier: http://pypi.python.org/pypi/collective.transmogrifier/
        .. _also: http://pypi.python.org/pypi/plone.app.transmogrifier/
        
        
        Repository: git_
        
        
        .. _git: https://github.com/collective/Products.csvreplicata.git
        
        A Makina Corpus add-on
        -------------------------
        |makinacom|_
        
        * `Planet Makina Corpus <http://www.makina-corpus.org>`_
        * `Contact us <mailto:python@makina-corpus.org>`_
        
        .. |makinacom| image:: http://depot.makina-corpus.org/public/logo.gif
        .. _makinacom:  http://www.makina-corpus.com
        
        
        Products.csvreplicata Installation
        ======================================
        
        To install Products.csvreplicata into the global Python environment (or a workingenv),
        using a traditional Zope 2 instance, you can do this:
        
         * When you're reading this you have probably already run
           ``easy_install Products.csvreplicata``. Find out how to install setuptools
           (and EasyInstall) here:
           http://peak.telecommunity.com/DevCenter/EasyInstall
        
         * If you are using Zope 2.9 (not 2.10), get `pythonproducts`_ and install it
           via::
        
               python setup.py install --home /path/to/instance
        
           into your Zope instance.
        
         * Create a file called ``Products.csvreplicata-configure.zcml`` in the
           ``/path/to/instance/etc/package-includes`` directory.  The file
           should only contain this::
        
               <include package="Products.csvreplicata" />
        
        .. _pythonproducts: http://plone.org/products/pythonproducts
        
        
        Alternatively, if you are using zc.buildout and the plone.recipe.zope2instance
        recipe to manage your project, you can do this:
        
         * Add ``Products.csvreplicata`` to the list of eggs to install, e.g.
        
        ::
        
            [buildout]
            ...
            eggs =
                ...
                Products.csvreplicata
        
          * Tell the plone.recipe.zope2instance recipe to install a ZCML slug
        
        ::
        
            [instance]
            recipe = plone.recipe.zope2instance
            ...
            zcml =
                Products.csvreplicata
        
          * Re-run buildout, e.g. with:
        
            $ ./bin/buildout
        
        You can skip the ZCML slug if you are going to explicitly include the package
        from another package's configure.zcml file.
        
        
        Because its top level Python namespace package is called ``Products``, this
        package can also be installed in Zope 2 as an old style **Zope 2 Product**.
        
        For that, move (or symlink) the ``csvreplicata`` folder of this project
        (``Products.csvreplicata/Products/csvreplicata``) into the ``Products`` directory of
        the Zope instance it has to be installed for, and restart the server.
        
        You can also skip the ZCML slug if you install this package the **Zope 2
        Product** way.
        
        Detailled documentation
        =======================
        
        
        Interfaces
        -------------
        import interfaces and classes ::
        
            >>> from zope.interface.verify import verifyClass
            >>> from zope.interface import implements
            >>> from Products.csvreplicata.handlers.base import CSVdefault
            >>> from Products.csvreplicata.handlers.file import CSVFile
            >>> from Products.csvreplicata.interfaces import ICSVDefault, ICSVFile
        
        Verify implementation ::
        
            >>> verifyClass(ICSVDefault, CSVdefault)
            True
            >>> verifyClass(ICSVFile, CSVFile)
            True
        
        
        
        Export / Import in plain format
        ------------------------------------
        
        Export
        +++++++++
        here we export folders and documents
        ::
        
            >>> self.setRoles(['Manager'])
            >>> id=self.folder.invokeFactory('Document'      , id='doc1'   , title="Document 1")
            >>> id=self.folder.invokeFactory('Document'      , id='doc2'   , title="Document 2")
            >>> id=self.folder.invokeFactory('News Item'     , id='news1'  , title="news 'super' 3")
            >>> id=self.folder.invokeFactory('Document'      , id='doc4'   , title="Document 4")
            >>> id=self.folder.invokeFactory('Folder'        , id='sub1'   , title="mytest")
            >>> id=self.folder.sub1.invokeFactory('Document' , id='doc11'  , title="Document 1 du dossier 1")
            >>> id=self.folder.sub1.invokeFactory('News Item', id='news21' , title="news 1 du dossier 1")
            >>> id=self.folder.sub1.invokeFactory('Document' , id='doc31'  , title="Document 2 du dossier 1")
            >>> self.portal.portal_csvreplicatatool.getPlainFormat()
            False
        
            >>> self.portal.portal_csvreplicatatool.getEncoding()
            'UTF-8'
            >>> self.portal.portal_csvreplicatatool.getDelimiter()
            ';'
            >>> self.portal.portal_csvreplicatatool.getStringdelimiter()
            '"'
            >>> self.portal.portal_csvreplicatatool.replicabletypes =  {'Document': ['default'], 'Folder': ['default'], 'News': ['Default']}
            >>> from Products.csvreplicata.interfaces import Icsvreplicata
            >>> repl = Icsvreplicata(self.folder)
            >>> print repl.csvexport(exportable_content_types=['Document', 'Folder', 'News Item'], depth=0).getvalue()
            "/plone/Members/test_user_1_";"..."
            "parent";"id";"type";"title";"description";"text";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "Parent folder";"Identifier";"Content type";"Title";"label_summary";"label_body_text";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "";"doc1";"Document";"Document 1";"";"";"simple_publication_workflow";"private";"";""
            "";"doc2";"Document";"Document 2";"";"";"simple_publication_workflow";"private";"";""
            "parent";"id";"type";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "Parent folder";"Identifier";"Content type";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "";"news1";"News Item";"simple_publication_workflow";"private"
            "parent";"id";"type";"title";"description";"text";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "Parent folder";"Identifier";"Content type";"Title";"label_summary";"label_body_text";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "";"doc4";"Document";"Document 4";"";"";"simple_publication_workflow";"private"
            "parent";"id";"type";"title";"description";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "Parent folder";"Identifier";"Content type";"Title";"label_description";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "";"sub1";"Folder";"mytest";"";"simple_publication_workflow";"private"
            "parent";"id";"type";"title";"description";"text";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "Parent folder";"Identifier";"Content type";"Title";"label_summary";"label_body_text";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "sub1";"doc11";"Document";"Document 1 du dossier 1";"";"";"simple_publication_workflow";"private"
            "parent";"id";"type";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "Parent folder";"Identifier";"Content type";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "sub1";"news21";"News Item";"simple_publication_workflow";"private"
            "parent";"id";"type";"title";"description";"text";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "Parent folder";"Identifier";"Content type";"Title";"label_summary";"label_body_text";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain";"ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state"
            "sub1";"doc31";"Document";"Document 2 du dossier 1";"";"";"simple_publication_workflow";"private"
            <BLANKLINE>
        
        This output is not that good to deal with CSV apis, we will try to export it to a flat structure.
        ::
        
            >>> from Products.csvreplicata import replicator
            >>> self.portal.portal_csvreplicatatool.setPlainFormat(True)
            >>> self.portal.portal_csvreplicatatool.getPlainFormat()
            True
            >>> repl = Icsvreplicata(self.folder)
            >>> print Icsvreplicata(self.folder).csvexport(exportable_content_types=['Document', 'Folder', 'News Item'], depth=0).getvalue() # doctest: +ELLIPSIS
            "startpoint";"replicata_export_date";"parent";"id";"type";"title";"description";...
            "Start point";"Export Date";"Parent folder";"Identifier";"Content type";"Title";...
            "/plone/Members/test_user_1_";"...";"";"doc1";"Document";"Document 1";...
            "/plone/Members/test_user_1_";"...";"";"doc2";"Document";"Document 2";...
            "/plone/Members/test_user_1_";"...";"";"news1";"News Item";...
            "/plone/Members/test_user_1_";"...";"";"doc4";"Document";"Document 4";...
            "/plone/Members/test_user_1_";"...";"";"sub1";"Folder";"mytest";...
            "/plone/Members/test_user_1_";"...";"sub1";"doc11";"Document";"Document 1 du dossier 1";...
            "/plone/Members/test_user_1_";"...";"sub1";"news21";"News Item";...
            "/plone/Members/test_user_1_";"...";"sub1";"doc31";"Document";"Document 2 du dossier 1";...
            <BLANKLINE>
        
        Redo the export but with a plugin that find the title on the object.
        ::
        
            >>> from Products.csvreplicata import adapters
            >>> class CustomExporter(adapters.CSVReplicataExportImportPluginAbstract):
            ...    def __init__(self, *args, **kwargs):
            ...        adapters.CSVReplicataExportImportPluginAbstract.__init__(self, *args, **kwargs)
            ...        self.ids.append('title')
            ...    def fill_values(self, row, row_ids):
            ...        """."""
            ...        for id in row_ids:
            ...            if id.replace(self.prefix, '') in self.ids:
            ...                index = row_ids.index(id)
            ...                if index < len(row):
            ...                    row[index] = self.context.Title()
            ...    def set_values(self, row, row_ids):
            ...        """."""
            ...        print "plugin.setValue called with %s <-> %s" % (row, row_ids)
            ...
            >>> provideAdapter(CustomExporter, (interfaces.Icsvreplicata, zope.interface.Interface), interfaces.ICSVReplicataExportImportPlugin, name ='fooplugin' )
            >>> from csv import DictReader
            >>> content = Icsvreplicata(self.folder).csvexport(exportable_content_types=['Document', 'Folder', 'News Item'], depth=None).getvalue()
            >>> items = [item for item in DictReader(StringIO(content), delimiter=";", quotechar='"')]
            >>> keys = items[0].keys();keys.sort()
        
        As we cant predict order of the keys, doing some magic to order them before testing.
        ::
         
            >>> pprint([[(key, item[key]) for key in keys if not 'date' in key]for item in items], width=130) # doctest: +REPORT_NDIFF
            [[('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain',
               'ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain'),
              ('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state',
               'ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state'),
              ('ReplicataPlugin_Products_csvreplicata_tests_test_doctests_CustomExporter_title',
               'ReplicataPlugin_Products_csvreplicata_tests_test_doctests_CustomExporter_title'),
              ('description', 'label_...'),
              ('id', 'Identifier'),
              ('parent', 'Parent folder'),
              ('startpoint', 'Start point'),
              ('text', 'label_body_text'),
              ('title', 'Title'),
              ('type', 'Content type')],
             [('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain', 'simple_publication_workflow'),
              ('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state', 'private'),
              ('ReplicataPlugin_Products_csvreplicata_tests_test_doctests_CustomExporter_title', 'Document 1'),
              ('description', ''),
              ('id', 'doc1'),
              ('parent', ''),
              ('startpoint', '/plone/Members/test_user_1_'),
              ('text', ''),
              ('title', 'Document 1'),
              ('type', 'Document')],
             [('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain', 'simple_publication_workflow'),
              ('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state', 'private'),
              ('ReplicataPlugin_Products_csvreplicata_tests_test_doctests_CustomExporter_title', 'Document 2'),
              ('description', ''),
              ('id', 'doc2'),
              ('parent', ''),
              ('startpoint', '/plone/Members/test_user_1_'),
              ('text', ''),
              ('title', 'Document 2'),
              ('type', 'Document')],
             [('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain', 'simple_publication_workflow'),
              ('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state', 'private'),
              ('ReplicataPlugin_Products_csvreplicata_tests_test_doctests_CustomExporter_title', "news 'super' 3"),
              ('description', ''),
              ('id', 'news1'),
              ('parent', ''),
              ('startpoint', '/plone/Members/test_user_1_'),
              ('text', ''),
              ('title', ''),
              ('type', 'News Item')],
             [('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain', 'simple_publication_workflow'),
              ('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state', 'private'),
              ('ReplicataPlugin_Products_csvreplicata_tests_test_doctests_CustomExporter_title', 'Document 4'),
              ('description', ''),
              ('id', 'doc4'),
              ('parent', ''),
              ('startpoint', '/plone/Members/test_user_1_'),
              ('text', ''),
              ('title', 'Document 4'),
              ('type', 'Document')],
             [('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain', 'simple_publication_workflow'),
              ('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state', 'private'),
              ('ReplicataPlugin_Products_csvreplicata_tests_test_doctests_CustomExporter_title', 'mytest'),
              ('description', ''),
              ('id', 'sub1'),
              ('parent', ''),
              ('startpoint', '/plone/Members/test_user_1_'),
              ('text', ''),
              ('title', 'mytest'),
              ('type', 'Folder')],
             [('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain', 'simple_publication_workflow'),
              ('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state', 'private'),
              ('ReplicataPlugin_Products_csvreplicata_tests_test_doctests_CustomExporter_title', 'Document 1 du dossier 1'),
              ('description', ''),
              ('id', 'doc11'),
              ('parent', 'sub1'),
              ('startpoint', '/plone/Members/test_user_1_'),
              ('text', ''),
              ('title', 'Document 1 du dossier 1'),
              ('type', 'Document')],
             [('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain', 'simple_publication_workflow'),
              ('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state', 'private'),
              ('ReplicataPlugin_Products_csvreplicata_tests_test_doctests_CustomExporter_title', 'news 1 du dossier 1'),
              ('description', ''),
              ('id', 'news21'),
              ('parent', 'sub1'),
              ('startpoint', '/plone/Members/test_user_1_'),
              ('text', ''),
              ('title', ''),
              ('type', 'News Item')],
             [('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_chain', 'simple_publication_workflow'),
              ('ReplicataPlugin_Products_csvreplicata_exportimport_plugins_WorkflowExportImporter_wf_state', 'private'),
              ('ReplicataPlugin_Products_csvreplicata_tests_test_doctests_CustomExporter_title', 'Document 2 du dossier 1'),
              ('description', ''),
              ('id', 'doc31'),
              ('parent', 'sub1'),
              ('startpoint', '/plone/Members/test_user_1_'),
              ('text', ''),
              ('title', 'Document 2 du dossier 1'),
              ('type', 'Document')]]
        
        
        
        
        Import
        +++++++++
        
        Then, now that we got a working export, what about importing it
        ::
        
            >>> id = self.folder.invokeFactory('Folder'      , id='fa'   , title="tests import")
            >>> id = self.folder.invokeFactory('Folder'      , id='fb'   , title="tests import")
            >>> fa = self.folder.fa; fb = self.folder.fb
        
        Import in CSVReplicata plain format, an entry per line without contextual type hinting
        ::
        
            >>> CSV = StringIO("""\
            ... "startpoint";"replicata_export_date";"parent";"id";"type";"title";"description";"ReplicataPlugin_Products_csvreplicata_tests_test_doctests_CustomExporter_title";"text"
            ... "Start point";"Export Date";"Parent folder";"Identifier";"Content type";"Title";"label_...";"ReplicataPlugin_Products_csvreplicata_tests_test_doctests_CustomExporter_title";"label_body_text"
            ... "/plone/Members/test_user_1_";20090101010101;"";"doc1";"Document";"Document 1";"";"Document 1";""
            ... "/plone/Members/test_user_1_";20090101010101;"";"doc2";"Document";"Document 2";"";"Document 2";""
            ... "/plone/Members/test_user_1_";20090101010101;"";"news1";"News Item";"";"";"news 'super' 3";""
            ... "/plone/Members/test_user_1_";20090101010101;"";"doc4";"Document";"Document 4";"";"Document 4";""
            ... "/plone/Members/test_user_1_";20090101010101;"";"sub1";"Folder";"mytest";"";"mytest";""
            ... "/plone/Members/test_user_1_";20090101010101;"sub1";"doc11";"Document";"Document 1 du dossier 1";"";"Document 1 du dossier 1";""
            ... "/plone/Members/test_user_1_";20090101010101;"sub1";"news21";"News Item";"";"";"news 1 du dossier 1";""
            ... "/plone/Members/test_user_1_";20090101010101;"sub1";"doc31";"Document";"Document 2 du dossier 1";"";"Document 2 du dossier 1";""
            ... """)
            >>> print Icsvreplicata(self.folder.fa).csvimport(CSV, datetimeformat='%d%m%Y', delimiter=";", stringdelimiter='"', plain_format=True)
            p...
            (8, 0, ..., [])
            >>> self.folder.fa.objectIds()
            ['doc1', 'doc2', 'news1', 'doc4', 'sub1']
            >>> self.folder.fa.sub1.objectIds()
            ['doc11', 'news21', 'doc31']
        
        Import in CSVReplicata orignal format, an entry per line with contextual type hinting
        :: 
        
            >>> CSV = StringIO("""\
            ... "/plone/Members/test_user_1_";20100101010101
            ... "parent";"id";"type";"title";"description";"text"
            ... "Parent folder";"Identifier";"Content type";"Title";"label_...";"label_body_text"
            ... "";"doc1";"Document";"Document 1";"";""
            ... "";"doc2";"Document";"Document 2";"";""
            ... "parent";"id";"type"
            ... "Parent folder";"Identifier";"Content type"
            ... "";"news1";"News Item"
            ... "parent";"id";"type";"title";"description";"text"
            ... "Parent folder";"Identifier";"Content type";"Title";"label_...";"label_body_text"
            ... "";"doc4";"Document";"Document 4";"";""
            ... "parent";"id";"type";"title";"description"
            ... "Parent folder";"Identifier";"Content type";"Title";"label_..."
            ... "";"sub1";"Folder";"mytest";""
            ... "parent";"id";"type";"title";"description";"text"
            ... "Parent folder";"Identifier";"Content type";"Title";"label_...";"label_body_text"
            ... "sub1";"doc11";"Document";"Document 1 du dossier 1";"";""
            ... "parent";"id";"type"
            ... "Parent folder";"Identifier";"Content type"
            ... "sub1";"news21";"News Item"
            ... "parent";"id";"type";"title";"description";"text"
            ... "Parent folder";"Identifier";"Content type";"Title";"label_...";"label_body_text"
            ... "sub1";"doc31";"Document";"Document 2 du dossier 1";"";""
            ... """)
            >>> print Icsvreplicata(self.folder.fb).csvimport(CSV, datetimeformat='%d%m%Y', delimiter=";", stringdelimiter='"', plain_format=False)
            p...
            (8, 0, ..., [])
            >>> self.folder.fb.objectIds()
            ['doc1', 'doc2', 'news1', 'doc4', 'sub1']
            >>> self.folder.fb.sub1.objectIds()
            ['doc11', 'news21', 'doc31'] 
        
        
        
        Default handlers
        ----------------
        
        Verify the default handlers provided for Archetypes fields::
        
            >>> handlersDict = self.portal.portal_csvreplicatatool.getHandlers()
            >>> handlers = handlersDict.keys();handlers.sort()
            >>> pprint(handlers)
            ['Products.Archetypes.Field.BooleanField',
             'Products.Archetypes.Field.DateTimeField',
             'Products.Archetypes.Field.FileField',
             'Products.Archetypes.Field.FloatField',
             'Products.Archetypes.Field.ImageField',
             'Products.Archetypes.Field.IntegerField',
             'Products.Archetypes.Field.LinesField',
             'Products.Archetypes.Field.ReferenceField',
             'Products.Archetypes.Field.StringField',
             'Products.Archetypes.Field.TextField',
             'Products.AttachmentField.AttachmentField.AttachmentField',
             'default_handler',
             ...]
            >>> pprint([(h, handlersDict[h]['handler_class']) for h in handlers], width=130)
            [('Products.Archetypes.Field.BooleanField', <Products.csvreplicata.handlers.base.CSVBoolean object at ...>),
             ('Products.Archetypes.Field.DateTimeField', <Products.csvreplicata.handlers.base.CSVDateTime object at ...>),
             ('Products.Archetypes.Field.FileField', <Products.csvreplicata.handlers.file.CSVFile object at ...>),
             ('Products.Archetypes.Field.FloatField', <Products.csvreplicata.handlers.base.CSVFloat object at ...>),
             ('Products.Archetypes.Field.ImageField', <Products.csvreplicata.handlers.file.CSVFile object at ...>),
             ('Products.Archetypes.Field.IntegerField', <Products.csvreplicata.handlers.base.CSVInteger object at ...>),
             ('Products.Archetypes.Field.LinesField', <Products.csvreplicata.handlers.base.CSVLines object at ...>),
             ('Products.Archetypes.Field.ReferenceField', <Products.csvreplicata.handlers.reference.CSVReference object at ...>),
             ('Products.Archetypes.Field.StringField', <Products.csvreplicata.handlers.base.CSVString object at ...>),
             ('Products.Archetypes.Field.TextField', <Products.csvreplicata.handlers.base.CSVText object at ...>),
             ('Products.AttachmentField.AttachmentField.AttachmentField', <Products.csvreplicata.handlers.file.CSVFile object at ...>),
             ('default_handler', <Products.csvreplicata.handlers.base.CSVdefault object at ...>),
             (...
        
        
        What happened with csvreplicata during import/export if MyField is not in tool's
        handlers. replicator.py apllies default_handler on it::
        
            {'default_handler':
             {'handler_class': base.CSVdefault(),'file': False}}
        
        
        
        
        
        The replicator exporter/downloader
        ------------------------------------
        
        Export in normal mode
        +++++++++++++++++++++++++
        here we export folders and documents ::
        
            >>> from Products.csvreplicata.browser import manager
            >>> self.portal.portal_csvreplicatatool.replicabletypes = \
            ... {'Document':['default'], 'Folder':['default'],
            ... 'News Item': ['default'], 'File': ['default'] }
            >>> import re
            >>> self.setRoles(['Manager'])
            >>> id=self.folder.invokeFactory('Document'      , id='doc1'   , title="Document 1")
            >>> id=self.folder.invokeFactory('Document'      , id='doc2'   , title="Document 2")
            >>> params = {"datetimeformat": '%d/%m/%Y %H:%M:%S',
            ... "vocabularyvalue": "No",
            ... "encoding": "UTF-8",
            ... "delimiter": ";",
            ... "stringdelimiter": '"',
            ... "exportable_content_types": ["News Items", "Document", "Folder"],
            ... }
            >>> req = make_request('/'.join(self.folder.getPhysicalPath())+'@@csvreplicata', **params)
            >>> repl = manager.ReplicationManager(self.folder, req)
            >>> from Products.csvreplicata import replicator
            >>> print ''.join([a for a in repl.doExport()])
            "/plone/Members/test_user_1_";...
            "parent";"id";"type";"title";"description";"text";...
            "Parent folder";"Identifier";"Content type";"Title";"label_summary";...
            "";"doc1";"Document";"Document 1";"";...
            "";"doc2";"Document";"Document 2";"";...
            <BLANKLINE>
        
            >>> items = list(req.response._headers.iteritems());items.sort();pprint(items)
            [('content-disposition', ['attachment; filename=export.csv']),
             ('content-length', ['...']),
             ('content-type', ['text/csv;charset=UTF-8'])]
        
        
        Export as zip when there are files out there and we want them
        ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
        
        Now adding and exporting a file::
        
            >>> params['exportfiles'] = 'Yes'
            >>> params["exportable_content_types"].append('File')
            >>> id=self.folder.invokeFactory('File', id='f1'  , title="File 1")
            >>> req = make_request('/'.join(self.folder.getPhysicalPath())+'@@csvreplicata', **params)
            >>> f1 = self.folder.f1
            >>> f1.getFile().data = 'foo'
            >>> f1.setFilename('bar')
            >>> repl = manager.ReplicationManager(self.folder, req)
            >>> from StringIO import StringIO
            >>> import zipfile
            >>> content = StringIO([a for a in repl.doExport()][0])
            >>> zip = zipfile.ZipFile(content)
            >>> print zip.read('export.csv')
            "/plone/Members/test_user_1_";...
            "parent";"id";"type";"title";"description";"...
            "Parent folder";"Identifier";"Content type";"Title";"label_summary";...
            "";"doc1";"Document";"Document 1";...
            "";"doc2";"Document";"Document 2";...
            "parent";"id";"type";"title";"description";...
            "Parent folder";"Identifier";"Content ...
            "";"f1";"File";"File 1";"";...
            <BLANKLINE>
        
        
            >>> zip.printdir()
            File Name...
            bar...
            export.csv...
            >>> items = list(req.response._headers.iteritems());items.sort();pprint(items)
            [('content-disposition', ['attachment; filename=export.zip']),
             ('content-length', ['...']),
             ('content-type', ['application/zip'])]
        
        
        The File Stream Iterator
        +++++++++++++++++++++++++
        
        This object returns a generator to read our big files!::
        
            >>> from Products.csvreplicata.browser.manager import FileStreamIterator, EphemeralStreamIterator
            >>> import tempfile
            >>> fpath = tempfile.mkstemp('foo')[1]
            >>> fobj = open(fpath, 'w');fobj.write('foo');fobj.close()
            >>> len(FileStreamIterator(fpath))
            3
        
        We can play with chunks to divide rendering into small parts    ::
        
            >>> [[a for a in FileStreamIterator(fpath, chunk=chunk)] for chunk in [1,2, 3,4]]
            [['f', 'o', 'o'], ['fo', 'o'], ['foo'], ['foo']]
        
            >>> os.unlink(fpath)
        
        
        The Ephemeral Stream Iterator
        +++++++++++++++++++++++++++++++++
        
        
        This object returns a generator to read our big files but delete them when they are read!::
        
            >>> from Products.csvreplicata.browser.manager import FileStreamIterator, EphemeralStreamIterator
            >>> import tempfile
            >>> fdir = tempfile.mkdtemp(); fdir2 = tempfile.mkdtemp(dir=fdir);fpath = os.path.join(fdir2, 'foo')
            >>> fobj = open(fpath, 'w');fobj.write('foo');fobj.close()
        
        Files are there, we can ask to not delete parents (default)::
        
            >>> [os.path.exists(p) for p in fdir,fdir2, fpath]
            [True, True, True]
            >>> len(EphemeralStreamIterator(fpath, delete_parent=False, delete_grand_parent=False))
            3
            >>> [a for a in EphemeralStreamIterator(fpath, delete_parent=False, delete_grand_parent=False)]
            ['foo']
            >>> [os.path.exists(p) for p in fdir,fdir2, fpath]
            [True, True, False]
        
        We have read it, the file and the parent are deleted::
        
            >>> fobj = open(fpath, 'w');fobj.write('foo');fobj.close()
            >>> [a for a in EphemeralStreamIterator(fpath, delete_parent=True, delete_grand_parent=True)]
            ['foo']
            >>> [os.path.exists(p) for p in fdir2, fdir, fpath]
            [False, False, False]
        
        
        
        Changelog
        ===========
        
        
        1.1.11 (2012-12-21)
        -------------------
        
        - fix rst markup
        
        
        1.1.10 (2012-12-21)
        -------------------
        
        - Better non exportable types[kiorky]
        - better references restore [kiorky]
        - fixed error (invalid literal for int) [potar]
        - fixed compatibility with SmartFolder [potar]
        
        
        1.1.8 (2012-05-28)
        ------------------
        - fix blob files export/import [kiorky]
        - handle years < 999 [kiorky]
        - let commits be more incremental [kiorky]
        - fix export order [kiorky]
        - improve retry policy [kiorky]
        
        
        1.1.7  (2010-08-20)
        ---------------------
        - Adding comments exporter/searcher [kiorky]
        
        
        1.1.6  (2010-07-30)
        ----------------------
        - Fix importObject (obj referenced before assignment) [danjacka]
        
        1.1.5  (2010-07-29)
        -------------------
        - Fix unittest for plone4 msgid changes [kiorky]
        - Implement import plugins [kiorky]
        - Add a meta.zcml with some basic plugins, like for wf state export/import [kiorky]
        - Rename objects for consistency purpose [kiorky]
        - Fix some Acquisition bugs [kiorky]
        - Support blobstorage [kiorky]
        
        1.1.4  (2010-03-05)
        -------------------
        
        - fix some encoding issues with file handler [kiorky]
        - better plain format import [kiorky]
        - Fix setSubject [kiorky]
        
        1.1.3  (2010-01-27)
        --------------------
        
        - Fixed missing import of interface.implements in browser.manager [fRiSi]
        
        1.1.2  (2010-01-27)
        ---------------------
        
        
        - Fix silly empty list bug in replicator.csvimport prototype. [kiorky]
        
        - fixed missing import of interface.implements in browser.manager [fRiSi]
        
        - Fix unicode error with reference fields [kiorky]
        
        - Add support for temporary export path [kiorky]
        
        - Add support for flattened CSV Files [kiorky]
        
        - Add tests, and tests infrastructure [kiorky]
        
        - Add Stream (Files) Iterators not to overhead the RAM [kiorky]
        
        1.1.1 - Unreleased
        -------------------
        
        - remove five:traversable directive (deprecated in plone3) [toutpt]
        
        - Add default config for importcsvStep [toutpt]
        
        - importcsvStep now depend on plone-final step [toutpt]
        
        - option ``ignore_content_errors`` added to import step config (set it to true) and to the replicator methods. It allows to log errors when setting fields instead of raising exceptions and stop. [kiorky]
        
        
        
        1.1  (2009-10-17)
        -----------------
        
        - A new import step to use csvreplicata to import contents [toutpt]
        
        - Fix creationflag issue [toutpt]
        
        - plugins system. Export other things than AT. take a look at interfaces and adpaters [kiorky]
        
        - some encoding bugs fixed [kiorky]
        
        - datetime format settings in the tool [kiorky]
        
        - images/files created with folder structure in export [kiorky]
        
        - Plone 2.5 compatibility back [kiorky]
        
        1.0.7  (2009-07-15)
        --------------------
        
        - Prevent fixTools from running outside of the context of Products.csvreplicata [pigeonflight]
        
        - Fix export when data value is not ascii [yboussard]
        
        1.0.6
        -------
        
        - support vocabulary defined as object method [Jim BAACK]
        
        1.0.5
        -----
        
        - fix bug crc32 with large media files
        - when you set a CSVHandledTypesSchematas default is automatically preselected for user convenience [Jean-Philippe CAMGUILHEM]
        
        1.0.4
        -----
        
        - Uninstall problem fix
        
          IMPORTANT NOTE: when migrating from previous versions, you need to uninstall csvreplicata and then reinstall csvreplicata. [Jean-Philippe CAMGUILHEM]
        
        1.0.3
        -----
        
        - Custom handlers can be now implemented outside the product, and dynamically declared to the csvreplicata tool. [Jean-Philippe CAMGUILHEM]
        
        1.0.2
        -----
        
        - Initial release.[Eric BREHAULT / Christophe SAUTHIER]
        
        1.0
        ---
        
        - Unreleased
        
        
Keywords: archetypes,import/export,csv,replication,synchronisation
Platform: UNKNOWN
Classifier: Framework :: Plone
Classifier: Programming Language :: Python
Classifier: Topic :: Software Development :: Libraries :: Python Modules
