Metadata-Version: 1.1
Name: md.recipe.celery
Version: 1.0
Summary: Buildout recipe for installing celery for use with Zope's ZCA and configuring it using an ini file.
Home-page: https://github.com/minddistrict/md.recipe.celery
Author: Remco Wendt
Author-email: remco.wendt@gmail.com
License: BSD
Description: Introduction
        ============
        
        Buildout recipe for installing Celery for use with Zope's ZCA and configuring
        it using an ini file.
        
        This recipe was originally based on collective.recipe.celery,
        but differs greatly so that it now has become its own package. This recipe
        defines a custom loader that can read celery configuration from an ini file.
        This allows you to generate the configuration by using a template for
        example, using collective.recipe.template. The custom loader also bootstraps
        the ZCA automatically when a worker initializes.
        
        You can use it in a part like this::
        
            [celery]
            recipe = md.recipe.celery
            eggs =
              ${buildout:eggs}
              celery[redis]
            celery_conf = etc/celery.ini
            logging_conf = etc/logging.ini
            zope_conf = etc/zope.conf
        
        
        Supported options
        =================
        
        General options
        ---------------
        
        eggs
            A list of additional eggs you want to make available to Celery. Use this to
            add additional dependencies such as ``kombu-sqlalchemy`` or the module(s)
            containing your task definitions.
        
        celery_conf
            Location of the Celery configuration .ini file.
        
        logging_conf
            Location of the file containing the python logging setup
        
        zope_conf
            Location of the zope configuration file.
        
        
        Configuration
        =============
        
        Example of an ini file that could be used by this recipe::
        
            [celery]
            BROKER_URL = sqlite:///celery_broker.db
            CELERY_IMPORTS = myapp.tasks,otherapp.tasks
            CELERY_RESULT_BACKEND = sqlite:///celery_results.db
            CELERY_RESULT_SERIALIZER = json
        
            [celerybeat:task1]
            # Execute every hour
            task = myapp.tasks.Task1
            type = crontab
            schedule = {"minute": 0}
        
            [celerybeat:task2]
            # Execute every 30 seconds
            task = myapp.tasks.Task2
            type = timedelta
            schedule = {"seconds": 30}
        
            [celerybeat:task3]
            # Execute at midnight
            task = otherapp.tasks.Task3
            type = crontab
            schedule = {"hour": 0, "minute": 0}
        
        The [celery] section defines all the celery options. Every [celerybeat]
        section defines an individual task.
        
        Example usage
        =============
        
        Create a celery ini file holding your celery configuration::
        
            >>> write(sample_buildout, 'celery.ini',
            ... """
            ... [celery]
            ... BROKER_URL = sqlite:///celery_broker.db
            ... CELERY_IMPORTS = myapp.tasks,otherapp.tasks
            ... CELERY_RESULT_BACKEND = sqlite:///celery_results.db
            ... CELERY_RESULT_SERIALIZER = json
            ...
            ... [celerybeat:task1]
            ... # Execute every hour
            ... task = myapp.tasks.Task1
            ... type = crontab
            ... schedule = {"minute": 0}
            ...
            ... [celerybeat:task2]
            ... # Execute every 30 seconds
            ... task = myapp.tasks.Task2
            ... type = timedelta
            ... schedule = {"seconds": 30}
            ...
            ... [celerybeat:task3]
            ... # Execute at midnight
            ... task = otherapp.tasks.Task3
            ... type = crontab
            ... schedule = {"hour": 0, "minute": 0}""")
        
        This file could be generated by a template such as collective.recipe.template.
        
        Next create a logging config::
        
            >>> write(sample_buildout, 'logging.ini',
            ... """
            ... [loggers]
            ... keys = root
            ...
            ... [handlers]
            ... keys = console
            ...
            ... [formatters]
            ... keys = generic
            ...
            ... [logger_root]
            ... level = INFO
            ... handlers = console
            ...
            ... [handler_console]
            ... class = StreamHandler
            ... args = (sys.stderr,)
            ... level = NOTSET
            ... formatter = generic
            ...
            ... [formatter_generic]
            ... format = %(asctime)s %(levelname)s [%(name)s] %(message)s""")
        
        Then we'll create a buildout that uses the recipe::
        
            >>> write(sample_buildout, 'buildout.cfg',
            ... """
            ... [buildout]
            ... parts = celery
            ... index = %(server)s/index
            ... find-links = %(server)s
            ... 
            ... [celery]
            ... recipe = md.recipe.celery
            ... celery_conf = celery.ini
            ... logging_conf = logging.ini
            ... zope_conf = zope.conf
            ... """% dict(
            ...     server=link_server))
        
        Running the buildout gives us::
        
            >>> print system(buildout)
            Installing celery.
            celery: Creating directory /sample-buildout/parts/celery.
            celery: Generated loader file /sample-buildout/parts/celery/zopeloader.py.
            Getting distribution for 'celery'.
            Got celery 2.3.1.
            Generated script '/sample-buildout/bin/celeryctl'.
            Generated script '/sample-buildout/bin/celeryd'.
            <BLANKLINE>
        
        Check that we have the celery scripts::
        
            >>> ls(sample_buildout, 'bin')
            -  buildout
            -  celeryctl
            -  celeryd
        
        Check that we got the celery loaders::
        
            >>> ls(sample_buildout, 'parts', 'celery')
            - loader.py
            - zopeloader.py
        
        If we run the celeryd script, it prints out the config data::
        
            >>> print(system(join(sample_buildout, 'bin', 'celeryctl')))
            [('BROKER_URL', 'sqlite:///celery_broker.db'),
             ('CELERYBEAT_SCHEDULE',
              {'task1': {'schedule': <crontab: 0 * * * * (m/h/d/dM/MY)>,
                         'task': 'myapp.tasks.Task1'},
               'task2': {'schedule': datetime.timedelta(0, 30),
                         'task': 'myapp.tasks.Task2'},
               'task3': {'schedule': <crontab: 0 0 * * * (m/h/d/dM/MY)>,
                         'task': 'otherapp.tasks.Task3'}}),
             ('CELERYD_HIJACK_ROOT_LOGGER', False),
             ('CELERY_IMPORTS', ['myapp.tasks', 'otherapp.tasks']),
             ('CELERY_RESULT_BACKEND', 'sqlite:///celery_results.db'),
             ('CELERY_RESULT_SERIALIZER', 'json')]
            <BLANKLINE>
        
        We can include additional eggs using the eggs option::
        
            >>> write(sample_buildout, 'buildout.cfg',
            ... """
            ... [buildout]
            ... parts = celery
            ... index = %(server)s/index
            ... find-links = %(server)s
            ...
            ... [celery]
            ... recipe = md.recipe.celery
            ... eggs =
            ...     other
            ... """% dict(server=link_server))
        
            >>> print system(buildout),
            Uninstalling celery.
            Installing celery.
            celery: Generated loader file /sample-buildout/parts/celery/zopeloader.py.
            Getting distribution for 'other'.
            Got other 1.0.
            Generated script '/sample-buildout/bin/celeryctl'.
            Generated script '/sample-buildout/bin/celeryd'.
            Generated script '/sample-buildout/bin/distutilsscript'.
        
        The recipe should handle updates as well, trigger this by pinning celery to
        another version and checking one of the outputted scripts.
        
            >>> write(sample_buildout, 'buildout.cfg',
            ... """
            ... [buildout]
            ... parts = celery
            ... index = %(server)s/index
            ... find-links = %(server)s
            ...
            ... [versions]
            ... celery = 2.3.0
            ...
            ... [celery]
            ... recipe = md.recipe.celery
            ... eggs =
            ...     other
            ... """% dict(server=link_server))
        
            >>> print system(buildout),
            Updating celery.
            celery: Generated loader file /sample-buildout/parts/celery/zopeloader.py.
            Getting distribution for 'celery==2.3.0'.
            Got celery 2.3.0.
            Generated script '/sample-buildout/bin/celeryctl'.
            Generated script '/sample-buildout/bin/celeryd'.
            Generated script '/sample-buildout/bin/distutilsscript'.
        
            >>> cat(sample_buildout, 'bin', 'celeryctl')
            #!/Users/rwendt/Projects/python/bin/python
            <BLANKLINE>
            import sys
            sys.path[0:0] = [
              '/sample-buildout/eggs/celery-2.3.0-py2.7.egg',
              '/sample-buildout/eggs/other-1.0-py2.7.egg',
              '/sample-buildout/parts/celery',
              ]
            ***
        
        We can control which scripts are generated using the scripts option.
        If no value is given, then script generation is disabled::
        
            >>> write(sample_buildout, 'buildout.cfg',
            ... """
            ... [buildout]
            ... parts = celery
            ... index = %(server)s/index
            ... find-links = %(server)s
            ...
            ... [celery]
            ... recipe = md.recipe.celery
            ... scripts =
            ... """% dict(server=link_server))
        
            >>> print system(buildout),
            Uninstalling celery.
            Installing celery.
            celery: Generated loader file /sample-buildout/parts/celery/zopeloader.py.
        
            >>> ls(sample_buildout, 'bin')
            -  buildout
        
        Let's create the celeryd script only::
        
            >>> write(sample_buildout, 'buildout.cfg',
            ... """
            ... [buildout]
            ... parts = celery
            ... index = %(server)s/index
            ... find-links = %(server)s
            ...
            ... [celery]
            ... recipe = md.recipe.celery
            ... scripts =
            ...     celeryd
            ... """% dict(server=link_server))
        
            >>> print system(buildout),
            Uninstalling celery.
            Installing celery.
            celery: Generated loader file /sample-buildout/parts/celery/zopeloader.py.
            Generated script '/sample-buildout/bin/celeryd'.
        
            >>> ls(sample_buildout, 'bin')
            -  buildout
            -  celeryd
        
        =======
        CHANGES
        =======
        
        1.0 (2013-12-18)
        ================
        
        - Initial release. Based loosely on collective.recipe.celery.
        
Platform: UNKNOWN
Classifier: Framework :: Buildout :: Recipe
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python
Classifier: Topic :: Software Development :: Build Tools
Classifier: Topic :: Software Development :: Libraries :: Python Modules
