Utils

flowtask.utils

DataIntegration Utils library.

AttrDict

AttrDict(*args, **kwargs)

Bases: builtins.dict

AttrDict. Allow to using a dictionary like an object

__doc__ class-attribute

__doc__ = '\n    AttrDict.\n    Allow to using a dictionary like an object\n    '

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__module__ class-attribute

__module__ = 'flowtask.types.typedefs'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__weakref__ property

__weakref__

list of weak references to the object

__getattr__ method descriptor

__getattr__()

x.getitem(y) <==> x[y]

__setattr__ method descriptor

__setattr__(key, value)

Set self[key] to value.

MailMessage

MailMessage(directory, raw, flags=None, validate_mime=False)

docstring for MailMessage

NullDefault

Bases: builtins.dict

NullDefault.

When an attribute is missing, return default.

__doc__ class-attribute

__doc__ = 'NullDefault.\n\n    When an attribute is missing, return default.\n    '

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__module__ class-attribute

__module__ = 'flowtask.types.typedefs'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__weakref__ property

__weakref__

list of weak references to the object

__missing__ method descriptor

__missing__(key)

NullDefault.missing(self, key)

SafeDict

Bases: builtins.dict

SafeDict.

Allow to using partial format strings

__doc__ class-attribute

__doc__ = '\n    SafeDict.\n\n    Allow to using partial format strings\n\n    '

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__module__ class-attribute

__module__ = 'flowtask.types.typedefs'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__weakref__ property

__weakref__

list of weak references to the object

__missing__ method descriptor

__missing__(key)

SafeDict.missing(self, str key) Missing method for SafeDict.

as_boolean method descriptor

as_boolean(value)

as_boolean(value) -> bool as_boolean. Converting any value to a boolean.

check_empty method descriptor

check_empty(obj)

check_empty(obj) -> bool check_empty. Check if a basic object is empty or not.

extract_path method descriptor

extract_path(value, regex, to_date=False, to_datetime=False, mask='%m/%d/%Y')

extract_path(unicode value, unicode regex, bool to_date=False, bool to_datetime=False, unicode mask=u'%m/%d/%Y')

extract_string method descriptor

extract_string(value, exp='_((\\d+)_(\\d+))_', group=1, parsedate=False, masks=['%Y-%m-%d %H:%M:%S'])

extract_string(unicode value, unicode exp=u'((\d+)(\d+))_', int group=1, bool parsedate=False, list masks=[u'%Y-%m-%d %H:%M:%S']) -> unicode

install_uvloop

install_uvloop()

install uvloop and set as default loop for asyncio.

is_empty method descriptor

is_empty(obj)

check_empty(obj) -> bool check_empty. Check if a basic object is empty or not.

encoders

BaseEncoder

BaseEncoder(*args, **kwargs)

Encoder replacement for json.dumps using orjson

executor

Function Executor.

getFunction

getFunction(fname)

Get any function using name.

functions

Main Functions for DataIntegrator.

__doc__ module-attribute

__doc__ = '\nMain Functions for DataIntegrator.\n'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__file__ module-attribute

__file__ = '/home/jesuslara/proyectos/parallel/flowtask/flowtask/utils/functions.cpython-311-x86_64-linux-gnu.so'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__name__ module-attribute

__name__ = 'flowtask.utils.functions'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__package__ module-attribute

__package__ = 'flowtask.utils'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__test__ module-attribute

__test__ = {}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

as_boolean method descriptor

as_boolean(value)

as_boolean(value) -> bool as_boolean. Converting any value to a boolean.

check_empty method descriptor

check_empty(obj)

check_empty(obj) -> bool check_empty. Check if a basic object is empty or not.

extract_path method descriptor

extract_path(value, regex, to_date=False, to_datetime=False, mask='%m/%d/%Y')

extract_path(unicode value, unicode regex, bool to_date=False, bool to_datetime=False, unicode mask=u'%m/%d/%Y')

extract_string method descriptor

extract_string(value, exp='_((\\d+)_(\\d+))_', group=1, parsedate=False, masks=['%Y-%m-%d %H:%M:%S'])

extract_string(unicode value, unicode exp=u'((\d+)(\d+))_', int group=1, bool parsedate=False, list masks=[u'%Y-%m-%d %H:%M:%S']) -> unicode

file_extension method descriptor

file_extension(path)

file_extension(unicode path) -> unicode

filename method descriptor

filename(path)

filename(unicode path) -> unicode

get_worker_list method descriptor

get_worker_list(workers)

get_worker_list(list workers) -> list Convert a list of workers in a tuple of worker:port for Scheduler.

is_empty method descriptor

is_empty(obj)

check_empty(obj) -> bool check_empty. Check if a basic object is empty or not.

snake_case method descriptor

snake_case(s)

snake_case(unicode s) -> unicode Convert string to snake-case.

json

JSON Encoder, Decoder.

__doc__ module-attribute

__doc__ = '\nJSON Encoder, Decoder.\n'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__file__ module-attribute

__file__ = '/home/jesuslara/proyectos/parallel/flowtask/flowtask/utils/json.cpython-311-x86_64-linux-gnu.so'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__name__ module-attribute

__name__ = 'flowtask.utils.json'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__package__ module-attribute

__package__ = 'flowtask.utils'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__test__ module-attribute

__test__ = {}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

JSONContent

Basic Encoder using orjson

__doc__ class-attribute

__doc__ = '\n    Basic Encoder using orjson\n    '

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__call__ method descriptor

__call__(*args, **kwargs)

Call self as a function.

__new__ builtin

__new__(*args, **kwargs)

Create and return a new object. See help(type) for accurate signature.

__reduce__ method descriptor

__reduce__()

JSONContent.reduce_cython(self)

__reduce_cython__ method descriptor

__reduce_cython__()

JSONContent.reduce_cython(self)

__setstate__ method descriptor

__setstate__()

JSONContent.setstate_cython(self, __pyx_state)

__setstate_cython__ method descriptor

__setstate_cython__(__pyx_state)

JSONContent.setstate_cython(self, __pyx_state)

decode method descriptor

decode(obj)

JSONContent.decode(self, obj)

default method descriptor

default(obj)

JSONContent.default(self, obj)

dump classmethod

dump(obj, **kwargs)

JSONContent.dump(cls, obj, **kwargs)

dumps method descriptor

dumps(obj, **kwargs)

JSONContent.encode(self, obj, **kwargs) -> str

encode method descriptor

encode(obj, **kwargs)

JSONContent.encode(self, obj, **kwargs) -> str

load classmethod

load(obj, **kwargs)

JSONContent.load(cls, obj, **kwargs)

loads method descriptor

loads(obj)

JSONContent.decode(self, obj)

MailMessage

MailMessage(directory, raw, flags=None, validate_mime=False)

docstring for MailMessage

__pyx_unpickle_JSONContent method descriptor

__pyx_unpickle_JSONContent(__pyx_type, __pyx_checksum, __pyx_state)

__pyx_unpickle_JSONContent(__pyx_type, long __pyx_checksum, __pyx_state)

json_decoder method descriptor

json_decoder(obj)

json_decoder(obj)

json_encoder method descriptor

json_encoder(obj)

json_encoder(obj) -> str

mail

MailMessage

MailMessage(directory, raw, flags=None, validate_mime=False)

docstring for MailMessage

parserqs

__file__ module-attribute

__file__ = '/home/jesuslara/proyectos/parallel/flowtask/flowtask/utils/parserqs.cpython-311-x86_64-linux-gnu.so'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__name__ module-attribute

__name__ = 'flowtask.utils.parserqs'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__package__ module-attribute

__package__ = 'flowtask.utils'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__test__ module-attribute

__test__ = {}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

prefixes module-attribute

prefixes = frozenset({'(', '[', '{'})

frozenset() -> empty frozenset object frozenset(iterable) -> frozenset object

Build an immutable unordered collection of unique elements.

ParseDict

Bases: flowtask.utils.parserqs.ParseQS

__annotations__ class-attribute

__annotations__ = {'result': 'dict'}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__module__ class-attribute

__module__ = 'flowtask.utils.parserqs'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

result class-attribute

result = {}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

ParseList

Bases: flowtask.utils.parserqs.ParseQS

__annotations__ class-attribute

__annotations__ = {'separator': 'str', 'prefix': 'str', 'result': 'list'}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__module__ class-attribute

__module__ = 'flowtask.utils.parserqs'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

prefix class-attribute

prefix = '['

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

result class-attribute

result = []

Built-in mutable sequence.

If no argument is given, the constructor creates a new empty list. The argument must be an iterable if specified.

separator class-attribute

separator = ','

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

ParseQS

__annotations__ class-attribute

__annotations__ = {'separator': 'str', 'prefix': 'str', 'value': 'str'}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__module__ class-attribute

__module__ = 'flowtask.utils.parserqs'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

__weakref__ property

__weakref__

list of weak references to the object

prefix class-attribute

prefix = ''

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

separator class-attribute

separator = ','

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

value class-attribute

value = ''

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

ParseTuple

Bases: flowtask.utils.parserqs.ParseQS

__annotations__ class-attribute

__annotations__ = {'separator': 'str', 'prefix': 'str', 'result': 'tuple'}

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object's (key, value) pairs dict(iterable) -> new dictionary initialized as if via: d = {} for k, v in iterable: d[k] = v dict(**kwargs) -> new dictionary initialized with the name=value pairs in the keyword argument list. For example: dict(one=1, two=2)

__module__ class-attribute

__module__ = 'flowtask.utils.parserqs'

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

prefix class-attribute

prefix = '('

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

result class-attribute

result = []

Built-in mutable sequence.

If no argument is given, the constructor creates a new empty list. The argument must be an iterable if specified.

separator class-attribute

separator = ','

str(object='') -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.str() (if defined) or repr(object). encoding defaults to sys.getdefaultencoding(). errors defaults to 'strict'.

stats

TaskMonitor.

Collect and saves stats for execution of tasks.

TaskMonitor

TaskMonitor(name, program, task_id, userid=None)

transformations

camelcase_split

camelcase_split(value)

camelcase_split.

Splits a CamelCase word in other words.

to_camel_case

to_camel_case(s)

to_camel_case.

Converts a phrase into CamelCase Format.

Parameters:

Name	Type	Description	Default
s	str	The string to convert.	required

Returns:

Type	Description
str	The converted string in CamelCase format.

to_snake_case

to_snake_case(s)

to_snake_case.

Converts an string into snake_case format.

Parameters:

Name	Type	Description	Default
s	str	The string to convert.	required

Returns:

Type	Description
str	The converted string in snake_case format.

uv

install_uvloop

install_uvloop()

install uvloop and set as default loop for asyncio.