Transformrows¶
Bases: FlowComponent
TransformRows
Overview
The TransformRows class is a component for transforming, adding, or modifying rows in a Pandas DataFrame based on
specified criteria. It supports single and multiple DataFrame transformations, and various operations on columns.
.. table:: Properties :widths: auto
+--------------------+----------+-------------------------------------------------------------------------------------------------------+
| Name | Required | Description |
+--------------------+----------+-------------------------------------------------------------------------------------------------------+
| fields | No | A dictionary defining the fields and corresponding transformations to be applied. |
+--------------------+----------+-------------------------------------------------------------------------------------------------------+
| filter_conditions | No | A dictionary defining the filter conditions for transformations. |
+--------------------+----------+-------------------------------------------------------------------------------------------------------+
| clean_notnull | No | Boolean flag indicating if non-null values should be cleaned, defaults to True. |
+--------------------+----------+-------------------------------------------------------------------------------------------------------+
| replace_columns | No | Boolean flag indicating if columns should be replaced, defaults to False. |
+--------------------+----------+-------------------------------------------------------------------------------------------------------+
| multi | No | Boolean flag indicating if multiple DataFrame transformations should be supported, defaults to False. |
+--------------------+----------+-------------------------------------------------------------------------------------------------------+
| function | No | View the list of function in the functions.py file on this directory |
+--------------------+----------+-------------------------------------------------------------------------------------------------------+
| _applied | No | List to store the applied transformations. |
+--------------------+----------+-------------------------------------------------------------------------------------------------------+
Return
The methods in this class manage the transformation of DataFrames, including initialization, execution, and result handling.
Example:
```yaml
TransformRows:
fields:
display_name:
value:
- concat
- columns:
- first_name
- last_name
legal_address:
value:
- concat
- columns:
- legal_street_address_1
- legal_street_address_2
work_address:
value:
- concat
- columns:
- work_location_address_1
- work_location_address_2
first_name:
value:
- capitalize
last_name:
value:
- capitalize
warp_id:
value:
- nullif
- chars:
- '*'
old_warp_id:
value:
- nullif
- chars:
- '*'
worker_category_description:
value:
- case
- column: benefits_eligibility_class_code
condition: PART-TIME
match: Part Time
notmatch: Full Time
file_number:
value:
- ereplace
- columns:
- position_id
- payroll_group
newvalue: ''
original_hire_date:
value:
- convert_to_datetime
hire_date:
value:
- convert_to_datetime
start_date:
value:
- convert_to_datetime
updated:
value:
- convert_to_datetime
gender_code:
value:
- convert_to_string
payroll_id:
value:
- convert_to_string
reports_to_payroll_id:
value:
- convert_to_string
```