Skip to content

Copytobigquery

flowtask.components.CopyToBigQuery

CopyToBigQuery

CopyToBigQuery(loop=None, job=None, stat=None, **kwargs)

Bases: CopyTo, PandasDataframe

CopyToBigQuery.

Overview

This component allows copying data into a BigQuery table,
using write functionality from AsyncDB BigQuery driver.

.. table:: Properties :widths: auto

+--------------+----------+-----------+--------------------------------------------+
| Name         | Required | Summary                                                |
+--------------+----------+-----------+--------------------------------------------+
| tablename    |   Yes    | Name of the table in                                   |
|              |          | BigQuery                                               |
+--------------+----------+-----------+--------------------------------------------+
| schema       |   Yes    | Name of the dataset                                    |
|              |          | where the table is located                             |
+--------------+----------+-----------+--------------------------------------------+
| truncate     |   Yes    | This option indicates if the component should empty    |
|              |          | before copying the new data to the table. If set to    |
|              |          | true, the table will be truncated before saving data.  |
+--------------+----------+-----------+--------------------------------------------+
| use_buffer   |   No     | When activated, this option allows optimizing the      |
|              |          | performance of the task when dealing with large        |
|              |          | volumes of data.                                       |
+--------------+----------+-----------+--------------------------------------------+
| credentials  |   No     | Path to BigQuery credentials JSON file                 |
|              |          |                                                        |
+--------------+----------+-----------+--------------------------------------------+
| project_id   |   No     | Google Cloud Project ID                                |
|              |          |                                                        |
+--------------+----------+-----------+--------------------------------------------+


Example:

```yaml
CopyToBigQuery:
  schema: hisense
  tablename: product_availability_all
```

default_connection

default_connection()

default_connection.

Default Connection to BigQuery.