Metadata-Version: 2.4
Name: synapseclient
Version: 4.12.0
Summary: A client for Synapse, a collaborative, open-source research platform that allows teams to share data, track analyses, and collaborate.
Home-page: https://www.synapse.org
Author: The Synapse Engineering Team
Author-email: platform@sagebase.org
License: Apache-2.0
Project-URL: Source, https://github.com/Sage-Bionetworks/synapsePythonClient
Project-URL: Tracker, https://github.com/Sage-Bionetworks/synapsePythonClient/issues
Project-URL: Documentation, https://python-docs.synapse.org
Project-URL: Changelog, https://python-docs.synapse.org/en/stable/news/
Platform: any
Classifier: Development Status :: 5 - Production/Stable
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: Unix
Classifier: Operating System :: POSIX :: Linux
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: <3.15,>=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Requires-Dist: requests<3.0,>=2.22.0
Requires-Dist: urllib3>=2.6.3
Requires-Dist: deprecated<2.0,>=1.2.4
Requires-Dist: opentelemetry-api>=1.21.0
Requires-Dist: opentelemetry-sdk>=1.21.0
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.21.0
Requires-Dist: opentelemetry-instrumentation-httpx>=0.48b0
Requires-Dist: opentelemetry-instrumentation-requests>=0.48b0
Requires-Dist: opentelemetry-instrumentation-threading>=0.48b0
Requires-Dist: opentelemetry-instrumentation-urllib>=0.48b0
Requires-Dist: nest-asyncio~=1.6.0
Requires-Dist: asyncio-atexit~=1.0.1
Requires-Dist: httpx>=0.27.0
Requires-Dist: httpcore>=1.0.9
Requires-Dist: tqdm<5.0,>=4.66.2
Requires-Dist: async-lru~=2.0.4
Requires-Dist: psutil>=5.9.8
Requires-Dist: setuptools>=80.10.1
Provides-Extra: dev
Requires-Dist: pytest~=8.2.0; extra == "dev"
Requires-Dist: pytest-mock<4.0,>=3.0; extra == "dev"
Requires-Dist: pytest-socket~=0.6.0; extra == "dev"
Requires-Dist: pytest-asyncio<2.0,>=1.2.0; extra == "dev"
Requires-Dist: flake8<4.0,>=3.7.0; extra == "dev"
Requires-Dist: pytest-xdist[psutil]<3.0.0,>=2.2; extra == "dev"
Requires-Dist: pytest-rerunfailures~=12.0; extra == "dev"
Requires-Dist: func-timeout~=4.3; extra == "dev"
Requires-Dist: pytest-cov~=4.1.0; extra == "dev"
Requires-Dist: pytest-html~=4.1.0; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Requires-Dist: filelock>=3.20.3; extra == "dev"
Requires-Dist: pandas<3.0,>=1.5; extra == "dev"
Provides-Extra: tests
Requires-Dist: pytest~=8.2.0; extra == "tests"
Requires-Dist: pytest-mock<4.0,>=3.0; extra == "tests"
Requires-Dist: pytest-socket~=0.6.0; extra == "tests"
Requires-Dist: pytest-asyncio<2.0,>=1.2.0; extra == "tests"
Requires-Dist: flake8<4.0,>=3.7.0; extra == "tests"
Requires-Dist: pytest-xdist[psutil]<3.0.0,>=2.2; extra == "tests"
Requires-Dist: pytest-rerunfailures~=12.0; extra == "tests"
Requires-Dist: func-timeout~=4.3; extra == "tests"
Requires-Dist: pytest-cov~=4.1.0; extra == "tests"
Requires-Dist: pytest-html~=4.1.0; extra == "tests"
Requires-Dist: pandas<3.0,>=1.5; extra == "tests"
Requires-Dist: jsonschema>=4.23.0; extra == "tests"
Provides-Extra: pandas
Requires-Dist: pandas<3.0,>=1.5; extra == "pandas"
Provides-Extra: curator
Requires-Dist: pandas<3.0,>=1.5; extra == "curator"
Requires-Dist: pandarallel>=1.6.4; extra == "curator"
Requires-Dist: inflection>=0.5.1; extra == "curator"
Requires-Dist: networkx>=2.2.8; extra == "curator"
Requires-Dist: dataclasses-json>=0.6.1; extra == "curator"
Requires-Dist: rdflib>=6.0.0; extra == "curator"
Provides-Extra: pysftp
Requires-Dist: pysftp<0.3,>=0.2.8; extra == "pysftp"
Requires-Dist: paramiko<4.0.0; extra == "pysftp"
Provides-Extra: boto3
Requires-Dist: boto3<2.0,>=1.7.0; extra == "boto3"
Provides-Extra: docs
Requires-Dist: mkdocs>=1.5.3; extra == "docs"
Requires-Dist: mkdocs-material>=9.4.14; extra == "docs"
Requires-Dist: mkdocstrings>=0.24.0; extra == "docs"
Requires-Dist: mkdocstrings-python>=2.0.0; extra == "docs"
Requires-Dist: termynal>=0.11.1; extra == "docs"
Requires-Dist: mkdocs-open-in-new-tab~=1.0.3; extra == "docs"
Requires-Dist: markdown-include~=0.8.1; extra == "docs"
Provides-Extra: all
Requires-Dist: pytest~=8.2.0; extra == "all"
Requires-Dist: pytest-mock<4.0,>=3.0; extra == "all"
Requires-Dist: pytest-socket~=0.6.0; extra == "all"
Requires-Dist: pytest-asyncio<2.0,>=1.2.0; extra == "all"
Requires-Dist: flake8<4.0,>=3.7.0; extra == "all"
Requires-Dist: pytest-xdist[psutil]<3.0.0,>=2.2; extra == "all"
Requires-Dist: pytest-rerunfailures~=12.0; extra == "all"
Requires-Dist: func-timeout~=4.3; extra == "all"
Requires-Dist: pytest-cov~=4.1.0; extra == "all"
Requires-Dist: pytest-html~=4.1.0; extra == "all"
Requires-Dist: black; extra == "all"
Requires-Dist: pre-commit; extra == "all"
Requires-Dist: filelock>=3.20.3; extra == "all"
Requires-Dist: pandas<3.0,>=1.5; extra == "all"
Requires-Dist: pandas<3.0,>=1.5; extra == "all"
Requires-Dist: pandas<3.0,>=1.5; extra == "all"
Requires-Dist: pandarallel>=1.6.4; extra == "all"
Requires-Dist: inflection>=0.5.1; extra == "all"
Requires-Dist: networkx>=2.2.8; extra == "all"
Requires-Dist: dataclasses-json>=0.6.1; extra == "all"
Requires-Dist: rdflib>=6.0.0; extra == "all"
Requires-Dist: pysftp<0.3,>=0.2.8; extra == "all"
Requires-Dist: paramiko<4.0.0; extra == "all"
Requires-Dist: boto3<2.0,>=1.7.0; extra == "all"
Requires-Dist: mkdocs>=1.5.3; extra == "all"
Requires-Dist: mkdocs-material>=9.4.14; extra == "all"
Requires-Dist: mkdocstrings>=0.24.0; extra == "all"
Requires-Dist: mkdocstrings-python>=2.0.0; extra == "all"
Requires-Dist: termynal>=0.11.1; extra == "all"
Requires-Dist: mkdocs-open-in-new-tab~=1.0.3; extra == "all"
Requires-Dist: markdown-include~=0.8.1; extra == "all"

Synapse Python Client
=====================

Branch  | Build Status
--------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
develop | [![Build Status develop branch](https://github.com/Sage-Bionetworks/synapsePythonClient/actions/workflows/build.yml/badge.svg?branch=develop)](https://github.com/Sage-Bionetworks/synapsePythonClient/actions?query=branch%3Adevelop)
master  | [![Build Status master branch](https://github.com/Sage-Bionetworks/synapsePythonClient/actions/workflows/build.yml/badge.svg?branch=master)](https://github.com/Sage-Bionetworks/synapsePythonClient/actions?query=branch%3Amaster)

[![Get the synapseclient from PyPI](https://img.shields.io/pypi/v/synapseclient.svg)](https://pypi.python.org/pypi/synapseclient/) [![Supported Python Versions](https://img.shields.io/pypi/pyversions/synapseclient.svg)](https://pypi.python.org/pypi/synapseclient/)

A Python client for [Sage Bionetworks'](https://sagebionetworks.org/) [Synapse](https://www.synapse.org/), a collaborative, open-source research platform that allows teams to share data, track analyses, and collaborate. The Python client can be used as a library for development of software that communicates with Synapse or as a command-line utility.

There is also a [Synapse client for R](https://github.com/Sage-Bionetworks/synapser/).

Documentation
-------------

For more information about the Python client, see:

 * [Python client API docs](https://python-docs.synapse.org)

For more information about interacting with Synapse, see:

 * [Synapse API docs](https://rest-docs.synapse.org/rest/)
 * [Use cases](https://help.synapse.org/docs/Use-Cases.1985151645.html)
 * [Getting Started Guide to Synapse](https://help.synapse.org/docs/Getting-Started.2055471150.html)

For release information, see:

 * [Release notes](https://python-docs.synapse.org/news/)

<!-- Subscribe to release and other announcements [here](https://groups.google.com/a/sagebase.org/forum/#!forum/python-announce)
or by sending an email to [python-announce+subscribe@sagebase.org](mailto:python-announce+subscribe@sagebase.org) -->


Installation
------------

The Python Synapse client has been tested on versions 3.10, 3.11, 3.12, 3.13 and 3.14 on Mac OS X, Ubuntu Linux and Windows.

**Starting from Synapse Python client version 3.0, Synapse Python client requires Python >= 3.10**

### Install using pip

The [Python Synapse Client is on PyPI](https://pypi.python.org/pypi/synapseclient) and can be installed with pip:

    # Here are a few ways to install the client. Choose the one that fits your use-case
    # sudo may optionally be needed depending on your setup

    pip install --upgrade synapseclient
    pip install --upgrade "synapseclient[pandas]"
    pip install --upgrade "synapseclient[pandas, pysftp, boto3]"

...or to upgrade an existing installation of the Synapse client:

    # sudo may optionally be needed depending on your setup
    pip install --upgrade synapseclient

The dependencies on `pandas`, `pysftp`, and `boto3` are optional. Synapse
[Tables](https://python-docs.synapse.org/reference/tables/) integrate
with [Pandas](http://pandas.pydata.org/). The library `pysftp` is required for users of
[SFTP](https://python-docs.synapse.org/guides/data_storage/#sftp) file storage. All
libraries require native code to be compiled or installed separately from prebuilt
binaries.

### Install from source

Clone the [source code repository](https://github.com/Sage-Bionetworks/synapsePythonClient).

    git clone git://github.com/Sage-Bionetworks/synapsePythonClient.git
    cd synapsePythonClient
    pip install .

Alternatively, you can use pip to install a particular branch, commit, or other git reference:

    pip install git+https://github.com/Sage-Bionetworks/synapsePythonClient@master

or

    pip install git+https://github.com/Sage-Bionetworks/synapsePythonClient@my-commit-hash

Command line usage
------------------

The Synapse client can be used from the shell command prompt. Valid commands
include: query, get, cat, add, update, delete, and onweb. A few examples are
shown.

### downloading test data from Synapse

    synapse -p auth_token get syn1528299

### getting help

    synapse -h

Note that a [Synapse account](https://www.synapse.org/#RegisterAccount:0) is required.


Usage as a library
------------------

The Synapse client can be used to write software that interacts with the Sage Bionetworks Synapse repository. More examples can be found in the Tutorial section found [here](https://python-docs.synapse.org/tutorials/home/)

### Examples

#### Log-in and create a Synapse object
```
import synapseclient

syn = synapseclient.Synapse()
## You may optionally specify the debug flag to True to print out debug level messages.
## A debug level may help point to issues in your own code, or uncover a bug within ours.
# syn = synapseclient.Synapse(debug=True)

## log in using auth token
syn.login(authToken='auth_token')
```

#### Sync a local directory to synapse
This is the recommended way of synchronizing more than one file or directory to a synapse project through the use of `synapseutils`. Using this library allows us to handle scheduling everything required to sync an entire directory tree. Read more about the manifest file format in [`synapseutils.syncToSynapse`](https://python-docs.synapse.org/reference/synapse_utils/#synapseutils.sync.syncToSynapse)
```
import synapseclient
import synapseutils
import os

syn = synapseclient.Synapse()

## log in using auth token
syn.login(authToken='auth_token')

path = os.path.expanduser("~/synapse_project")
manifest_path = f"{path}/my_project_manifest.tsv"
project_id = "syn1234"

# Create the manifest file on disk
with open(manifest_path, "w", encoding="utf-8") as f:
    pass

# Walk the specified directory tree and create a TSV manifest file
synapseutils.generate_sync_manifest(
    syn,
    directory_path=path,
    parent_id=project_id,
    manifest_path=manifest_path,
)

# Using the generated manifest file, sync the files to Synapse
synapseutils.syncToSynapse(
    syn,
    manifestFile=manifest_path,
    sendMessages=False,
)
```

#### Store a Project to Synapse
```
import synapseclient
from synapseclient.models import Project

syn = synapseclient.Synapse()

## log in using auth token
syn.login(authToken='auth_token')

project = Project('My uniquely named project')
project.store()

print(project.id)
print(project)
```

#### Store a Folder to Synapse (Does not upload files within the folder)
```
import synapseclient
from synapseclient.models import Folder

syn = synapseclient.Synapse()

## log in using auth token
syn.login(authToken='auth_token')

folder = Folder(name='my_folder', parent_id="syn123")
folder.store()

print(folder.id)
print(folder)

```

#### Store a File to Synapse
```
import synapseclient
from synapseclient.models import File

syn = synapseclient.Synapse()

## log in using auth token
syn.login(authToken='auth_token')

file = File(
    path="path/to/file.txt",
    parent_id="syn123",
)
file.store()

print(file.id)
print(file)
```

#### Get a data matrix
```
import synapseclient
from synapseclient.models import File

syn = synapseclient.Synapse()

## log in using auth token
syn.login(authToken='auth_token')

## retrieve a 100 by 4 matrix
matrix = File(id='syn1901033').get()

## inspect its properties
print(matrix.name)
print(matrix.description)
print(matrix.path)

## load the data matrix into a dictionary with an entry for each column
with open(matrix.path, 'r') as f:
    labels = f.readline().strip().split('\t')
    data = {label: [] for label in labels}
    for line in f:
        values = [float(x) for x in line.strip().split('\t')]
        for i in range(len(labels)):
            data[labels[i]].append(values[i])

## load the data matrix into a numpy array
import numpy as np
np.loadtxt(fname=matrix.path, skiprows=1)
```


Authentication
--------------
Authentication toward [Synapse](https://www.synapse.org/#RegisterAccount:0) can be accomplished with the clients using personal access tokens. Learn more about [Synapse personal access tokens](https://help.synapse.org/docs/Managing-Your-Account.2055405596.html#ManagingYourAccount-PersonalAccessTokens)

Learn about the [multiple ways one can login to Synapse](https://python-docs.synapse.org/tutorials/authentication/).


Synapse Utilities (synapseutils)
--------------------------------

The purpose of synapseutils is to create a space filled with convenience functions that includes traversing through large projects, copying entities, recursively downloading files and many more.

### Example

    import synapseutils
    import synapseclient
    syn = synapseclient.login()

    # copies all Synapse entities to a destination location
    synapseutils.copy(syn, "syn1234", destinationId = "syn2345")

    # copies the wiki from the entity to a destination entity. Only a project can have sub wiki pages.
    synapseutils.copyWiki(syn, "syn1234", destinationId = "syn2345")


    # Traverses through Synapse directories, behaves exactly like os.walk()
    walkedPath = synapseutils.walk(syn, "syn1234")

    for dirpath, dirname, filename in walkedPath:
        print(dirpath)
        print(dirname)
        print(filename)

OpenTelemetry (OTEL)
--------------------------------
[OpenTelemetry](https://opentelemetry.io/) helps support the analysis of traces and spans which can provide insights into latency, errors, and other performance metrics. The synapseclient is ready to provide traces should you want them. The Synapse Python client supports OTLP Exports and can be configured via environment variables as defined [here](https://opentelemetry.io/docs/specs/otel/protocol/exporter/).

Read more about OpenTelemetry in Python [here](https://opentelemetry.io/docs/instrumentation/python/)


### Exporting Synapse Client Traces to Jaeger for developers
The following shows an example of setting up [jaegertracing](https://www.jaegertracing.io/docs/1.50/deployment/#all-in-one) via docker and executing a simple python script that implements the Synapse Python client.

#### Running the jaeger docker container
Start a docker container with the following options:
```
docker run --name jaeger \
  -e COLLECTOR_OTLP_ENABLED=true \
  -p 16686:16686 \
  -p 4318:4318 \
  jaegertracing/all-in-one:latest
```
Explanation of ports:
* `4318` HTTP port for OTLP data collection
* `16686` Jaeger UI for visualizing traces

Once the docker container is running you can access the Jaeger UI via: `http://localhost:16686`

#### Environment Variable Configuration

By default, the OTEL exporter sends trace data to `http://localhost:4318/v1/traces`. You can customize the behavior through environment variables:

* `OTEL_SERVICE_NAME`: Defines a unique identifier for your application or service in telemetry data (defaults to 'synapseclient'). Set this to a descriptive name that represents your specific implementation, making it easier to filter and analyze traces in your monitoring system.
* `OTEL_EXPORTER_OTLP_ENDPOINT`: Specifies the destination URL for sending telemetry data (defaults to 'http://localhost:4318'). Configure this to direct data to your preferred OpenTelemetry collector or monitoring service.
* `OTEL_DEBUG_CONSOLE`: Controls local visibility of telemetry data. Set to 'true' to output trace information to the console, which is useful for development and troubleshooting without an external collector.
* `OTEL_SERVICE_INSTANCE_ID`: Distinguishes between multiple instances of the same service (e.g., 'prod', 'development', 'local'). This helps identify which specific deployment or environment generated particular traces.
* `OTEL_EXPORTER_OTLP_HEADERS`: Configures authentication and metadata for telemetry exports. Use this to add API keys, authentication tokens, or custom metadata when sending traces to secured collectors or third-party monitoring services.


#### Enabling OpenTelemetry in your code
To enable OpenTelemetry with the Synapse Python client, simply call the
`enable_open_telemetry()` method on the Synapse class. Additionally you can access an
instance of the OpenTelemetry tracer via the `get_tracer()` call. This will allow you
to create new spans for your code.

```python
import synapseclient

# Enable OpenTelemetry with default settings
synapseclient.Synapse.enable_open_telemetry()
tracer = synapseclient.Synapse.get_tracer()

# Then create and use the Synapse client as usual
with tracer.start_as_current_span("my_function_span"):
    syn = synapseclient.Synapse()
    syn.login(authToken='auth_token')
```

### Exporting Synapse Client Traces to SigNoz Cloud for developers

#### Prerequisites
1. Create an account and obtain access to Signoz Cloud.
2. Create an ingestion key by following the step [here](https://signoz.io/docs/ingestion/signoz-cloud/keys/).

#### Environment Variable Configuration
The following environment variables are required to be set:
- `OTEL_EXPORTER_OTLP_HEADERS`: `signoz-ingestion-key=<key>`
- `OTEL_EXPORTER_OTLP_ENDPOINT`: `https://ingest.us.signoz.cloud`
- `OTEL_SERVICE_NAME`: `your-service-name`

Explanation of both required and optional environment variables:
##### Required
* `OTEL_EXPORTER_OTLP_ENDPOINT`: The OTLP endpoint to which telemetry is exported.
* `OTEL_EXPORTER_OTLP_HEADERS`: Authentication/metadata for exports (e.g., API keys, tokens). For SigNoz, use `signoz-ingestion-key=<key>`.

##### Optional
* `OTEL_SERVICE_NAME`: Unique identifier for your app/service in telemetry data (defaults to synapseclient). Use a descriptive name so you can easily filter and analyze traces per service.
* `OTEL_DEBUG_CONSOLE`: Controls local visibility of telemetry data. Set to 'true' to output trace information to the console, which is useful for development and troubleshooting without an external collector.
* `OTEL_SERVICE_INSTANCE_ID`: Distinguishes between multiple instances of the same service (e.g., 'prod', 'development', 'local'). This helps identify which specific deployment or environment generated particular traces.

#### Enabling OpenTelemetry in your code
To enable OpenTelemetry with the Synapse Python client, simply call the
`enable_open_telemetry()` method on the Synapse class. Additionally you can access an
instance of the OpenTelemetry tracer via the `get_tracer()` call. This will allow you
to create new spans for your code.

```python
import synapseclient
from dotenv import load_dotenv

# Set environment variables
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://ingest.us.signoz.cloud"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "signoz-ingestion-key=<your key>"
os.environ["OTEL_SERVICE_NAME"] = "your-service-name"
os.environ["OTEL_SERVICE_INSTANCE_ID"] = "local"

# Enable OpenTelemetry with default settings
synapseclient.Synapse.enable_open_telemetry()
tracer = synapseclient.Synapse.get_tracer()

# Then create and use the Synapse client as usual
with tracer.start_as_current_span("my_function_span"):
    syn = synapseclient.Synapse()
    syn.login(authToken='auth_token')
```

#### Advanced Configuration

You can pass additional resource attributes to `enable_open_telemetry()`:

```python
import synapseclient

# Enable with custom resource attributes
synapseclient.Synapse.enable_open_telemetry(
    resource_attributes={
        "deployment.environment": "development",
        "service.version": "1.2.3", # Overrides the `OTEL_SERVICE_NAME` environment variable
        "service.instance.id": "4.5.6",  # Overrides the `OTEL_SERVICE_INSTANCE_ID` environment variable
        "custom.attribute": "value"
    }
)
```

When OpenTelemetry is enabled in the Synapse client, the following happens automatically:

1. Instrumentation is set up for:
   - **Threading** (via `ThreadingInstrumentor`): Ensures proper context propagation across threads, which is essential for maintaining trace continuity in multi-threaded applications
   - **HTTP libraries**:
     - `requests` (via `RequestsInstrumentor`): Captures all HTTP requests made using the requests library, including methods, URLs, status codes, and timing information
     - `httpx` (via `HTTPXClientInstrumentor`): Tracks both synchronous and asynchronous HTTP requests made with the httpx library
     - `urllib` (via `URLLibInstrumentor`): Monitors lower-level HTTP operations made directly with Python's standard library
   - Each instrumented HTTP library includes custom hooks that extract Synapse entity IDs from URLs when possible and add them as span attributes

2. Traces are configured to collect spans across your application:
   - Spans automatically capture operation duration, status, and errors.
   - An attribute propagation mechanism ensures that certain attributes (like `synapse.transfer.direction` and `synapse.operation.category`) are properly passed to child spans for uploads/downloads.
   - Trace data is exported via OTLP (OpenTelemetry Protocol).

3. Resource information is automatically added to your traces, including:
   - Python version
   - OS type
   - Synapse client version
   - Service name (defaults to "synapseclient" but can be customized via environment variables)
   - Service instance ID

Note that once enabled, OpenTelemetry cannot be disabled in the same process - you would need to restart your Python interpreter to disable it.


License and Copyright
---------------------

&copy; Copyright 2013-25 Sage Bionetworks

This software is licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0).
