Metadata-Version: 2.4
Name: nexus-client-sdk
Version: 1.5.5
Summary: Nexus SDK library for Python.
License: Apache-2.0
License-File: LICENSE
Author: ECCO Sneaks & Data
Author-email: esdsupport@ecco.com
Maintainer: ECCO Sneaks & Data
Maintainer-email: esdsupport@ecco.com
Requires-Python: >=3.11,<3.13
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: adapta[aws,datadog,storage] (>=3.5.23,<4.0.0)
Requires-Dist: dynaconf (>=3.2,<4.0)
Requires-Dist: injector (>=0.22.0,<0.23.0)
Project-URL: Repository, https://github.com/SneaksAndData/nexus-sdk-py
Description-Content-Type: text/markdown

# Introduction
Nexus SDK Py is a Python development kit for Nexus client applications. It builds upon [Golang Client](https://github.com/SneaksAndData/nexus-sdk-go) via `cgo`. 
Python SDK does not use any Python-level HTTP middleware for Nexus interactions, but authentication might require it.

SDK is tested against a Nexus stack in a `docker-compose` deployment, backed by `kind` Kubernetes clusters.

## Quickstart

Install CGO library from [Go SDK](https://github.com/SneaksAndData/nexus-sdk-go) by running:
```shell
chmod +x ./sdk-installer.sh
./sdk-installer.sh
```

In case you are testing changes for Go SDK, clone branch you are testing and compile the `.so` file from source:
```shell
go build -v -buildmode=c-shared -o nexus_sdk.so main.go
```

Afterwards, copy the `nexus_sdk.so` under `nexus_client_sdk/.extensions/nexus_sdk.so`. 

Initialize a client and retrieve results for a tagged submission:

```python
from nexus_client_sdk.models.access_token import AccessToken
from nexus_client_sdk.models.scheduler import SdkCustomRunConfiguration
from nexus_client_sdk.clients.nexus_scheduler_client import NexusSchedulerClient

token = "..."
client = NexusSchedulerClient.create("https://localhost:8080", lambda: AccessToken.empty())

alg_params = {"field1": {"field2": 1, "field3": "abc"}, "field4": "cde"}

# create a run
new_run = client.create_run(
    algorithm_parameters=alg_params,
    algorithm_name="test-algorithm",
    custom_configuration=SdkCustomRunConfiguration.create(version="v1.2.3"),
    tag="test-py-sdk",
    payload_valid_for="6h",
)

print(f"Run id: {new_run}")

for result in client.get_run_results("abc"):
    print(result)
```

## Nexus Development Framework

Apart from API clients for Nexus, SDK ships a development framework under `nexus` subpackage. It allows to create production-grade, `asyncio`-native ML/AI solutions that use a unified structure and are compose of objects and object relations, rather than methods. Nexus turns ML/AI apps into standard Python applications and removes the common noise found in notebook-
type code, such as variable reassignment, frequent data copying due to lack of reusable code, copy-paste of code etc. 

Nexus's design makes life even easier when using AI code generation, as it is essentially a framework an AI agent can follow to generate a working data science pipeline. Nexus takes care of result accounting, error handling, logging, metric reporting and, most importantly, *execution flow*. A key feature in Nexus is automatic resolution of execution graph via **dependency injection**.
In essence, a developer just needs to specify which inputs are required for an algorithm to run, and provide class implementations for this, and Nexus will take care of the rest. This also implies that whether an IO operation happens, such as a database read or a file load, Nexus will utilize `asyncio` coroutines to run multiple IO ops in parallel, significantly increasing the execution speed, without any need for a developer to understand async programming.

For a example of how to use Nexus, take a look at a [Sample Algorithm](tests/sample_algorithm) and a corresponding [test configuration](tests/conftest.py) and a [test](tests/test_sdk.py) itself.

### Execution tree

Nexus provides a set of utilities that allow viewing and inspecting the execution tree:

```python
from nexus_client_sdk.nexus.execution.trees import get_tree
from tests.sample_algorithm.sample_main import TestAlgorithm

print(get_tree(TestAlgorithm).serialize())

# graph TB
# TESTALGORITHM["TestAlgorithm"] --> XYPROCESSOR["XYProcessor"] --> XYREADER["XYReader"]
# TESTALGORITHM["TestAlgorithm"] --> ZPROCESSOR["ZProcessor"] --> ZREADER["ZReader"]
```

## Handling Compressed Payloads

Nexus supports reading compressed payloads for efficient data transfer. When a payload is compressed, it must include both the compressed content and a reference to the decompression function.

### Payload Structure

A compressed payload should be a json with the following keys:
- `content`: The compressed data (as a base64-encoded string).
- `decompressor_import_path`: The Python import path to the decompression function.

Example:
```python
{
    "content": "SGVsbG8gd29ybGQ=",  # base64-encoded string of compressed bytes
    "decompressor_import_path": "my_module.my_decompress"
}
```

When Nexus receives such a payload, it will:
1. Base64-decode the `content` field to obtain the compressed bytes.
2. Dynamically import and call the function specified by `decompressor_import_path` to decompress the payload.
3. Use the decompressed data as the actual payload for the algorithm.

This mechanism allows for flexible, pluggable decompression logic, as long as the function path is importable and callable in the runtime environment.


## Automatic Payload Compression

Nexus can automatically compress and decompress payloads when using `RemoteAlgorithm`. To use this feature, you must first configure it with environment variables and then explicitly enable it in your `RemoteAlgorithm` implementation.

### Step 1: Configuration (Environment Variables)

First, you need to provide the Python import paths for your compression and decompression logic. Setting these environment variables allows Nexus to create an injectable `Compressor` service.

  * `NEXUS__REMOTE_ALGORITHM__COMPRESSION_IMPORT_PATH`: The import path to your **compression** function (e.g., `my_module.my_compress`).
  * `NEXUS__REMOTE_ALGORITHM__DECOMPRESSION_IMPORT_PATH`: The import path to your **decompression** function (e.g., `my_module.my_decompress`).

### Step 2: Enabling Compression in Your Algorithm

Once the environment variables are set, you can activate compression on a `RemoteAlgorithm` instance by providing two arguments during its initialization:

1.  **`compress_payload=True`**: This boolean flag signals your intent to use compression for this remote algorithm.
2.  **`compressor=<injected_compressor_instance>`**: You must inject the `Compressor` service that Nexus creates from your environment variables.


### Important Requirement

For compression to work, both conditions must be met. The application will raise an error if `compress_payload` is set to `True` but a valid `Compressor` instance is not injected. Ensure that the required environment variables are set so the `Compressor` service can be created and injected successfully.


