Metadata-Version: 2.4
Name: mobilint-qb-runtime
Version: 1.1.0
Summary: Runtime library for Mobilint NPUs
Author-email: "Mobilint Inc." <tech-support@mobilint.com>
Project-URL: Home, https://www.mobilint.com/
Keywords: NPU,inference,mobilint,mblt,aries,regulus
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: System Administrators
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.19.0

# Mobilint SDK qb Runtime Library : qbruntime

**qbruntime** is runtime library for Mobilint NPU products. Runtime Library is required to run inference on Mobilint NPUs using precompiled or custom models.

> **IMPORTANT**: Please refer to [official documentation](https://docs.mobilint.com) for installing driver before using runtime library.

## Prerequisites

Currently, pip installation is limited to below Ubuntu environments. For other environment, please refer to [official documentation](https://docs.mobilint.com).

- Ubuntu: 20.04 / 22.04 / 24.04
- Python: 3.8 / 3.9 / 3.10 / 3.11 / 3.12
- numpy

## Installation

You may install python runtime library with following command

```bash
pip install mobilint-qb-runtime
```

For C++ library installation or downloading library from [mobilint distribution site](https://dl.mobilint.com), please refer to [official documentation](https://docs.mobilint.com).

## Inference process

Running inference on Mobilint's NPU using the `qbruntime` runtime library involves four essential steps:

1. Initialize the NPU device.
2. Load the compiled model (MXQ file).
3. Upload model to the NPU device.
4. Run inference with user input.

### Step 1: `Accelerator`

The `Accelerator` object represents the NPU device to be used. It abstracts a single device identified by the number appended to the device name (e.g., `/dev/aries0` -> `acc1`, `/dev/aries2` -> `acc2`). If no specific number is provided, NPU device `0` is used by default.

```python
acc = qbruntime.Accelerator(0)
```

### Step 2: `Model`

The `Model` object represents a model contained in an MXQ file. Upon creation, it immediately reads the MXQ file and stores the relevant information.

```python
model = qbruntime.Model(MXQ_FILE_PATH)
```

### Step 3: Pass `Model` information to the `Accelerator`

Pass the information with the {doxylink}`mobilint::Model::launch(Accelerator & acc)` method of `Model` object. It then uses the `Accelerator` object to run inference using this model.

### Step 4: Run inference using passed `Model` information and input data

Get the input data and run inference using the `mobilint::Model::infer()` method of `Model` object.

```python
model = qbruntime.Model(MXQ_FILE_PATH)
result = model.infer([INPUT])
```

### Inference Scenario

```python
## Python example
import qbruntime

MXQ_PATH = "path/to/mxq.mxq"

acc = qbruntime.Accelerator()               ## Step 1
model = qbruntime.Model(MXQ_PATH)           ## Step 2
model.launch(acc)                           ## Step 3

## Some preprocessing for input data

result = model.infer(preprocessed_input)    ## Step 4
```
