Metadata-Version: 2.4
Name: dgxarley
Version: 0.0.20
Summary: Integration testing, streaming utilities, and repetition detection for distributed LLM inference on DGX Spark clusters
Project-URL: Homepage, https://github.com/vroomfondel/dgxarley
Project-URL: Repository, https://github.com/vroomfondel/dgxarley
Project-URL: Issues, https://github.com/vroomfondel/dgxarley/issues
Author-email: Henning Thieß <ht@xomox.cc>
License-File: LICENSE.md
Keywords: dgx,inference,k3s,llm,ollama,openwebui,repetition-detection,sglang,streaming
Classifier: Development Status :: 4 - Beta
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Distributed Computing
Requires-Python: >=3.14
Requires-Dist: aiohttp
Requires-Dist: ascii-magic
Requires-Dist: loguru
Requires-Dist: pillow
Requires-Dist: pydantic
Requires-Dist: pyserial
Requires-Dist: pyyaml
Requires-Dist: requests
Requires-Dist: rich
Requires-Dist: tabulate
Provides-Extra: ansible
Requires-Dist: ansible; extra == 'ansible'
Requires-Dist: ansible-pylibssh; extra == 'ansible'
Requires-Dist: dnspython; extra == 'ansible'
Requires-Dist: huggingface-hub; extra == 'ansible'
Requires-Dist: jinja2; extra == 'ansible'
Requires-Dist: kubernetes; extra == 'ansible'
Requires-Dist: paramiko; extra == 'ansible'
Requires-Dist: pygithub; extra == 'ansible'
Provides-Extra: build
Requires-Dist: hatch; extra == 'build'
Provides-Extra: dev
Requires-Dist: black; extra == 'dev'
Requires-Dist: isort; extra == 'dev'
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: sglang; extra == 'dev'
Requires-Dist: types-pyserial; extra == 'dev'
Requires-Dist: types-pyyaml; extra == 'dev'
Requires-Dist: types-requests; extra == 'dev'
Provides-Extra: web
Requires-Dist: fastapi; extra == 'web'
Requires-Dist: uvicorn[standard]; extra == 'web'
Description-Content-Type: text/markdown

[![mypy and pytests](https://github.com/vroomfondel/dgxarley/actions/workflows/mypynpytests.yml/badge.svg)](https://github.com/vroomfondel/dgxarley/actions/workflows/mypynpytests.yml)
[![black-lint](https://github.com/vroomfondel/dgxarley/actions/workflows/checkblack.yml/badge.svg)](https://github.com/vroomfondel/dgxarley/actions/workflows/checkblack.yml)
[![Cumulative Clones](https://img.shields.io/endpoint?logo=github&url=https://gist.githubusercontent.com/vroomfondel/3e9b5513788bd4a8af48560724376534/raw/dgxarley_clone_count.json)](https://github.com/vroomfondel/dgxarley)


# dgxarley

Tooling for the DGX Arley K3s inference cluster — integration tests, streaming utilities, and CLI entry points for SGLang, Ollama, and OpenWebUI services.


**Heureka! — Qwen3-235B-A22B MoE (AWQ 4-bit) running distributed inference across both DGX Sparks:**

![235B AWQ Heureka](https://raw.githubusercontent.com/vroomfondel/dgxarley/main/media/Bildschirmfoto_2026-03-17_21-53-58_blurred.png)


### `sglang-raw` — Dual-panel SSE stream viewer

![sglang-raw: rendered response + raw JSON chunks](https://raw.githubusercontent.com/vroomfondel/dgxarley/main/media/Bildschirmfoto_2026-03-18_17-29-30_blurred.png)

Dual-panel Rich TUI for inspecting SGLang's OpenAI-compatible streaming API in real time. The top half renders the AI response as it arrives, while the bottom half displays the raw JSON SSE stream chunks — showing fields like `chat_completion_chunk`, `choices`, `delta`, `finish_reason`, and `model`. Useful for debugging streaming behaviour, verifying token delivery, and understanding the wire format of the API.

### `sglang-raw` — Think/text token classification

![sglang-raw: token table with think/text classification](https://raw.githubusercontent.com/vroomfondel/dgxarley/main/media/Bildschirmfoto_2026-03-18_17-35-04_blurred.png)

Token-level stream inspection with per-chunk breakdown in a structured table. Columns show the token type (`think` vs `text`), content, finish reason, and cumulative token count — visualizing how reasoning tokens (from `<think>...</think>` blocks) are separated from the actual output tokens. This view helps when tuning thinking budgets, verifying `reasoning_parser` behaviour, or diagnosing unexpected token classification.

## What's included

### CLI tools

| Command | Description |
|---------|-------------|
| `sglang-raw` | Interactive SSE stream viewer with dual-panel Rich display (interpreted output + raw JSON chunks) |
| `sglang-test` | Direct SGLang client with sequential and parallel load testing (live Rich TUI) |
| `openwebui-test` | OpenWebUI / LLM client with preset management and streaming |
| `ollama-test` | Ollama API health, model, embedding, and chat completions tests |
| `kceve-kvm` | RS232 serial control for KCEVE KVM1001A 10-port KVM switches (switch ports, query state, sniff) |
| `kceve-kvm-web` | Web UI for KCEVE KVM control (FastAPI, requires `dgxarley[web]`) |
| `kceve-kvm-web-plain` | Lightweight web UI for KCEVE KVM control (stdlib `http.server`, no extra dependencies) |

### `kceve-kvm-web` — KCEVE KVM1001A Web UI

![kceve-kvm-web: KVM switch control via browser](https://raw.githubusercontent.com/vroomfondel/dgxarley/main/media/Bildschirmfoto_2026-04-04_17-09-49.png)

Browser-based control panel for the KCEVE KVM1001A 10-port KVM switch via RS232 serial. Shows the currently active input port on a virtual 7-segment display and allows switching between all 10 inputs with a single click. Commands are sent over a USB-to-RS232 adapter at 115200 baud using the `X<channel>,1$` ASCII protocol.

[Demo video](https://raw.githubusercontent.com/vroomfondel/dgxarley/main/media/simplescreenrecorder-2026-04-04_17.10.24.mp4)

<details>
<summary>Remote test setup (serial over SSH tunnel)</summary>

If the KVM is connected to a remote host, you can tunnel the serial port via TCP:

```bash
# 1. On k3smaster (where the USB-RS232 adapter is connected): expose serial as TCP server
root@k3smaster ~ # socat tcp-listen:7000,reuseaddr,fork /dev/ttyACM0,b115200,raw,echo=0

# 2. On workstation: SSH tunnel to remote TCP port
user@workstation ~ $ ssh -N -L 7000:127.0.0.1:7000 root@k3smaster &

# 3. On workstation: create local PTY from TCP tunnel
user@workstation ~ $ socat pty,link=/tmp/kvm-serial,raw,echo=0 tcp:127.0.0.1:7000 &

# 4. On workstation: start web UI on the local PTY
user@workstation ~ $ kceve-kvm-web -d /tmp/kvm-serial -p 8080
```

Then open `http://localhost:8080` in a browser.
</details>

### Libraries

| Module | Description |
|--------|-------------|
| `dgxarley.integration.repetition_detector` | Offline n-gram, sentence, and loop repetition analysis for completed LLM outputs |
| `dgxarley.integration.streaming_repetition_guard` | Real-time repetition detection for token streams with configurable thresholds |

## Installation

```bash
pip install dgxarley
```

## Quick start

```python
from dgxarley.integration.repetition_detector import detect_repetition

report = detect_repetition(llm_output)
print(report.summary())
# [LOW] score=0.12 — N-Gram 'this is a test' x2
```

```python
from dgxarley.integration.streaming_repetition_guard import RepetitionGuard

guard = RepetitionGuard()
for chunk in llm_stream:
    token = chunk.choices[0].delta.content or ""
    result = guard.feed(token)
    if result.should_stop:
        print(f"STOP: {result.reason}")
        break
```

## Requirements

- Python >= 3.14

## Source & documentation

Full documentation, network architecture, and Ansible playbooks: [GitHub](https://github.com/vroomfondel/dgxarley)

## License
This project is licensed under the LGPL where applicable/possible — see [LICENSE.md](https://github.com/vroomfondel/dgxarley/blob/main/LICENSE.md). Some files/parts may use other licenses: [MIT](https://github.com/vroomfondel/dgxarley/blob/main/LICENSEMIT.md) | [GPL](https://github.com/vroomfondel/dgxarley/blob/main/LICENSEGPL.md) | [LGPL](https://github.com/vroomfondel/dgxarley/blob/main/LICENSELGPL.md). Always check per‑file headers/comments.


## Authors
- Repo owner (primary author)
- Additional attributions are noted inline in code comments


## Acknowledgments
- Inspirations and snippets are referenced in code comments where appropriate.


## ⚠️ Note

This is a development/experimental project. For production use, review security settings, customize configurations, and test thoroughly in your environment. Provided "as is" without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the software. Use at your own risk.


