Metadata-Version: 2.4
Name: arbis-llmwrap
Version: 0.1.0
Summary: Decorator to wrap LLM calls with hidden prompt formatting, Steps/Answer parsing, CSV logging, and retry on parse failure.
License: MIT
Keywords: llm,decorator,prompt,logging,cython
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: cython>=3.0; extra == "dev"
Requires-Dist: wheel; extra == "dev"
Dynamic: license-file

# llmwrap

**llmwrap** is a small Python library that wraps your LLM-calling function with a **hidden** prompt (so the model is instructed to reply in a fixed format), parses the reply into **Steps** and **Answer**, logs every attempt to a CSV file, and returns **only the Answer** to you. If parsing fails, it retries by asking the model to reformat its previous response (up to `max_tries` attempts). The core logic is compiled into a binary extension (Cython); it is not shipped as plain Python source in the wheels.

## Install

```bash
pip install arbis-llmwrap
```

## Usage

Decorate your LLM function so that it receives the **wrapped** prompt and so that the library parses the response and returns only the **Answer**:

```python
from llmwrap import wrap_llm_call

@wrap_llm_call(log_path="runs.csv", max_tries=3)
def user_llm(prompt: str) -> str:
    # Call your LLM (e.g. OpenAI, Anthropic, local model). You receive
    # the internally wrapped prompt; return the raw string response.
    response = some_client.chat(prompt)
    return response

# You get back only the parsed "Answer" part.
answer = user_llm("What is 2+2?")
```

- **Prompt wrapping**: Your function receives a prompt that already includes the instructions (Steps + Answer format). You must not provide this wrapper yourself; it is applied inside the library.
- **Logging**: Each call and each retry is logged as one row in the CSV (see columns below).
- **Retry**: If the parser does not find a valid "Answer" (or it’s empty), the library calls your function again with a “reformat your previous response” prompt, up to `max_tries` times.
- **Fallback**: If all attempts fail, the library returns the last non-empty parsed answer if any, otherwise the last raw output.

## Log file (CSV)

The CSV has exactly these columns:

| Column           | Description                                      |
|------------------|--------------------------------------------------|
| `id`             | Unique call id (same for all attempts of one call) |
| `timestamp`      | Unix timestamp                                  |
| `attempt`        | Attempt number (1, 2, …)                         |
| `latency_ms`     | Time for this attempt in milliseconds            |
| `prompt`         | Original user prompt                             |
| `wrapped_prompt` | Full prompt sent to your LLM function             |
| `raw_output`     | Entire raw LLM output for this attempt           |
| `parsed_steps`   | Extracted “Steps” section                        |
| `parsed_answer`  | Extracted “Answer” section                       |
| `parse_success`  | Whether parsing succeeded (true/false)           |

The file is append-only. The directory for `log_path` is created if it does not exist.

## Parsing (deterministic)

Parsing is done with regexes; no LLM is used to parse.

- **Answer labels**: `Answer:`, `Final Answer:`, `Final:` (case-insensitive).
- **Steps labels**: `Steps:`, `Reasoning:`, `Explanation:` (case-insensitive).
- Common markdown (e.g. `#` headings, `**bold**`) is stripped before parsing.
- If there are multiple “Answer” labels, the **last** one is used.

## Retry behaviour

1. Your function is called with the wrapped prompt.
2. The raw output is logged and parsed. If an “Answer” is found and valid, it is returned and we stop.
3. If not, your function is called again with a reformat prompt (same call id; new attempt number and row in the CSV), up to `max_tries` times.
4. If all attempts fail, the library returns the last non-empty parsed answer if any, else the last raw output.

## Binary wheels (no plain-Python core)

The core implementation is compiled with Cython into a binary extension (`.so` on Linux/macOS, `.pyd` on Windows). When you install from a wheel, you do not get the core logic as readable `.py`; only the small public stub in `llmwrap/__init__.py` and the compiled module are shipped.

## Release steps

1. **Bump version** in `src/llmwrap/_version.py` and in `pyproject.toml` (keep them in sync).
2. **Commit and push**, then create a tag, e.g. `v0.1.1`.
3. **CI** (GitHub Actions) builds wheels for Windows, macOS, and Linux (CPython 3.9–3.12) and uploads them as workflow artifacts.
4. **Publish**:
   - **TestPyPI**: download the wheel artifacts from the workflow run (or build locally with cibuildwheel), then e.g. `twine upload --repository testpypi dist/*.whl` (use `TWINE_USERNAME=__token__` and `TWINE_PASSWORD=<TestPyPI token>`).
   - **PyPI**: same with `twine upload dist/*.whl` and a PyPI API token, or use **Trusted Publishing** (OIDC) in the same workflow so no long-lived token is needed.

### Local development install

```bash
pip install -e ".[dev]"
```

(Add a `[dev]` optional dependency in `pyproject.toml` if you use one; otherwise `pip install -e .` is enough.) Requires a working C compiler and Cython to build the extension.

### Local build (best-effort)

From the repo root, with Cython and a compiler installed:

```bash
pip install Cython wheel setuptools
pip wheel . --no-deps -w dist
```

This produces a wheel for your current platform and Python version. For many platforms and Python versions, use **cibuildwheel** (as in CI):

```bash
pip install cibuildwheel
python -m cibuildwheel --output-dir dist
```

### CI build for all OSes

Push a tag (e.g. `v0.1.0`) or run the “Build wheels” workflow manually. The workflow runs cibuildwheel on `ubuntu-latest`, `windows-latest`, and `macos-latest` with `CIBW_BUILD="cp39-* cp310-* cp311-* cp312-*"` and `CIBW_SKIP="pp*"`, and uploads the wheels as artifacts. Download the artifacts and upload to (Test)PyPI with twine, or add the optional publish job (see the commented section in `.github/workflows/build_wheels.yml`).

## License

MIT. See [LICENSE](LICENSE).
