Metadata-Version: 2.4
Name: cist-kernel
Version: 0.0.43
Summary: CLI for cist kernel telemetry and compilation
Author-email: tooig research lab <oyebamijo@tooig.com>
License: tooig software license
Requires-Python: >=3.9
Requires-Dist: pydantic>=2.0.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: requests>=2.31.0
Requires-Dist: rich>=13.0.0
Requires-Dist: typer>=0.9.0
Provides-Extra: dev
Requires-Dist: black>=23.0.0; extra == 'dev'
Requires-Dist: build>=0.10.0; extra == 'dev'
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Description-Content-Type: text/markdown

# CIST Monorepo

Polyglot monorepo for the Contract Instruction Set (`.cis`) language.

This repository ships three user-facing pieces together:

- `cist-kernel` Python package — the CLI, the compiler, and the cloud compilation client.
- A VS Code extension bundled inside the package — `cist init` installs it into your local VS Code extensions folder. Recognises both `.cis` source files and `.mex` compiled binaries.
- A build orchestrator (`scripts/build.py`) for cutting releases.

## Language Snapshot

```cis
use math
use compute
use risk

market_data := from data (source: "binance", symbols: ["BTC-USD"], resolution: "1m")

sma_20 := math.sma(market_data.close, 20)
signal := compute.crossover(market_data.price, sma_20)

if signal and position.size == 0:
	execute(market_buy(size: risk.calculate_size(portfolio_value: 10000)))
elif position.pnl < -100:
	execute(close_position())
```

## Repository Layout

```text
cist/
├── pyproject.toml
├── cist.toml                   ← workspace config (created by cist init)
├── README.md
├── .gitignore                  ← *.mex and .env excluded by default
├── scripts/
│   └── build.py
├── cist_cli/
│   ├── __init__.py
│   ├── main.py
│   ├── network.py
│   ├── compiler/               ← NEW: compilation pipeline
│   │   ├── __init__.py
│   │   ├── lexer.py            ← CISTCompiler: syntax + semantic validation → IR
│   │   └── mex.py              ← MexWriter / MexReader: .mex binary format
│   ├── workspace/              ← NEW: workspace scaffold
│   │   ├── __init__.py
│   │   ├── init.py             ← init_workspace(): creates directory structure
│   │   └── templates.py        ← all generated-file templates
│   └── assets/
│       └── *.vsix              ← bundled VS Code extension
├── tests/
│   ├── test_compiler.py
│   ├── test_mex.py
│   ├── test_workspace.py
│   ├── test_entrypoint.py
│   ├── test_build_script.py
│   └── test_vscode_grammar.py
└── extensions/
    ├── package.json            ← declares .cis and .mex language contributions
    ├── language-configuration.json
    └── syntaxes/
        └── cis.tmLanguage.json
```

Each contract lives in its own sub-directory under `contracts/`:

```text
contracts/
└── {contract_name}/
    ├── {contract_name}.cis     ← main source file
    ├── program.mex             ← compiled binary (created by cist compile)
    └── bench/
        └── example.cis         ← test/bench scenarios (extend freely)
```

## Getting Started

### 1. Install or upgrade

```bash
python -m pip install --upgrade cist-kernel
```

The `--upgrade` flag matters. A plain `pip install cist-kernel` keeps an existing older version and will not pull in new features.

If `cist` is not on your `PATH` after install, use the fallback:

```bash
python -m cist_cli
```

On Windows, the installer will print the exact `PowerShell` one-liner to add the scripts folder to your `PATH` permanently the first time you run via `python -m cist_cli`.

### 2. Initialise a workspace

```bash
cist init              # in the current directory
cist init --path ~/my-workspace
```

`cist init` creates the following structure from scratch (safe to re-run — existing files are never overwritten):

```text
<workspace>/
├── cist.toml                    project name, kernel version, compiler settings
├── .env                         CIST_CLOUD_API_KEY placeholder
├── .gitignore                   *.mex and .env pre-seeded
└── contracts/
    └── example/
        ├── example.cis          starter contract (edit freely)
        └── bench/
            └── example.cis      bench / test scenarios (extend freely)
```

It also silently extracts the bundled VS Code extension into every detected editor extensions folder (`.vscode`, `.vscode-insiders`, `.antigravity`), removing any stale legacy versions. Reload the window once with `Ctrl+Shift+P → Developer: Reload Window` to activate syntax highlighting for `.cis` and `.mex` files.

### 3. Add your API key

Open the `.env` file and fill in your key:

```bash
CIST_CLOUD_API_KEY=sk_live_your_key_here
```

### 4. Write your contract

Edit `contracts/example/example.cis` or create a new sub-directory:

```cis
use math
use hyper

contract OilPriceMonitor:
    which defines:
        while True:
            oil_price = hyper.get_price("OIL")

            then:
                if oil_price > 80.0:
                    print("Consider selling.")
```

Add test scenarios in `contracts/example/bench/`:

```cis
contract BenchHighPrice:
    which defines:
        oil_price = 90.0
        then:
            if oil_price > 80.0:
                print("bench: high price path OK")
```

### 5. Compile

```bash
cist compile contracts/example/example.cis
```

What happens:

1. **Syntax check** — the compiler validates brackets, strings, `use` imports, `from data(…)` expressions, indentation, and scans `contract` and `write fn` declarations.
2. **Binary emit** — if the file is valid, `program.mex` is written next to the source file. This binary contains the compressed IR; the raw source is never stored inside it, and the file is excluded from version control via `.gitignore`.
3. **Cloud stream** — the IR is streamed to the CIST Cloud compiler. Logs, artifact IDs, and errors are printed as they arrive.

```text
✓ example.cis  42 lines · modules: math, hyper · 0 data stream(s)
✓ Binary written → contracts/example/program.mex
Connecting to CIST Cloud...
```

Add `--deploy` to mark the build for live execution:

```bash
cist compile contracts/example/example.cis --deploy
```

Add `--verbose` to see the full module and data-stream breakdown:

```bash
cist compile contracts/example/example.cis --verbose
```

---

## CLI Reference

### `cist init [--path <dir>]`

Initialises a CIST workspace.

- Creates `contracts/{name}/{name}.cis` and `contracts/{name}/bench/example.cis`.
- Writes `cist.toml` (project name, kernel version, compiler settings).
- Writes `.env` stub with `CIST_CLOUD_API_KEY` placeholder.
- Seeds `.gitignore` with `*.mex` and `.env` (appends if the file already exists; never duplicates).
- Removes stale legacy extension versions and silently extracts the bundled `.cis` + `.mex` VS Code extension.

### `cist compile <file.cis> [--deploy] [--verbose]`

Compiles a `.cis` contract.

- **Step 1 — local validation**: bracket matching, quote closure, `use` imports, `from data(…)` expressions, mixed tab/space indentation, UTF-8 encoding.
- **Step 2 — binary emit**: writes a `program.mex` binary alongside the source. The binary is zlib-compressed, integrity-checked with CRC-32, and does not contain the original source text.
- **Step 3 — cloud stream**: streams the compiled IR to the CIST Cloud. Logs, artifact IDs, and errors are printed in real time.
- `--deploy` flags the build as live.
- `--verbose` prints the full module list and data-stream count.

### `cist sync-vscode-extension`

Re-installs or updates the bundled VS Code extension without running a full `init`.

---

## The `.mex` Binary Format

When `cist compile` succeeds it writes a `program.mex` file alongside the source. The format is:

| Offset | Size | Content |
|--------|------|---------|
| 0 | 4 B | Magic: `MEX\x00` |
| 4 | 2 B | Format version (big-endian uint16, currently `1`) |
| 6 | 4 B | CRC-32 of the compressed payload (big-endian uint32) |
| 10 | N B | zlib-compressed UTF-8 JSON payload |

The payload holds the compiler's IR (module list, contract names, function names, data stream count, line count, warnings). The original source text is stripped before writing.

`.mex` files are recognised by the VS Code extension (file-tree icon, language ID `mex`) but are intentionally non-editable binary blobs. They are excluded from version control via the workspace `.gitignore`.

---

## Configuration

### `cist.toml`

Created by `cist init` at the workspace root. Controls project metadata and compiler behaviour:

```toml
[project]
name = "example"
version = "0.1.0"
description = ""

[kernel]
version = "0.0.38"   # kernel version this workspace was initialised with

[compiler]
strict = false        # set true to treat warnings as errors
```

### Environment Variables

Loaded from `.env` (if present) and from the shell environment.

| Variable | Required | Default | Purpose |
|----------|----------|---------|---------|
| `CIST_CLOUD_API_KEY` | Yes (for compile) | — | Bearer token for the cloud compiler |
| `CIST_CLOUD_ENDPOINT` | No | `https://api.cist.io/v1` | Override the cloud endpoint |
| `CIST_SKIP_VSCODE_INSTALL` | No | — | Set to `1` to skip extension install (useful in CI) |
| `VSCODE_EXTENSIONS` | No | — | Override the VS Code extensions root directory |

---

## Build Artifacts

Build both deliverables with:

```bash
python scripts/build.py
```

Expected outputs:

- `dist/cist_kernel-*.whl` — Python wheel containing the CLI and the bundled `.vsix`.

The build script verifies that the VSIX version inside the wheel matches the project version in `pyproject.toml` before declaring success.

---

## Notes

- Local validation is lightweight and non-executing. It never runs contract code.
- Cloud compilation requires `CIST_CLOUD_API_KEY`. The `.env` file is loaded automatically.
- The VS Code grammar uses Python-style scope categories with CIST-specific handling for `use`, `contract`, `write fn`, `which defines:`, `then:`, and `from data(…)`.
- The `cist_cli/compiler/` and `cist_cli/workspace/` packages are independently importable for use in tooling or test fixtures.
