# InceptBench REST API

FastAPI application for evaluating educational content via HTTP.

## Quick Start

```bash
# Install dependencies
pip install -r inceptbench_new/requirements.txt

# Configure environment
export OPENAI_API_KEY=your-key
export INCEPT_API_KEY=your-incept-api-key
export INCEPTBENCH_API_KEY=your-api-key
export GEMINI_API_KEY=your-gemini-key          # optional
export ANTHROPIC_API_KEY=your-anthropic-key    # optional

# Langfuse observability (optional — omit to disable tracing)
export LANGFUSE_SECRET_KEY=sk-lf-...           # optional
export LANGFUSE_PUBLIC_KEY=pk-lf-...           # optional
export LANGFUSE_BASE_URL=https://cloud.langfuse.com  # optional

# Start server
uvicorn inceptbench_new.api.main:app --reload
```

Access:

- API: http://localhost:8000
- Docs: http://localhost:8000/docs

## Legacy Version Support (Production API)

The production API at `api.inceptbench.com` uses the latest evaluator by default. For compatibility with existing integrations, previous versions are available via the `version` query parameter:

```bash
# Use legacy version (if needed for compatibility)
curl -X POST "https://api.inceptbench.com/evaluate?version=2.2.0" \
  -H "Authorization: Bearer $API_KEY" \
  -d '{"generated_content": [...]}'
```

See [docs/API_VERSIONING.md](../../docs/API_VERSIONING.md) for details.

**Note:** The local development server always uses the installed version.

## Endpoints

### `POST /evaluate`

Evaluate educational content.

**Query Parameters:**

| Parameter | Type   | Required | Default | Description                              |
| --------- | ------ | -------- | ------- | ---------------------------------------- |
| `version` | string | No       | Current | Legacy version for compatibility (optional) |

**Request:**

```json
{
  "generated_content": [
    {
      "id": "q1",
      "request": {
        "curriculum": "common_core",
        "grade": "7",
        "subject": "mathematics",
        "type": "mcq",
        "difficulty": "medium",
        "locale": "en-US",
        "skills": {
          "lesson_title": "Congruent and Similar Triangles",
          "substandard_id": "CCSS.MATH.CONTENT.7.G.A.1+3"
        },
        "instruction": "A real-world problem involving congruent and similar triangles"
      },
      "content": {
        "question": "What is x in 3x + 7 = 22?",
        "answer": "C",
        "options": ["3", "4", "5", "6"]
      }
    }
  ],
  "curriculum_version": "1.2"
}
```

**Top-level Fields:**

| Field               | Required | Default | Description                                    |
| ------------------- | -------- | ------- | ---------------------------------------------- |
| `generated_content` | Yes      | -       | Array of content items to evaluate (one or more) |
| `curriculum_version`| No       | Latest  | Curriculum version (e.g., "1.2")               |

**Content Item Fields:**

| Field        | Required | Default        | Description                          |
| ------------ | -------- | -------------- | ------------------------------------ |
| `content`    | Yes      | -              | Content to evaluate (string or JSON) |
| `id`         | No       | Auto-generated | Unique identifier                    |
| `request`    | No       | `null`         | Generation metadata (see below)      |

**Request Metadata Fields (all optional):**

| Field         | Description                                                                        |
| ------------- | ---------------------------------------------------------------------------------- |
| `curriculum`  | Curriculum for alignment (e.g., `common_core`, `ap_us_history`). Defaults to `common_core` |
| `grade`       | Grade level (e.g., "7", "K", "12")                                                |
| `subject`     | Subject area (e.g., "mathematics", "english")                                     |
| `type`        | Content type (e.g., "mcq", "fill-in", "article")                                  |
| `difficulty`  | Difficulty level (e.g., "easy", "medium", "hard")                                 |
| `locale`      | Locale/language code (e.g., "en-US", "es-MX")                                     |
| `skills`      | Skills info (JSON object or string)                                                |
| `instruction` | Generation instruction/prompt                                                      |

**Images in Content:**

Images are automatically detected from the content string (no separate field needed):

```json
{
  "content": "![triangle](https://example.com/triangle.png)\n\nWhat is the area?"
}
```

Supported: Direct URLs, Markdown `![](url)`, HTML `<img src="url">`, inline `<svg>` tags (requires Cairo system libraries).

**Response:**

```json
{
  "request_id": "uuid",
  "evaluations": {
    "q1": {
        "content_type": "question",
        "overall": {
          "score": 0.85,
          "reasoning": "...",
          "suggested_improvements": "..."
        },
      "factual_accuracy": { "score": 1.0, "reasoning": "..." },
      "educational_accuracy": { "score": 1.0, "reasoning": "..." },
      "weighted_score": 0.82
    }
  },
  "evaluation_time_seconds": 12.34,
  "inceptbench_version": "2.3.0",
  "curriculum_version": "1.2",
  "failed_items": null
}
```

**Partial Success:**

```json
{
  "evaluations": {"q1": {...}},
  "failed_items": [{"item_id": "q2", "error": "Evaluation failed: timeout"}]
}
```

### `GET /health`

```json
{
  "status": "healthy",
  "version": "2.1.0",
  "service": "Educational Content Evaluator"
}
```

### `GET /versions`

Returns available InceptBench versions that can be used via the `version` query parameter.

```json
{
  "current": "2.3.3",
  "supported_versions": ["2.3.3", "2.3.2", "2.3.1"],
  "default": "2.3.3"
}
```

**Using a specific version:**

```bash
curl -X POST "https://api.inceptbench.com/evaluate?version=2.3.1" \
  -H "Authorization: Bearer $API_KEY" \
  -d '{"generated_content": [...]}'
```

### `GET /curriculums`

Returns available curriculums with version information.

```json
{
  "default_curriculum": "common_core",
  "curricula": {
    "common_core": {
      "versions": [
        {"version": "1.2", "version_date": "2026-01-14"}
      ],
      "latest": "1.2",
      "description": "Common Core State Standards (CCSS)"
    }
  }
}
```

### `GET /`

API info and endpoint listing.

## Authentication

Bearer token authentication:

```bash
curl -H "Authorization: Bearer your-api-key" http://localhost:8000/evaluate
```

Configure API keys:

```bash
# Single key
export INCEPTBENCH_API_KEY=your-api-key

# Multiple keys
export INCEPTBENCH_API_KEY=key1,key2,key3
```

Generate a secure key:

```bash
python3 -c "import secrets; print(secrets.token_urlsafe(32))"
```

## Configuration

| Variable                     | Default  | Description                          |
| ---------------------------- | -------- | ------------------------------------ |
| `OPENAI_API_KEY`             | Required | OpenAI API key (powers evaluations)  |
| `INCEPT_API_KEY`             | Required | For curriculum search API            |
| `INCEPTBENCH_API_KEY`        | Required | API authentication key(s)            |
| `GEMINI_API_KEY`             | Optional | For image analysis                   |
| `ANTHROPIC_API_KEY`          | Optional | For image analysis (fallback)        |
| `MAX_CONCURRENT_EVALUATIONS` | `10`     | Max parallel evaluations per request |
| `LANGFUSE_SECRET_KEY`        | Optional | Enables [Langfuse](https://langfuse.com) LLM tracing |
| `LANGFUSE_PUBLIC_KEY`        | Optional | Langfuse project public key          |
| `LANGFUSE_BASE_URL`          | Optional | Langfuse server URL                  |
| `LANGFUSE_TRACING_ENVIRONMENT` | Optional | Environment tag (e.g. `production`) |

## Examples

**curl:**

```bash
# Simple evaluation
curl -X POST http://localhost:8000/evaluate \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-key" \
  -d '{"generated_content": [{"content": "What is 2+2?"}]}'

# With metadata
curl -X POST http://localhost:8000/evaluate \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-key" \
  -d '{
    "generated_content": [{
      "id": "q1",
      "request": {"curriculum": "common_core", "grade": "5", "subject": "math"},
      "content": {"question": "What is 2+2?", "answer": "4"}
    }]
  }'

# With specific curriculum version
curl -X POST http://localhost:8000/evaluate \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-key" \
  -d '{
    "generated_content": [{"content": "What is 2+2?"}],
    "curriculum_version": "1.2"
  }'

# Batch
curl -X POST http://localhost:8000/evaluate \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-key" \
  -d '{"generated_content": [{"content": "Q1"}, {"content": "Q2"}]}'
```

**Python:**

```python
import requests

response = requests.post(
    "http://localhost:8000/evaluate",
    headers={"Authorization": "Bearer your-api-key"},
    json={"generated_content": [{"content": "What is 2+2?"}]}
)

result = response.json()
evaluation = result["evaluations"][list(result["evaluations"].keys())[0]]
print(f"Score: {evaluation['overall']['score']:.2f}")
```

## Deployment

### Docker

```bash
docker-compose up --build
```

### Google Cloud Run

```bash
./deploy_to_cloudrun.sh
```

Options:

| Flag              | Default           | Description             |
| ----------------- | ----------------- | ----------------------- |
| `--project`       | -                 | Google Cloud project ID |
| `--region`        | `us-central1`     | Deployment region       |
| `--service`       | `inceptbench-api` | Service name            |
| `--memory`        | `2Gi`             | Memory allocation       |
| `--cpu`           | `2`               | CPU allocation          |
| `--min-instances` | `0`               | Minimum instances       |
| `--max-instances` | `10`              | Maximum instances       |

```bash
./deploy_to_cloudrun.sh --memory 4Gi --cpu 4 --min-instances 1
```

## AP Evaluation

Specialized evaluation for AP Social Studies questions (APUSH, APWH, APEH, APGOV, APHG).

**Curriculums:** `ap_us_history`, `ap_world_history`, `ap_european_history`, `ap_government`, `ap_human_geography`

**Question types:** `mcq`, `mcq_set`, `saq`, `leq`, `dbq`

**Pass criteria:** No hard-fail dimension failures, plus score logic aligned to AP evaluator thresholds (`PASS_THRESHOLD=0.85`, with borderline handling between `0.80` and `0.90`).

See [docs/AP_EVALUATION.md](../../docs/AP_EVALUATION.md) for input format and examples.

## Limits

- Maximum **100 items** per request
- Parallel processing with configurable concurrency (default: 10)
- Failed items don't block successful evaluations

## Error Codes

| Status | Description                                         |
| ------ | --------------------------------------------------- |
| 200    | Success (check `failed_items` for partial failures) |
| 400    | Invalid input                                       |
| 401    | Missing/invalid API key                             |
| 422    | Schema validation failed                            |
| 500    | Internal server error                               |
| 503    | Service unavailable (API keys not configured)       |
