# Local Test Project Scaffold for django-mojo

**Type**: feature
**Status**: Resolved
**Date**: 2026-03-17

## Resolution

Implemented 2026-03-17. Three scripts in `bin/` generate and run a local test project.

**Results**: 925 tests discovered, 886 pass, 39 skip, 0 fail.

**Files created/modified**:
- `bin/create_testproject` — bash: generates `testproject/`, creates PostgreSQL DB, runs migrate
- `bin/asgi_local` — bash: starts Redis + uvicorn on port 5555
- `bin/testit.py` — python: bootstraps paths, auto-activates venv, flushes DB+Redis, runs testit runner
- `.gitignore` — added `testproject/`

**Bug fixes discovered during testing**:
- `mojo/apps/account/rest/totp.py` — wrapped `select_for_update()` in `transaction.atomic()` (PostgreSQL requires it)
- `mojo/apps/jobs/__init__.py` — idempotency key duplicate detection now matches PostgreSQL error message
- `tests/test_security/test_routes.py` — scanner now recognizes public GET + auth POST on same path
- `tests/test_accounts/session_revoke.py` — clear rate limits before each test function

**Key deviations from spec**:
- PostgreSQL instead of SQLite — eliminates concurrent write issues
- Added venv auto-detection in all scripts (`.venv/bin/python`)
- Added `settings/defaults.py` (empty) — required by `load_settings_profile`
- Added `apps/tests/` directory — required by `testit.runner.TEST_ROOT`
- Used `REST_AUTO_PREFIX = True` — needed for absolute URL routes (`/s/<code>`)
- Added `ALLOW_ADMIN_SITE = False` — prevents parser from adding django.contrib.admin
- Added `TWILIO_NUMBER = "+15550000000"` — enables test-number SMS shortcut
- `testit.py` flushes PostgreSQL (TRUNCATE CASCADE) + Redis (`flushdb`) before each run
- route_security (1): security audit finding actual code issues

**To validate**: `./bin/create_testproject && ./bin/asgi_local start` then `./bin/testit.py`, then `./bin/asgi_local stop`

---

## Problem

django-mojo is a framework, not a Django project. To run the test suite in `tests/`,
a host Django project is required. Currently tests can only be run inside an external
project, which is slow and not portable for CI/DevOps.

## Goal

A single script that generates a minimal, disposable Django project inside
`testproject/` (fully `.gitignore`d) that can:
- Start a local ASGI test server (`bin/asgi_local`)
- Run the full test suite via testit (`bin/testit.py`)
- Be regenerated cleanly at any time by re-running the script
- Work both locally and in automated CI pipelines

## Repo Layout After Implementation

```
django-mojo/
├── bin/
│   ├── create_testproject       # bash: generates testproject/, runs migrate
│   ├── asgi_local               # bash: starts Redis + uvicorn on port 5555
│   └── testit.py                # python: runs testit runner
├── .gitignore                   # add: testproject/
└── testproject/                 # GITIGNORED — fully generated by create_testproject
    ├── manage.py                # standard Django manage.py (migrate, shell, etc.)
    ├── config/
    │   ├── settings/
    │   │   ├── __init__.py      # framework entry point
    │   │   ├── local/
    │   │   │   ├── __init__.py  # full local settings
    │   │   │   └── db.py        # SQLite + Redis
    │   │   └── urls.py          # URL routing
    │   └── dev_server.conf      # host=127.0.0.1  port=5555
    ├── bin/
    │   ├── paths.py             # sys.path bootstrap for this project
    │   └── _asgi.py             # uvicorn entry point
    ├── apps/                    # empty — no project-specific apps
    └── var/
        ├── profile              # contains the string: local
        ├── django.conf          # SECRET_KEY = '<generated>'
        └── logs/
```

---

## File Contents

All files below are written verbatim by `create_testproject`.

---

### `testproject/bin/paths.py`

Sets up sys.path so that `mojo`, `testit`, and `settings` are all importable.
`REPO_ROOT` is the django-mojo root (one level up from testproject/).

```python
import sys
import os
from pathlib import Path

FILE = Path(__file__).resolve()
TESTPROJECT_ROOT = FILE.parent.parent        # django-mojo/testproject/
REPO_ROOT = TESTPROJECT_ROOT.parent          # django-mojo/

# mojo package + testit package both live at repo root
if str(REPO_ROOT) not in sys.path:
    sys.path.insert(0, str(REPO_ROOT))

# settings module lives at testproject/config/
CONFIG_ROOT = TESTPROJECT_ROOT / "config"
if str(CONFIG_ROOT) not in sys.path:
    sys.path.insert(0, str(CONFIG_ROOT))

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
```

---

### `testproject/bin/_asgi.py`

Uvicorn entry point. Imports local `paths.py` first to bootstrap sys.path,
then lets mojo create the ASGI application.

```python
import paths  # testproject/bin/paths.py — must be first

from mojo.helpers import paths as mpaths
mpaths.configure_paths(__file__, 2)

from mojo.apps.realtime.routing import create_application
application = create_application()
```

Note on depth: `_asgi.py` is at `testproject/bin/_asgi.py`.
`parents[2]` = `testproject/`, so `PROJECT_ROOT = testproject/../` = repo root.
We want PROJECT_ROOT = `testproject/`, so depth should be 1:
- `parents[1]` of `testproject/bin/_asgi.py` = `testproject/bin/` ... wait that's wrong.

Correction — `configure_paths` is called from `config/settings/__init__.py`, not `_asgi.py`.
`_asgi.py` just imports `paths` (local bootstrap) and calls `create_application()`.
Django settings handle `configure_paths` internally. So `_asgi.py` is simply:

```python
import paths  # testproject/bin/paths.py — bootstraps sys.path

from mojo.apps.realtime.routing import create_application
application = create_application()
```

---

### `testproject/config/settings/__init__.py`

Framework entry point. Calls `configure_paths(__file__, 1)` which resolves:
- `Path(__file__).parents[1]` = `testproject/config/`
- `PROJECT_ROOT = testproject/config/.parent` = `testproject/` ✓
- `VAR_ROOT = testproject/var/` ✓
- `APPS_ROOT = testproject/apps/` ✓
- `CONFIG_ROOT = testproject/config/` ✓

```python
from mojo.helpers import paths, modules, settings

paths.configure_paths(__file__, 1)
modules.load_module_to_globals(paths, globals())
settings.load_settings_profile(globals())   # reads var/profile → imports settings.local
settings.load_settings_config(globals())    # reads var/django.conf → overrides (SECRET_KEY)
```

`load_settings_profile` reads `VAR_ROOT/profile` ("local"), then does
`from settings.local import *` into globals.
`load_settings_config` reads `VAR_ROOT/django.conf` and evals each line into globals
(used to inject the generated SECRET_KEY).

---

### `testproject/config/settings/local/__init__.py`

Full local/test settings. INSTALLED_APPS is defined here (not via apps.json).

```python
from .db import *

DEBUG = True
SECRET_KEY = "set-in-var-django-conf"

AUTH_USER_MODEL = "account.User"
ROOT_URLCONF = "settings.urls"
ALLOWED_HOSTS = ["*"]
APPEND_SLASH = False
SITE_ID = 1

USE_TZ = True
TIME_ZONE = "UTC"
USE_I18N = True
USE_L10N = True

DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"

INSTALLED_APPS = [
    "mojo.apps.account",
    "mojo.apps.logit",
    "mojo.apps.metrics",
    "mojo.apps.fileman",
    "mojo.apps.incident",
    "mojo.apps.aws",
    "mojo.apps.jobs",
    "mojo.apps.docit",
    "mojo.apps.realtime",
    "mojo.apps.phonehub",
    "mojo.apps.filevault",
    "mojo.apps.shortlink",
    "django.contrib.humanize",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
]

MIDDLEWARE = [
    "mojo.middleware.cors.CORSMiddleware",
    "django.middleware.security.SecurityMiddleware",
    "django.contrib.sessions.middleware.SessionMiddleware",
    "django.middleware.common.CommonMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django.contrib.messages.middleware.MessageMiddleware",
    "mojo.middleware.mojo.MojoMiddleware",
    "mojo.middleware.auth.AuthenticationMiddleware",
    "mojo.middleware.logging.LoggerMiddleware",
]

TEMPLATES = [
    {
        "BACKEND": "django.template.backends.django.DjangoTemplates",
        "DIRS": [],
        "APP_DIRS": True,
        "OPTIONS": {
            "context_processors": [
                "django.template.context_processors.debug",
                "django.template.context_processors.request",
                "django.contrib.auth.context_processors.auth",
                "django.contrib.messages.context_processors.messages",
            ],
        },
    },
]

STATIC_URL = "/static/"

EMAIL_BACKEND = "django.core.mail.backends.console.EmailBackend"

CORS_ALLOW_ALL_ORIGINS = True
CORS_ALLOW_CREDENTIALS = True
CORS_ALLOW_HEADERS = [
    "authorization",
    "content-type",
    "accept",
    "origin",
    "x-requested-with",
    "x-mojo-bouncer-token",
    "x-mojo-bouncer-duid",
]

LOGIT_DB_ALL = True
LOGIT_FILE_ALL = False
LOGIT_RETURN_REAL_ERROR = True
LOGIT_ASYNC_LOGGING = False   # synchronous in tests — no background thread surprises

LOGGING = {"version": 1, "disable_existing_loggers": False}

INCIDENT_EVENT_METRICS = False  # no metrics processing needed in tests

JOBS_CHANNELS = ["default", "email"]
JOBS_DEFAULT_MAX_RETRIES = 0
JOBS_ENGINE_MAX_WORKERS = 2
JOBS_RUNNER_HEARTBEAT_SEC = 5

AUTH_BEARER_HANDLERS = {}
AUTH_BEARER_NAME_MAP = {
    "bearer": "user",
    "apikey": "user",
}

ALLOW_SELF_DEACTIVATION = True
```

---

### `testproject/config/settings/local/db.py`

```python
from mojo.helpers import paths

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.sqlite3",
        "NAME": paths.VAR_ROOT / "db.sqlite3",
    }
}

REDIS_DB = {"host": "localhost", "port": "6379"}

CACHES = {
    "default": {
        "BACKEND": "mojo.cache.MojoRedisCache",
        "TIMEOUT": 300,
        "KEY_PREFIX": "mojot",
        "LOCATION": "localhost:6379",
    }
}
```

---

### `testproject/config/settings/urls.py`

```python
from mojo.helpers.response import JsonResponse, HttpResponse
from django.urls import path, include

urlpatterns = [
    path("", lambda request: HttpResponse("<html><body><h1>django-mojo test server</h1></body></html>", content_type="text/html")),
    path("", include("mojo.urls")),
]

def handler404(request, exception):
    return JsonResponse({"error": "Not found", "code": 404, "status": False}, status=404)
```

---

### `testproject/config/dev_server.conf`

```
host=127.0.0.1
port=5555
```

---

### `testproject/manage.py`

Standard Django manage.py. Sets up the same path bootstrap as `_asgi.py`.

```python
#!/usr/bin/env python
import sys
import os
from pathlib import Path

TESTPROJECT = Path(__file__).resolve().parent
REPO_ROOT = TESTPROJECT.parent

sys.path.insert(0, str(REPO_ROOT))
sys.path.insert(0, str(TESTPROJECT / "config"))

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")

from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
```

---

## Bin Scripts (checked into repo)

### `bin/create_testproject`

Bash script. Idempotent — safe to re-run (wipes and recreates testproject/).

Logic:
1. Resolve `REPO_ROOT` (one level up from script)
2. Remove `testproject/` if it exists
3. Create all directories: `testproject/{bin,config/settings/local,apps,var/logs}`
4. Write all files listed above verbatim
5. Generate SECRET_KEY via Python and write to `var/django.conf`
6. Write `var/profile` = `local`
7. Run `python testproject/manage.py migrate --run-syncdb`

SECRET_KEY generation (inline Python):
```bash
SECRET_KEY=$(python3 -c "import secrets, string; print(''.join(secrets.choice(string.ascii_letters + string.digits + '!@#$%^&*') for _ in range(50)))")
echo "SECRET_KEY = '${SECRET_KEY}'" > "$TESTPROJECT/var/django.conf"
```

---

### `bin/asgi_local`

Bash script. Supports `start`, `stop`, `restart`, `status` subcommands.

Usage:
```bash
./bin/asgi_local            # foreground (Ctrl-C to stop)
./bin/asgi_local start      # background daemon (PID in testproject/var/asgi.pid)
./bin/asgi_local stop       # stop the daemon (graceful, then force after 5s)
./bin/asgi_local restart    # stop + start
./bin/asgi_local status     # check if running
```

Logic (shared by all modes):
1. Resolve `REPO_ROOT`
2. Confirm `testproject/` exists (exit with message if not — run `create_testproject` first)
3. Check Redis with `redis-cli ping`; if not running, start `redis-server` in background
4. Read `host` and `port` from `testproject/config/dev_server.conf`
5. Set `DJANGO_SETTINGS_MODULE=settings`, `PYTHONUNBUFFERED=1`
6. Run: `uvicorn _asgi:application --app-dir testproject/bin --host $HOST --port $PORT --reload`

Daemon mode writes PID to `testproject/var/asgi.pid` and logs to `testproject/var/asgi.log`.

---

### `bin/testit.py`

Python script. Sets up paths then hands off to `testit.runner`.

```python
#!/usr/bin/env python
import os
import sys
from pathlib import Path

SCRIPT_DIR = Path(__file__).resolve().parent   # django-mojo/bin/
REPO_ROOT = SCRIPT_DIR.parent                  # django-mojo/
TESTPROJECT = REPO_ROOT / "testproject"

if not TESTPROJECT.exists():
    print("ERROR: testproject/ not found. Run ./bin/create_testproject first.")
    sys.exit(1)

sys.path.insert(0, str(REPO_ROOT))                      # mojo, testit
sys.path.insert(0, str(TESTPROJECT / "config"))         # settings

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")

from mojo.helpers import paths
paths.configure_paths(str(TESTPROJECT / "config" / "settings" / "__init__.py"), 1)

from testit import runner

if __name__ == "__main__":
    runner.main(runner.setup_parser())
```

---

## `.gitignore` Update

Add to existing `.gitignore`:
```
testproject/
```

---

## Acceptance Criteria

- `./bin/create_testproject` runs cleanly on a fresh checkout (with deps installed)
- `./bin/asgi_local` (or `./bin/asgi_local start`) starts Redis if needed and serves on `http://127.0.0.1:5555`
- `./bin/asgi_local stop` cleanly shuts down the daemon
- `./bin/testit.py` discovers and runs all tests in `tests/` and reports pass/fail
- Re-running `./bin/create_testproject` wipes and recreates cleanly
- Works in a headless CI environment (no interactive prompts)
- `testproject/` does not appear in `git status`
