# How Memory Works

ntrp's memory system gives an AI assistant persistent, long-term memory across conversations. It stores raw facts, discovers patterns through consolidation, and retrieves relevant context at query time.

## Data Model

```
facts ──→ observations
  │            │
  └── entities ┘
       (links)

dreams (separate, cross-domain)
```

**Facts** are atomic pieces of information. "Alice joined the backend team in March." Each fact has an embedding, timestamps (`created_at`, `happened_at`), and access tracking.

**Entities** are named things extracted from facts (people, projects, places). Entity refs link facts to entities, forming a graph. This graph powers entity-based expansion during recall.

**Observations** are higher-level patterns synthesized from multiple facts. "Alice has transitioned from mobile to backend engineering." They track their source facts (ordered by addition time), change history, and evidence count.

**Dreams** are cross-domain insights generated by finding structural patterns between unrelated fact clusters. Each dream has a `bridge` (the structural connection between clusters) and an `insight`. They live in a separate table and don't feed back into the recall pipeline. Basically for fun feature, but may give interesting insights.

## Write Path

When a fact is stored:

1. **Embed** the text (vector embedding)
2. **Extract entities** via LLM (proper nouns only: people, orgs, projects, places)
3. **Link** entities to the fact via `entity_refs`
4. **Queue** for consolidation (`consolidated_at = NULL`)

## Consolidation (Background Loop)

A background loop runs every 30 seconds, processing unconsolidated facts:

### Per-Fact Consolidation

For each pending fact:
1. Search for similar existing observations (vector search, top 5)
2. Fetch source facts for each candidate observation (with timestamps, sorted chronologically)
3. Ask LLM to decide: **update** an existing observation, **create** a new one, or **skip**
4. LLM sees temporal context (observation change history, fact `happened_at` timestamps) to handle transitions correctly
5. Apply the decision: re-embed updated observations, record history entries

The LLM is instructed to produce observations at a *higher abstraction level* than source facts – patterns, trajectories, inferences – not rephrases.

### Temporal Pass (Daily)

Scans for entities with 3+ recent facts and looks for temporal patterns no single fact captures:

- **Trends**: values changing over time (declining sleep, increasing workload)
- **Transitions**: role/state changes across facts
- **Cycles**: recurring patterns
- **Correlations**: co-occurring changes across domains

Before creating a new observation, checks if a very similar one already exists (cosine > 0.90). If so, enriches the existing observation with the new source facts instead of creating a duplicate.

### Observation Merge (Daily)

Deduplicates observations that became too similar over time. Uses recursive pairwise merging:

1. Load all observations with embeddings
2. Find the most similar pair above 0.90 cosine threshold
3. Ask LLM: merge into one, or skip (genuinely distinct)
4. If merge: update the keeper (higher evidence count), re-embed, delete the other
5. Repeat until no pairs remain above threshold

This is hierarchical agglomerative clustering with an LLM as the merge function.

### Fact Merge (Daily)

Deduplicates near-identical facts:

1. Find fact pairs above 0.90 cosine similarity
2. Ask LLM to classify: `same` or `different`
3. Pick a keeper (more entity refs > higher access count > newer)
4. Transfer entity refs and access counts to the keeper
5. Rewrite `source_fact_ids` in any observations referencing the removed fact
6. Delete the duplicate

### Dream Pass (Daily)

Generates cross-domain insights:

1. K-means cluster all facts by embedding similarity
2. Randomly sample up to 6 cluster pairs (caps LLM calls)
3. For each pair, extract core facts and supporters
4. LLM finds structural patterns, hidden dependencies, or ironic contradictions between domains – outputs both a `bridge` (the structural connection) and an `insight`
5. A separate evaluator LLM filters candidates, keeping only those that would genuinely surprise the user
6. Dedup survivors against the 100 most recent dreams (cosine > 0.85 = duplicate)
7. Store remaining dreams

### Execution Order

```
every 30s:  consolidate pending facts
daily:      temporal pass → observation merge → fact merge → dream pass
```

## Read Path (Recall)

When the assistant needs context for a query, retrieval runs in two phases:

### Phase 1 – Observations

Hybrid search (vector + FTS) over the observations table, merged with Reciprocal Rank Fusion, scored with decay and recency. Top 5 observations are selected. For each, the most recent source facts are bundled for display.

### Phase 2 – Facts

1. **Hybrid search**: vector search + full-text search over facts, merged with RRF
2. **Entity expansion**: one-hop graph traversal from seed facts through shared entities, weighted by IDF (rare entities create stronger connections)
3. **Temporal expansion**: if query has a time context, fetch temporally proximate facts and filter by vector similarity
4. **Reranking**: cross-encoder reranker (ZeroEntropy zerank-2) rescores all candidates, with multi-signal fallback
5. **Decay + recency**: all results are weighted by access-based decay (`0.99^(hours/strength)`) and event recency (`exp(-hours/72)`)
6. **Filter**: facts already bundled under an observation from Phase 1 are excluded

Observations with change history include their most recent transitions:
```
- Alice leads backend rewrite (previously: "Alice is the mobile app lead", changed Feb 2026)
```

## Storage

SQLite with WAL mode. Vector search via `sqlite-vec` (cosine distance). Full-text search via FTS5 (shared `build_fts_query()` with stopword filtering). All tables in a single `memory.db`.

## File Map

```
ntrp/memory/
  facts.py              FactMemory: main interface, consolidation loop
  consolidation.py      per-fact LLM consolidation into observations
  temporal.py           temporal pattern detection pass
  observation_merge.py  recursive pairwise observation deduplication
  fact_merge.py         pairwise fact deduplication
  dreams.py             cross-domain insight generation
  extraction.py         entity extraction from facts
  chat_extraction.py    fact extraction from chat messages
  retrieval.py          hybrid search, entity/temporal expansion, observation retrieval
  reranker.py           cross-encoder reranker (zerank-2)
  formatting.py         context string assembly for system prompt
  decay.py              access-based decay + recency boost
  fts.py                shared FTS5 query builder with stopword filtering
  models.py             Fact, Observation, Dream, Entity, HistoryEntry, etc.
  prompts.py            all LLM prompts
  service.py            memory service layer
  indexable.py          indexable content interface
  store/
    base.py             schema, GraphDatabase
    facts.py            FactRepository (CRUD, search, entity graph)
    observations.py     ObservationRepository (CRUD, search, merge)
    dreams.py           DreamRepository
```
