Memory formation and consolidation¶

This document specifies the write path for long-term memory: how raw episodes become durable, curated beliefs and an evolving model of the user. It is the detailed design behind Milestone 9 (see the engineering plan) and is recorded as ADR-0018.

Scope: formation only. Representation details beyond what formation needs and the entity-graph layer are separate specs; the read path is specified in memory retrieval and ranking. This document assumes the storage primitives from Milestone 9 (the MemoryRecord belief store) and the episodic event log from Section 6.8.

Why formation is the hard part¶

Retrieval is a ranking problem with well-understood tools. Formation — deciding what is worth remembering, in what form, and how it reconciles with what is already believed — is where memory quality is actually won or lost. A wrong or stale belief is worse than a missing one, because the agent will act on it confidently. This spec optimizes for precision, provenance, and reversibility over raw capture.

Principles¶

Memory is a governed projection over the episodic log. The append-only event log (Section 6.8) is ground truth. Beliefs are derived from it, never a lossy side-channel. This makes memory re-derivable, auditable, and correctable (see "Re-derivation" below).
Formation is deliberate and gated, never an implicit effect of reading content. Untrusted content can never create owner-authority belief; the attributed-communication exception is locally rendered and remains inferred.
Precision over recall. Prefer to miss a fact than to fabricate one.
Everything is provenance-linked, user-visible, and reversible. Every belief points back to the events and the consolidation run that produced it; the user can inspect, edit, and delete; edits and deletes are themselves events.

Memory states and tiers¶

Two different things get called "tiers"; the design keeps them as separate axes.

Confidence lifecycle (how established a belief is). This is a continuous confidence score, not a stack of discrete tiers. The named states are thresholds and lifecycle transitions over that score:

candidate - proposed in the pipeline, not yet committed.
provisional - committed but low-confidence, on probation; contributes weakly to retrieval; promotes to active when reinforcement pushes confidence and corroboration over a threshold, or decays out if never seen again.
active - established and durable; full retrieval weight.
superseded - replaced by a newer belief; retained (bi-temporal), excluded from live retrieval.
expired / retired - past validity or decayed below the floor; retained for audit and re-derivation, excluded from live retrieval.

Keeping confidence continuous (with a couple of thresholds) rather than many hard tiers means granularity can be tuned later by moving thresholds - no schema change and no extra classification burden on the extractor. Start with provisional -> active -> retired; add finer bands only if evals show a need.

Memory hierarchy (where a memory lives). Orthogonal to confidence, and the tiering that most shapes long-term memory building. Information flows up as it is consolidated and out as it decays:

Tier	Store	Role
Working	in-context (Section 7 working state)	the current task; hottest, ephemeral
Episodic	event log (Section 6.8)	what happened; recent, high-volume, cheap
Semantic (long-term)	belief store	consolidated model of the user and world
Archival	cold episodes	retained for re-derivation; not routinely retrieved

Formation is the promotion mechanism from episodic to semantic; decay and forgetting demote or evict; retrieval pages across the tiers into working memory. A belief always lives in the semantic tier - its confidence state is the separate axis above.

The pipeline¶

Formation is an incremental pipeline from episode to committed belief. Stages 1–7 run per consolidation; stage 8 is background maintenance.

episodic log ─▶ 1 trigger ─▶ 2 segment & select ─▶ 3 extract candidates
     ▲                                                     │
     │                                                     ▼
 8 decay / re-derive ◀─ 7 commit ◀─ 6 gate ◀─ 5 resolve ◀─ 4 filter

1. Trigger — when consolidation runs¶

Tiered, so the expensive stages run rarely:

Turn-boundary (cheap flag). After a run completes, a lightweight check decides whether anything memory-worthy happened. It does not extract; it only marks the session for consolidation. Avoids running the extractor every turn.
Session-boundary / idle (primary). When a session ends or goes idle, run full consolidation over that session's new episodes. This is the main cadence — the analogue of sleep consolidation.
Explicit. The agent's memory.remember tool, or the user saying "remember that…". High precision; skips the salience gate but still goes through resolve, gate, and commit.
Scheduled reconsolidation. A periodic background job revisits, merges, decays, and can re-derive (stage 8).

Cadence is configured per agent / policy profile. Default: session-boundary + explicit + scheduled, with turn-boundary as a cheap flagger only.

2. Segment & select¶

Operate only over episodes after the consolidation watermark — a per-session cursor recording the last consolidated event sequence. This makes formation incremental and idempotent (re-running consolidates nothing new).
Select salient spans: user corrections, decisions, stated preferences, new entities, explicit "remember this", task outcomes. De-prioritize routine tool chatter.
Working state is a second input. WorkingState.established_facts surviving to the session boundary are offered as candidates with their provenance and trust level attached (context engine, ADR-0020). They are candidates, not beliefs: they enter here and pass through every stage below, including the trust gate. A fact the run derived from untrusted content is still untrusted.
Typed trust gate at selection. Arbitrary spans that are solely EXTERNAL_UNTRUSTED (web, browser, file, or unknown tool content) are never a formation source. ADR-0090 admits one narrower class: a recognized, principal-scoped communication event may form a bounded attributed summary without becoming an owner assertion. Gmail and SMS retain untrusted policy treatment and can produce only inferred, tentative, local, sensitive hypotheses that cannot retract owner memory. This distinction, plus exact event-shape admission, is the primary prompt-injection defense on the write path.

3. Extract candidates¶

A consolidation model (a cheap aux tier — most consolidation is small) reads the selected episodes plus a compact view of related existing beliefs, so it proposes deltas, not duplicates. It emits candidate memories with schema-constrained structured output (Section 10 / structured outputs):

class MemoryCandidate(BaseModel):
    belief_type: str          # "fact" | "preference" | "relationship"
                              # | "user_model_attr" | "procedure_pointer"
    subject: str              # the entity the belief is about
    statement: str            # the asserted content
    polarity: str             # "assert" | "retract"
    source_event_ids: list[int]   # mandatory provenance
    model_confidence: float
    proposed_scope: str       # "user" | "project" | "global"
    proposed_portability: str # portable | contextual | local
    sensitivity_guess: str
    valid_from: datetime | None
    expires_hint: datetime | None

Candidates are proposals, not writes.

Candidate boundaries are semantic boundaries. Separate first-person clauses and separate text parts are extracted independently; a conjunction such as “I prefer tea and we decided to deploy Fridays” must not turn the decision into part of the preference. Ordinary experience narratives use the same rule: a newly started activity, prior experience with a related activity, and a recurring physical symptom are independently correctable beliefs with semantic subjects rather than one generic hobby or user bucket. Physical symptoms are sensitive and flagged for review. The deterministic extractor has a separate 256-proposal scan ceiling for resource safety. The governed service, not the extractor, applies the smaller twelve-candidate commit ceiling and records the number the extractor returned plus every proposal it rejected, including overflow. The rejected audit count also includes candidates that are idempotent same-source replays: they are safe no-op outcomes, but counting them makes every proposed candidate reconcile to a terminal outcome even though the schema has no separate unchanged field.

Portability has a deterministic ceiling. Each belief_type carries a default portability — preferences, user-model attributes, and procedure pointers are portable; facts and relationships default to contextual. The extractor may lower a candidate's portability but never raise it, so a model cannot make a belief travel further than its type allows. Local provider-claim rendering clamps an over-ceiling proposal to that deterministic ceiling rather than discarding an otherwise grounded candidate; the governed service retains the ceiling check as a defense-in-depth rejection gate. Portability governs how a belief behaves outside the project it was learned in, and is defined in memory retrieval and ranking.

4. Filter — eligibility and salience¶

Eligibility (hard gates, reject on any). Derived solely from untrusted content; contains secrets/credentials/tokens; PII beyond policy; transient task detail; a restatement of platform instructions or private reasoning. This mirrors the "never automatically store" list in Milestone 9 and ADR-0006/0007. The tool boundary reports an origin-trust rejection as tool.trust_rejected, distinct from malformed arguments, so the model does not retry the same unsafe write as though its JSON shape were wrong.
Salience (soft ranking). Keep candidates above a worth-remembering threshold: durability (will this matter next week?), specificity, corroboration, and user-signaled importance. Sub-threshold candidates are dropped or held in the provisional state (see "Memory states and tiers") that promotes to active only on later reinforcement. A model's confidence is proposal metadata rather than user authority: an automatically inferred belief enters with confidence no greater than 0.55, regardless of the extractor's self-reported score, and must earn promotion by later reinforcement. Explicit user-authored memory writes retain their separate authority and confidence path.

5. Resolve against existing memory¶

For each surviving candidate, match against existing beliefs by subject + semantic + lexical similarity, then:

Relationship	Action
New	Insert as a new belief.
Duplicate / near-duplicate	Reinforce: increment corroboration, bump confidence, append `source_event_ids`, update `last_reinforced_at`. No new record.
Refinement	Update/extend the existing belief (e.g. more specific).
Conflict	Never overwrite and never order. Commit the new belief beside the existing one, link both directions of `conflicts_with`, flag both for review, and request confirmation.
Contradiction	Never overwrite. Insert the new belief and link `conflicts_with` / `supersedes`; apply the conflict policy below.

A replay is only a replay inside its own session. The same-source shortcut that makes re-consolidating an already-consolidated episode a no-op compares event sequences, and sequences are allocated per session, so it applies only when the existing belief's source session is the session being consolidated. A later session numbers its own first event one as well; treating that as the same source would classify every later session as a replay and leave the belief it contradicts live. A resolver that is not told which session is consolidating keeps the sequence-only comparison.

Conflict-resolution policy (default). Resolve by source authority, then recency: a direct user statement supersedes an inferred one; a more recent user statement supersedes an older user statement. The superseded belief is retained with valid_to set (bi-temporal — we keep what was believed and when), marked superseded. Formation is fully autonomous: resolution is never blocked. Ambiguous or sensitive resolutions are still committed (by authority then recency) but marked flagged_for_review and surfaced to the user, who can correct or delete after the fact - the safety model is after-the-fact review, not a pre-commit gate.

What neither authority nor recency orders is a conflict. Milestone 16 adds the fourth outcome above between duplicate and contradiction, so the order is same source, duplicate, conflict, contradiction. Incoming evidence conflicts when its authority ranks below the existing belief's, or when the two rank equally and nothing places them in time: the same session with no later source event, or a different session in which the existing belief's evidence onset, valid_from, is no older than the incoming instant — the time of the newest source event backing the incoming candidate, not the time it is being consolidated. Replaying an older session therefore conflicts with the belief that replaced it rather than reverting to it. Polarity is never the deciding factor, so an ordinary retraction still supersedes what it retracts, and a resolver that is not told the incoming instant keeps the ordering it had before the rule existed, exactly as one that is not told the session keeps the sequence-only comparison. A conflict leaves both of its beliefs live, so the same-source shortcut is decided over every related belief before any of them is acted on, and re-consolidating a conflicted session is a no-op like any other replay. The rule and what a conflict then commits are stated in memory-evaluation-and-lifecycle.md:911-936.

Bi-temporal validity is what lets "Andy works at Acme" become false without being deleted, and lets the agent answer "what did I believe last month".

Promotion across scopes. Matching is performed within the principal, not within the project, so a candidate formed in one project matches an existing belief formed in another. When the same belief is independently corroborated in two or more distinct project scopes, it is promoted to user scope: independent observation in unrelated contexts is the evidence that a belief describes the principal rather than one project. Promotion sets scope = "user", retains every contributing origin_scope in provenance, and emits memory.promoted. This is the write-path half of beliefs carrying across projects; the read path supplies additional evidence by recording which carried beliefs proved useful outside their origin.

Promotion applies only to portable and contextual beliefs. A local belief (an endpoint, a service name, a deadline) that happens to look similar across two projects is a coincidence, not a generalization, and is never promoted.

6. Gate — policy and safety¶

Each proposed write passes the deterministic policy engine, exactly like a tool call (Section 9):

Injection scan of the candidate statement.
Sensitivity classification. Formation is fully autonomous, so sensitive writes are not held for confirmation; they are committed and marked flagged_for_review for the user. The user clears the flag, retires the belief, or localizes it through the review route and native actions of ADR-0117 (memory-read-api-and-browser.md). The deterministic eligibility gates in stage 4 (never form secrets, credentials, untrusted instructions, or PII-beyond-policy) are hard filters that always apply - autonomy governs confirmation, never those.
Volume/rate caps, so one pathological session cannot flood memory.

7. Commit¶

Write the belief(s) through the MemoryStore port in a short transaction.
Emit a memory.formed / memory.reinforced / memory.superseded / memory.needs_confirmation event to the log — memory changes are themselves auditable episodes (and part of what makes re-derivation possible).
Advance the consolidation watermark after successful extraction, a non-retryable provider failure, or bounded retry exhaustion. A retryable provider failure commits deterministic proposals but leaves the source prefix pending until the provider succeeds or exhausts the fixed retry policy.
If the belief is a user_model_attr, update the user-model projection (a curated view over user-scoped beliefs — the "deepening model of who you are").

8. Decay and re-derivation (background)¶

Decay. Provisional and low-confidence beliefs that go unused lose confidence over time; expired beliefs (past expires_at) retire. Reinforcement resets decay — used, corroborated beliefs persist; noise fades.
Consolidation/merge. Many related low-level beliefs compress into fewer higher-level ones, provenance retained.
Re-derivation ("re-remember"). Because the event log is ground truth, a scheduled or opt-in job can re-run formation over history with an improved extractor, prompt, or model, and reconcile — producing a better belief set without losing auditability. Every belief records the consolidation_policy_version that formed it, so we always know which run produced which belief. This capability is the main advantage of deriving memory from an episodic log rather than writing it as a lossy side effect.

Milestone 16 builds the decay and explicit re-derivation parts of this stage. Whole-store merging and grounded connections are now authorized as Milestone 32; memory-reconsolidation.md defines that mechanism and its twenty-four gates; runtime and offline comparisons are implemented; verification and supervised deployment remain pending. memory-evaluation-and-lifecycle.md gives decay a bounded sweep with per-belief-type half-lives, a confidence step, and a retirement floor; adds the AFFIRMED authority and the working-state facts that enter at it; makes an unresolved contradiction a fourth resolver outcome that commits flagged and surfaced rather than superseding; and makes re-derivation an explicit --confirm command. It also supplies the benchmark that measures whether any of it helped, which is what the tracked metrics below have been waiting for.

User corrections are evidence¶

Beliefs are inspectable, so they are also rejectable. A rejection arrives with the belief id it refers to, from the recall trace the user was reading (memory retrieval and ranking), which makes which belief unambiguous. What is wrong about it is not, and the surface asks rather than guesses — the three answers are three different writes:

The user says	Formation does
Not true	Retire the belief: `valid_to = now`, status `superseded`, no replacement.
Was true, has changed	Supersede it with the correction as a new belief at user authority.
True elsewhere, not here	Lower `portability` and record a negative scope override. The belief survives in its origin; it stops being carried here.
(unspecified)	Flag for review and down-weight. Never retire on an ambiguous signal.

A rejection is evidence at the highest authority — a direct user statement, which outranks anything inferred and any older user statement under the existing conflict-resolution policy. It is not a special case in the resolver; it enters through the same door as any other user statement.

Rejections are events, not edits, and re-derivation replays them. This is the sharp edge. Because the event log is ground truth, a re-derivation run re-forms beliefs from history — and would cheerfully re-form everything the user has ever corrected, since the improved extractor sees the same original episodes. A rejection is therefore an input to formation rather than a correction applied to its output: every consolidation run, including re-derivation, applies the outstanding rejections for the principal before commit.

That matching cannot be by belief id, because re-derivation mints new ids. A BeliefRejection stores the rejected subject, statement, and belief_type, and matching is by content similarity on the same comparison the resolver already uses for duplicates. A belief that a user has rejected does not come back with a new id.

Rejecting is not deleting. "This is wrong" is a correction and keeps its content for audit and replay. "Delete this" is a data-removal request: the belief record is removed and the tombstone retains a content hash rather than the statement, so re-derivation can still refuse to re-form it without the platform continuing to hold what the user asked it to forget.

Deletion also sanitizes older correction tombstones for that belief or its replacement and removes copied influence from generated context and replies. Its audit event retains only the opaque belief ID. People-linked deletion uses the same durable generated-copy fence and bounded cleanup receipts as forgetting a person, without deleting the person or suppressing unrelated original messages.

Rejection rate is also the cheapest formation-quality signal available — corrections per hundred rendered beliefs, measured against real usage rather than a rubric.

Persona nomination and curated promotion¶

Milestone 22 adds one output to the pipeline and changes nothing about how beliefs form. When consolidation commits or reinforces a belief that clears the persona bar — active, direct rather than hypothesis, a durable type at user scope, corroborated and confidence-qualified past configured floors, sensitivity within bounds, outside every never-auto-store category, and clean under the injection and secret scans — the governed service records a persona nomination carrying the belief's canonical statement. Nothing else may nominate: not a tool call, not a model output, not an operator script.

A nomination waits for the owner. Affirming it appends the statement to the persona document at USER authority and marks the source belief promoted, linked in both directions; the belief itself is never superseded or deleted by promotion, and removing the persona entry later clears the marker. Declining is durable: the belief is never nominated again, including after a full re-derivation, the same standard corrections hold. A nomination whose source belief supersedes or retires before review is withdrawn. The outstanding set is bounded, so review stays a minute's work rather than a queue.

The mechanism — the nomination record, its states, the bounded set, the promoted marker, and the gates that watch them — is specified in persona-surface.md and ADR-0079. This document owns only the pipeline hook: nomination happens inside the same governed unit of work as the commit that qualified the belief, and never on a retry or no-work path.

Data-model additions¶

Extends the Milestone 9 MemoryRecord:

class MemoryRecord(BaseModel):
    # ... existing Milestone 9 fields (id, tenant_id, principal_id,
    #     scope,
    #     subject, statement, source_event_ids, confidence, sensitivity,
    #     valid_from, expires_at, status) ...
    belief_type: str
    polarity: str                       # "assert" | "retract"
    portability: str                    # portable | contextual | local
    origin_scopes: list[str]            # every scope corroborating it
    corroboration_count: int = 1
    last_reinforced_at: datetime
    # bi-temporal: when the belief stopped holding
    valid_to: datetime | None
    superseded_by: UUID | None
    # `status` (from Milestone 9) carries the lifecycle state:
    #   candidate | provisional | active | superseded | expired
    flagged_for_review: bool = False    # committed, surfaced for review
    formation_run_id: UUID              # which consolidation produced it
    consolidation_policy_version: str

class BeliefRejection(BaseModel):
    id: UUID
    tenant_id: str
    principal_id: str
    belief_id: UUID                     # as rejected; ids may change
    # kind: untrue | changed | not_here | unspecified
    kind: str
    # content keys survive re-derivation
    subject: str
    # statement is None once the user has asked for deletion
    statement: str | None
    statement_sha256: str               # the tombstone key
    belief_type: str
    scope: str                          # where the rejection was made
    replacement_id: UUID | None         # set for "was true, has changed"
    trace_id: UUID | None               # what the user was looking at
    created_at: datetime

class ConsolidationRun(BaseModel):
    id: UUID
    tenant_id: str
    principal_id: str
    # trigger: session | explicit | scheduled | ...
    trigger: str
    scope: str
    watermark_before: int
    watermark_after: int
    model: str
    policy_version: str
    candidates_proposed: int
    committed: int
    reinforced: int
    superseded: int
    rejected: int
    started_at: datetime
    finished_at: datetime | None

New relationships between beliefs: conflicts_with, supersedes.

Ports and runtime placement¶

MemoryStore port: get, query, upsert_belief, reinforce, supersede, list, edit, delete, reject, outstanding_rejections, record_consolidation, and principal-scoped consolidation listing. Backends: Postgres (FTS + normalized belief tables) first; pgvector and an external provider (e.g. Honcho) later, behind the same port (ADR-0014).
MemoryConsolidator port: run(trigger, scope, since_watermark) -> ConsolidationResult. The builtin implementation is LLM extraction as above; an external memory provider can be delegated to behind this port.
MemoryCandidateExtractor port: extract(events, principal, scope) -> list[MemoryCandidate] | MemoryExtractionResult. The result extension remains list-compatible and adds only content-free provider-failure metadata. Extractors propose only; the service owns the candidate cap and every provenance, scope, portability, salience, rejection, and conflict check. agent_core.memory.SHIPPED_MEMORY_CANDIDATE_EXTRACTORS is the authoritative implementation census; the shared contract parameterizes itself from that production-package registry so a new shipped extractor cannot bypass the common proposal and fallback assertions.
Salience and ConflictResolver are replaceable strategies.
Runtime placement. Consolidation runs as a restricted child run (subagent, Section 27.6 / Milestone 10) or a dedicated background worker job: tool set limited to memory read/write and retrieval, its own budget, sandboxed, returning a concise summary — the same governance model as the self-improving skills in Section 30. It never blocks the interactive turn.

Governance and safety (cross-cutting)¶

Formation is autonomous but fully transparent and reversible: every belief links to its source events and formation run; the user can list, inspect, edit, and delete; those actions are events.
Untrusted content can never form owner-authority memory. Owner statements and the agent's affirmed conclusions remain the only owner/affirmed sources. A recognized communication may form only a channel-attributed, inferred, tentative summary under ADR-0090; arbitrary untrusted content remains barred.
Never form secrets, credentials, tokens, raw untrusted instructions, or private reasoning (ADR-0006/0007).
Formation is fully autonomous: no belief requires synchronous human confirmation. Safety rests on the deterministic eligibility gates, the untrusted-content write ban, and after-the-fact transparency and reversibility; sensitive or ambiguous beliefs are committed but flagged for review.
Every write is tenant- and principal-scoped.

The initial human surface is agent memory: list (optionally including inactive beliefs or filtering by source session), get, edit, and delete make the belief store inspectable and reversible. formations exposes the principal-scoped consolidation audits, including model, policy, watermarks, and candidate outcome counts. diagnose --session joins formation flags, the current watermark, provider selection and attempt audits, consolidation runs, and formed beliefs without exposing prompt or response bodies. replay --session --confirm reprocesses the original session events through the same governed service and retains their source-event provenance; it never inserts a belief directly. This is a local/operator CLI surface over the same composition as the runtime; no separate write path exists.

Milestone 10 ordinary-conversation maturation¶

Milestone 9 shipped the governed store, explicit write path, narrow deterministic extractor, conflict lifecycle, and session-close callback. It did not connect the runtime's documented post-run formation flag to maintenance, and its extractor returned at most one candidate from only two phrasings. Consequently a normal client that never called the internal session-close service could finish many runs without forming memory, and a single utterance mentioning an Apple Watch and a BMW X3 formed neither belief.

The first authorized Milestone 10 memory slice closes that lifecycle and widens the deterministic fallback while preserving the model-assisted design above:

Every terminal run appends one memory.formation.requested event after the terminal transaction commits. Its derivation key is the terminal run id, so a callback retry cannot enqueue the same work twice. A waiting run is not terminal and does not enqueue the flag.
Full extraction remains off the interactive path. The maintenance role selects flagged sessions after 30 seconds without committed activity and invokes consolidation; an explicit session close remains an immediate boundary. The fixed delay is part of deterministic formation@7, not a deployment override that can silently change the policy represented by that version. Both the session-idle cutoff and the flag's persisted not_before must be satisfied; not_before is authoritative even when the session is otherwise idle, while legacy flags without it fall back to their event time. A malformed or timezone-naive value is ineligible and cannot poison the rest of a maintenance sweep. Selection is oldest first without an arbitrary look-ahead window. The in-memory selector streams the newest-first session pages through a batch-sized oldest-candidate buffer, and PostgreSQL uses an event-type/session/sequence index for the same scan. After extraction, the writer takes a per-principal, per-session claim; PostgreSQL holds a transaction-scoped advisory lock through belief writes, the audit, and watermark advancement, so concurrent workers cannot form the same prefix twice.
MemoryCandidate becomes a domain value and MemoryCandidateExtractor a formation port. The deterministic v2 implementation emits multiple candidates for independently addressable ownership, preference, user-attribute, relationship, project-decision, and task-outcome spans. It also recognizes ordinary entity retractions, including coordinated possessives such as “I no longer have my watch and my car” without reasserting the trailing entity. The formation service, rather than any extractor, applies the fixed ceiling of twelve proposals per consolidation and preserves accepted candidate valid_from and expiry hints on the resulting record.
The service independently verifies every candidate source against the selected log prefix. An owner-assertion source must be a user.message.created event authored by the principal or an authenticated paired surface bound to that principal. An attributed-communication source must match a closed, repository-owned event contract: first-party Gmail read output or device-ingested SMS in version 1. Such candidates are forced to inferred authority, hypothesis derivation, tentative longevity, local portability, at least sensitive classification, and affirmative channel attribution; they cannot retract or supersede owner memory. The service also rejects a proposed scope that differs from the consolidation job's authorized scope. This rule applies even when a later model-assisted extractor proposes the candidate.
Candidate subjects are conflict keys, not a generic user bucket. Device entities remain separate; answer style, interface theme, indentation style, and measurement units are separate preference subjects. Unclassified preferences derive a stable topic key from the preference object instead of sharing one fallback key. A retraction therefore supersedes the matching entity while unrelated memories survive, and a negated possessive span cannot also propose positive ownership from the same source.
One consolidation audit covers the claimed event prefix and reports new commits, reinforcements, supersessions, safety rejections, and extractor overproduction separately. Its elapsed interval begins before extraction. Automatic candidates do not create misleading nested one-candidate audits. A PostgreSQL supersession uses a nested transaction so a stale-current conflict rolls back its just-inserted replacement even when the caller catches the conflict and continues the outer consolidation transaction.
Provider failure is a formation outcome, not a successful watermark advance. Failure audits retain only normalized safe fields: failure kind, provider code, HTTP status, provider parameter, whether the stream produced output, and retryability. Transient and protocol failures retain the original prefix and append an idempotent memory.formation.requested retry after 60 seconds and then 300 seconds. Maintenance eligibility follows the latest formation request, so an older ready flag cannot bypass the retry delay, while the attempt counter is restored from the greatest valid pending provider_retry attempt in the source window so a later terminal-run request cannot reset the retry budget. After three total attempts the watermark advances and a content-free memory.provider_extraction.retry_exhausted process event records the terminal disposition. Deterministic candidates committed on an earlier attempt remain idempotent same-source outcomes on every replay.

ADR-0051 introduced a schema-constrained hybrid reference extractor over trusted principal-authored events and a compact existing-belief view. Its focused tests retain regression coverage for grounding, strict output, independent limits, content-free memory.extraction.completed auditing, and deterministic fallback. Those tests are not production activation evidence. Accepted ADR-0057 supersedes the earlier routed-policy selection rule. Normal composition does not select a provider extractor until a version-bound artifact for the exact runtime tuple passes startup validation. The balanced OpenAI gpt-5.6-sol and default-policy tuple has reviewed passing formation@10 evidence from the checked-in 25-case corpus, gathered on a populated store, so auto activates provider assistance for exactly that tuple, preferring formation@9 where its comparative evidence also matches, and stays on deterministic formation@7 for every other one. The frozen formation@8 artifact for that tuple was withdrawn under ADR-0086, as the superseded formation@4 artifact was deleted before it, because a stale or duplicate artifact would leave the activated policy version wrong or ambiguous.

Evaluation-gated provider assistance (`formation@8`)¶

The first provider-assisted implementation is a dedicated maintenance extractor, not an interactive call or a general-purpose subagent. It implements the same MemoryCandidateExtractor port, receives only selected owner assertions plus a compact view of at most fifty existing beliefs, advertises no tools, and requests one schema-constrained semantic-claim batch. The provider selects from a closed claim-kind vocabulary, cites an exact evidence substring, and supplies source-grounded values; it does not author final belief prose or scope. Governed local code renders each accepted claim into a canonical MemoryCandidate. The fixed policy budget is one model call, 16,000 input tokens, 4,096 output tokens, USD 0.05, and 30 seconds. Before the call, catalog pricing and a conservative input estimate reduce the requested output-token maximum as needed to fit the cost ceiling; a tuple that cannot afford one output token falls back without calling. Provider output is still a proposal: the service rechecks provenance, scope, portability, salience, corrections, conflicts, and its twelve-candidate commit ceiling. For automatic formation it also scans the authoritative cited source text for secret, injection, and transient markers, so provider paraphrasing cannot erase a safety signal before the write gate.

Activation uses auto, off, and required rollout modes. auto is the default: it searches an explicit operator artifact and then evidence bundled with the installed release, activates an exact match, and otherwise retains deterministic formation with a content-free selection audit. off never resolves the formation provider. required refuses startup without matching evidence. The JSON schema binds the result to the extractor version, formation policy, model policy, provider, model, policy profile, and compiled policy version. It must report strictly more supported candidates than the deterministic baseline across at least twenty labeled samples, complete support for at least eighty percent of positive cases, zero fabricated candidates in either arm, and no additional policy failures. The schema derives that eighty-percent minimum from the positive case count and rejects an artifact that attempts to lower it. A non-activating evaluation constructor exists to gather that evidence without creating the circular requirement that an unevaluated extractor already be active. It is reachable only through an explicit code-level evaluation build flag, is mutually exclusive with required, and is refused in production.

agent eval memory-formation removes manual artifact authorship. With the live evaluation opt-in, it runs a checked-in labeled corpus through isolated paired deterministic and provider-assisted arms, derives the active model tuple and corpus hash, and atomically writes evidence only when the schema's lift, coverage, fabrication, and policy conditions pass. The current 25-case corpus has twenty-one positive examples and four protected no-memory examples; scoring uses checked-in normalized labels, not a second model judge. Every result reports per-case normalized beliefs, consolidation counts, shared fallback beliefs, provider-added beliefs, and content-free extraction-audit counts for both arms. A failed run exits non-zero, reports aggregate counts and the failing case identifiers, and leaves no activation artifact. Release engineers may place reviewed artifacts in the package evidence directory; those files share the installed code's distribution trust boundary. The repository defines no independent release-signing root, so it does not pretend that a detached file signature would authenticate an otherwise unsigned release.

Every attempted provider extraction emits one process-scoped audit containing its principal and session, agent and policy versions, authorized scope, empty tool scopes, exact source-event sequences, provider and model, evidence identifiers, budget, deadline, usage, cost, outcome, and safe normalized failure fields. Prompt and response bodies and exception messages are never stored there; only their SHA-256 hashes are retained. A failed, timed-out, malformed, or over-budget call records failure and returns deterministic proposals. Retryable failure keeps the prefix pending under the bounded schedule above; permanent failure and exhausted retry advance it. A recorded usage violation is a local permanent budget failure and never enters the provider retry schedule; replaying the same over-budget response would not change that policy outcome. A cancelled call records the cancellation and propagates it rather than disguising shutdown as successful formation. A successful batch is merged with the deterministic fallback and passes through the same service gates. Provider-assisted consolidations and beliefs record formation@8, or formation@10 under the repaired budget and related-belief view that adaptive-memory-distillation.md specifies and ADR-0086 records; the frozen formation@8 budget above is the reason production formed almost nothing from populated stores, and its artifact is withdrawn. The default deterministic path records formation@7. The activation decision is recorded in ADR-0057, and the retry, diagnosis, replay, and policy-version correction is recorded in ADR-0068. Admitting working-state established facts as AFFIRMED candidates advanced those versions from formation@5 and formation@6, because it changes what formation produces from the same events (ADR-0069).

Hard gates¶

Formation cannot be tuned without measurement; build the harness alongside the first formation layer (Section 20).

Contradiction handling — inject a preference change; verify supersession (not duplication) and that retrieval returns the current belief. M9.
No fabrication — it must not form beliefs unsupported by episodes. M9.
Injection resistance — an untrusted "remember X" must not form a belief. M9.
Correction durability — a rejected belief must not return, including after a full re-derivation under a newer consolidation policy. A hard gate. M9.
No policy regression — adapt LOCOMO-style long-horizon scenarios to exercise the write path. Gate: memory improves target eval cases without increasing policy failures. M9.
Multiple ordinary memories — one ordinary user utterance naming two durable entities forms two separate, provenance-linked beliefs, and an independent preference in the same utterance remains a third belief. M10.
Automatic-source integrity — automatic candidates name only admitted owner assertions or recognized principal-scoped communications. Paired surfaces retain owner attribution; Gmail and SMS form only attributed, tentative, sensitive hypotheses. Assistant, model, arbitrary tool or web, malformed, unpaired, foreign-principal, and foreign-tenant content cannot become a source. ADR-0090 widens the implementation under Milestone 21 without moving this original gate's assignment. M10.
Idle lifecycle — a terminal run enqueues one idempotent formation flag, returns without extracting, and maintenance consolidates the session only once both the session-idle boundary and the flag's persisted not_before have elapsed. M10.
Bounded automatic formation — a pathological utterance cannot commit more than twelve candidates in one consolidation, a secret-shaped candidate is still rejected, and the consolidation audit accounts for extractor overproduction rather than silently truncating it. M10.
Ordinary correction isolation — a natural-language retraction supersedes the matching entity while unrelated entities and preference topics continue to coexist. M10.
Governed inspection and reversibility — the public memory surface can list and inspect beliefs, source-session formation audits, and recall traces; diagnose one session's formation lifecycle and replay its original evidence; edit, delete, diagnosis, and replay use the same principal-scoped service, and a foreign principal can observe or mutate none of them. M10.
Extractor contract coverage — every shipped MemoryCandidateExtractor implementation is registered against the shared contract for separate, scope-correct, provenance-bound proposals, including deterministic fallback when an assisted extractor receives unusable provider output. M10.
Evidence-bound activation — provider assistance is eligible only when the evaluation artifact exactly matches extractor version, formation policy, model policy, provider, model, policy profile, and compiled policy version; a mismatch in any field does not activate it. M10.
Provider provenance boundary — provider proposals remain proposals: invented source ids cannot pass the governed service's provenance check, and secret-shaped source content cannot pass its safety checks. M10.
Audited deterministic fallback — a failed provider attempt records a content-free failed audit with normalized failure fields and returns the independently derived deterministic proposals rather than suppressing memory formation. M10.
Evidence publication — the checked-in paired evaluation derives the active provider tuple and corpus hash and atomically publishes activation evidence only after formation lift, at least eighty percent positive-case coverage, zero fabrication in either arm, and no policy regression pass; a failed evaluation leaves no artifact. M10.
Canonical provider claims — the provider returns a closed semantic claim with an exact cited evidence substring; deterministic local rendering owns the final belief type, canonical subject, statement, and authorized scope. M10.
Inspectable evaluation failure — paired evaluation results retain each arm's normalized beliefs, consolidation counts, provider audit outcome and candidate counts, and shared-versus-provider-added attribution; a failed gate returns those diagnostics and the CLI exits non-zero without publishing an activation artifact. M10.
Positive formation coverage — aggregate lift alone is insufficient: provider assistance must fully support at least sixteen of the twenty labeled positive cases before evidence can be published. M10.
Authoritative source safety — secret, injection, or transient markers in authoritative cited source text cannot be erased by provider normalization before the automatic eligibility check. M10.

Tracked metrics¶

Formation precision — of committed beliefs, the fraction correct and worth keeping (rubric/graded). The primary metric.
Recall of consequential facts — does it capture what later tasks need.
Rejection rate — user corrections per hundred rendered beliefs. A precision proxy measured against real usage rather than a rubric.
Cost — consolidation tokens per session within budget.

Build sequence (incremental, each gated by evals)¶

The builtin consolidation path is built to parity first (steps 1-5); an external provider is a later comparison option (step 6), not the initial path.

Explicit memory.remember + belief store + provenance + user list/get/edit/delete CLI + reinforce-on-duplicate. No automatic formation yet.
Session-boundary consolidation: extraction + eligibility gate + dedupe.
Conflict detection + supersession + bi-temporal validity.
Typed rejections and their replay, before re-derivation exists to violate them.
Decay + scheduled reconsolidation + re-derivation.
External-provider adapter option; user-model projection; graph edges (handed to the separate graph spec).

Decisions¶

Formation is fully autonomous from the start. No belief requires synchronous confirmation; safety is the deterministic eligibility gates, the untrusted-content write ban, and after-the-fact transparency and reversibility; sensitive or ambiguous beliefs are committed and flagged for review.
Build the builtin consolidation path to parity first. An external provider (Honcho/Mem0-style) behind the MemoryConsolidator port is a later comparison option, not the initial path.
Keep the provisional tier, and model "tiers" as two axes (see Memory states and tiers): a continuous confidence lifecycle (provisional -> active -> retired) and an explicit memory hierarchy (working -> episodic -> semantic -> archival) that formation promotes across. This is the tiered memory system - not merely two tiers.
The user model is a projection over user-scoped beliefs, not a separately maintained artifact - one source of truth, no drift.
Beliefs are matched within the principal, not within the project, and corroboration across two or more project scopes promotes a belief to user scope. The agent learns from every project and environment it works in. Portability is bounded by belief_type so that project-local facts never generalize.
Re-derivation is opt-in per principal, not automatic on consolidation-policy upgrades - privacy-conscious users should not have old episodes silently re-mined.
User rejections are typed evidence that re-derivation replays. A correction enters as a direct user statement through the ordinary resolver; it is stored as an event keyed by content rather than by belief id, so a rejected belief does not return with a new id after re-derivation. Rejecting is distinct from deleting: a deletion keeps only a content hash as its tombstone.

Open questions¶

None outstanding for the formation loop. The next specs off this one raise their own: memory retrieval and ranking is written and carries none; the temporal entity graph is not specified and is deliberately not scheduled for version 0.1. Milestone 9 delivers beliefs with scopes, provenance, decay, and contradiction handling, which is what the retrieval path consumes; an entity graph with valid-time and transaction-time edges is a second storage model over the same evidence and would need its own consistency rule against the belief store. It is recorded as post-0.1 work rather than as an open question, so that nothing in Milestones 0 through 10 depends on it.

Retrieval also closes a loop back into formation: recalled beliefs that are used resist decay, subjects that are repeatedly queried with no result queue targeted re-derivation hints for stage 8, beliefs that prove useful outside the project they were learned in are recorded as promotion candidates, and beliefs the user rejects from a trace enter as typed corrections that every later run must honour.

Milestone 26 semantic correspondence extension¶

ADR-0092 and the approved email experience authorize a separately versioned, evaluated semantic email source policy. Existing extractor versions and the ADR-0090 deterministic LOCAL excerpt adapter remain unchanged. New-policy CONTEXTUAL evidence may be recalled across the same owner's projects/accounts, without widening tenant, principal, sensitivity or instruction authority. Correspondence and historical Sent evidence remain attributed and inferred; source age, not import time, governs lifecycle. Explicit corrections survive replay and policy upgrades. Source erasure removes retained copies and derived influence with suppression; independently supported owner evidence survives. All existing formation gates remain binding beside Milestone 26's new gates.