post-launch wave-5: agent decision lineage / decisions endpoint over Phi

[DubovskiyIM]

Why

Three angles converge on the same primitive that Fold already has but doesn't expose as a feature:

1. Research finding (~/Desktop/IDF/2026-05-03-market-research-provenance-explainable.html): "decision lineage" / "AI assurance" emerging as adjacent category to data-lineage (Atlan, Collibra, Informatica heading there). White space — none of them carry decision-level semantics, only data-flow.
2. GDPR Article 22 post CJEU C-203/22 (Feb 2025): right to "meaningful information about the logic involved in automated decision-making". Compliance-by-design XAI frameworks (XAI-Compliance-by-Design MDPI Dec 2025) explicitly call for structured per-decision audit trails.
3. Fold already has it. Φ event log + anchoring + witnesses → every confirmed effect carries provenance back to the intent that proposed it, the role that allowed it, the conditions that passed, the invariants that held. Witness-of-proof is sitting in the codebase since Phase 2 SDK extraction. We just don't expose it as a query surface yet.

What

A new endpoint surface and a new domain-feature: "decisions" as a first-class queryable artifact derived from Φ.

Endpoint

GET /api/document/:domain/decisions?as=role&entity=Deal&id=42

Returns:

{
  "subject": { "entity": "Deal", "id": "deal_42" },
  "lineage": [
    {
      "effectId": "e_abc123",
      "intentId": "create_deal",
      "alpha": "add",
      "proposer": { "id": "user_x", "role": "customer" },
      "at": "2026-04-12T14:21:03Z",
      "conditions": [
        { "expression": "task.status = 'published'", "evaluated": true }
      ],
      "invariantsChecked": [
        { "name": "deal_status_transition", "passed": true },
        { "name": "max_active_deals_per_customer", "passed": true }
      ],
      "preapproval": null,
      "irreversibility": null
    },
    {
      "effectId": "e_def456",
      "intentId": "release_payment",
      "alpha": "replace",
      "target": "Deal.paidAt",
      "proposer": { "id": "user_x", "role": "customer" },
      "at": "2026-04-15T11:02:18Z",
      "irreversibility": { "point": "high", "at": "release", "reason": "Money is gone — forward-correction only" },
      "preapproval": null,
      "conditions": [
        { "expression": "deal.status = 'completed'", "evaluated": true }
      ],
      "invariantsChecked": [...]
    }
  ],
  "currentState": { ... folded snapshot ... },
  "format": "json | html | pdf"
}

This is not new data — it's a new materialization of existing Φ. The "decisions" reader is a view function over Φ_confirmed filtered by entity+id, joined with intent-schema metadata, formatted as a document graph.

Three demo questions this answers

1. "Why did the agent reject my refund?" → returns the rejected effect lineage with failedCondition + failedCheck
2. "What's the audit trail for Deal #42?" → returns full chain of intents that touched it, who proposed, what invariants held
3. "What if we'd rejected this approval — what would the world look like?" → counterfactual: fold(Φ \ { effect: e_xyz }). This is wave-5.5 — a separate primitive worth its own issue.

Why this is wave-5 and not wave-2/3/4

Wave-2 (deploy-pipeline) — adds new domain. Wave-3 (multi-agent) — adds new pattern (co-selection). Wave-4 (compliance) — adds new ICP (regulated). Wave-5 adds new endpoint surface across all existing domains. Once shipped, every IDF domain gets /decisions for free. Highest leverage.

But also highest design risk:

• Performance: lineage query for a row touched by 100+ effects requires careful indexing. Φ schema has parent_id chain — must traverse efficiently.
• Authorization: lineage exposes who-did-what. filterWorldForRole must apply to lineage rows too — can't show preparer in case study to other preparer.
• PII: even with role-filter, raw cells in lineage can contain PII. Need redaction policy similar to gapPolicy already in /api/agent/world.

Implementation paths

Path A — minimal viable (~1-2 days):

• New server/routes/document.js endpoint :domain/decisions
• Recursive lineage traversal via parent_id (already in Φ schema)
• JSON output only
• Apply filterWorldForRole to each effect row before inclusion
• Example domain: invest, where decisions are: delegate_to_agent, agent_execute_preapproved_order, recompute_risk_score

Path B — full document materialization (~1 week):

• Path A +
• HTML output (server-side rendered with the IDF document materialiser — same one that powers /api/document)
• PDF output (downstream — server-side puppeteer or pdf-lib)
• React component primitive (renderer package): <DecisionLineageView> — for in-app use
• Counterfactual hook: ?withoutEffect=e_xyz query param → folds Φ minus that effect

Path C — runtime XAI surface (~2-3 weeks, post-wave-4):

• Path B +
• Live SSE channel /api/document/:domain/decisions/stream — agent or auditor subscribes
• Compliance-aligned export formats (regulatory schema TBD: SR 11-7 audit format? GDPR Art 22? FDA SaMD?)
• Custom domain decisions.fold.software if we go enterprise

Three rejection types this surface showcases

This is not a new domain, so rejection types overlap with existing domains. But the lineage view exposes them as queryable:

1. Failed condition — agent tries release_payment on uncompleted deal. Lineage shows the rejected effect with failedCondition: { entity: "deal", field: "status", op: "=", expected: "completed", actual: "in_progress" }.
2. Invariant violation cascade — Φ has a cascadeReject trail: when invariant fires post-confirmation, it can rollback subsequent effects. Lineage shows the cascade tree.
3. Preapproval daily-sum hit — agent's 8th order today, dailySum exceeded. Lineage records all 7 preceding orders + the rejection.

What's new that previous waves don't carry

• First read-only-but-rich endpoint: previous quickstarts only show exec (write) + world (snapshot read). Decisions endpoint is a historical / temporal read.
• First time provenance becomes a product feature, not implementation detail: Wave-1-4 use Φ as engine. Wave-5 sells Φ as the artifact.
• Bridges to wave-4: compliance domain's "why was this approved" becomes one URL. Wave-4 enterprise sales motion uses wave-5 endpoint as the demo.

Open questions

1. PDF generation in Docker quickstart? Would balloon the image (puppeteer ~150MB). Lean: skip PDF for quickstart, offer JSON+HTML.
2. Counterfactual queries (?withoutEffect=) are computationally expensive — full Φ re-fold per query. Acceptable for low-traffic enterprise audit surface, not for high-frequency. Lean: rate-limit or paid-tier.
3. Should this be cross-domain? I.e., /api/document/global/decisions?entity=... aggregating across all domains. Useful for federated audit. Hard for: scoping, performance, auth. Lean: per-domain only in wave-5; cross-domain is wave-6.
4. Do agents themselves consume /decisions? Probably yes — gives them retrospective awareness. But that's also recursive: agents learning from past rejections. Lean: yes, expose via /api/agent/:domain/decisions mirror of document endpoint with same role-filter.

When to ship

Two scenarios:

Scenario A (most likely): After wave-2 ships and gets some "but where's the audit trail" feedback in HN/X comments. That's the inbound signal that this is the next-asked feature.

Scenario B (if wave-4 path opens): Enterprise design partner explicitly asks for SOX-404 audit-trail format. Wave-5 becomes the deliverable for that pilot.

Don't open this if:

• Wave-1-3 metrics show people care about building agents, not auditing agents. Then this is over-engineering for the actual user.
• We commit to wave-4 (enterprise compliance) before wave-2 (dev-tool depth). In that case wave-5 becomes part of wave-4 deliverables, not a separate issue.

Source narrative

Φ has carried decision-provenance since the first day. Wave-5 makes it queryable.

"Software you can interrogate."

If wave-5 ships, the comparison-table across competitors gets a new row:

	LangChain	Permit.io	Lakera	Datadog Audit	Fold
Logs what was called	✓	partial		✓	✓
Logs what was rejected		✓	✓	partial	✓
Logs why with structured lineage					✓
Counterfactual: "what would've happened"					✓

This is the row no one else has. Wave-5 productizes it.

[storreira]

Strong foundation here. Your Φ event log architecture gives you a significant head start on Article 22 compliance - most teams are scrambling to retrofit audit capabilities.

Key gap to address: GDPR Article 22 requires explanations be "meaningful and accessible to the data subject" - not just technically complete. Your decision lineage needs to bridge from Φ events to plain-language explanations that satisfy regulators.

Post-C-203/22, supervisory authorities are requiring:

• Human-interpretable decision factors (not just technical provenance)
• Counterfactual explanations ("what would change the outcome")  
• Appeal/contest mechanisms with clear remediation paths