Explorations & Technical Spikes (EXP-*)¶

In modern software development, teams often face critical unknowns: Will this embedded database meet our latency targets? Can this messaging queue handle 50k events/sec? Does this library support our edge deployment target?

Traditional teams handle these questions with ad-hoc "spikes". But traditional spikes suffer from two dangerous extremes: 1. The Write-Only Spike: An engineer tests an idea on a local branch, learns it fails, and deletes it. Six months later, another engineer or AI agent repeats the exact same failed experiment because zero learnings were preserved. 2. The Leaking Prototype: A quick prototype written for a spike gets merged into production because "it works", bringing unverified architectural shortcuts into the core codebase.

Eidos solves this through Explorations (EXP-*): schema-validated, hypothesis-driven spike artifacts that live in Git under eidos/explorations/.

1. The Exploration Lifecycle¶

Explorations follow a strict, auditable lifecycle:

graph TD
    A["Uncertainty / Technical Risk"] --> B["eidos spec create exploration<br/><i>eidos/explorations/exp-*.adoc</i><br/>status: proposed"]
    B --> C["Prototype Branch<br/><i>git checkout -b spike/exp-*</i><br/>status: in_progress"]
    C --> D{"Evaluate Exit Criteria"}
    D -->|Hypothesis Proven| E["status: adopted<br/>Graduates to SpecDelta (DLT-*)"]
    D -->|Hypothesis Disproven| F["status: discarded<br/>Archived Post-Mortem in Git"]

Lifecycle States (exploration.schema.yaml):¶

2. Anatomy of an Exploration Manifest¶

Exploration documents live in eidos/explorations/exp-<year>-<num>.adoc (or EXP-<YEAR>-<NUM>.adoc):

---
type: product/exploration
id: EXP-2026-001
title: "Evaluate SQLite in WAL Mode for Edge Node Caching"
status: adopted # proposed | in_progress | adopted | discarded

decision:
  outcome: adopted # adopted | discarded | pivoted
  resulting_deltas:
    - DLT-2026-010
---
= Exploration: Evaluate SQLite in WAL Mode for Edge Node Caching
:toc: left

== Problem Statement & Hypothesis
Our current edge daemon requires Redis running as a sidecar, increasing memory overhead on constrained edge gateways. We hypothesize that adopting SQLite with WAL mode and memory-mapped I/O will reduce cache read latency by 60% compared to local Redis, while eliminating external daemon dependencies.

== Exit Criteria
* **Max Duration**: 3 days
* **Benchmark**: P99 read latency < 5ms under 50 concurrent readers with zero locking exceptions.

== Questions to Answer
* Does SQLite in WAL mode handle 50 concurrent readers without database locking errors?
* What is the memory footprint under a 500MB working set size?
* How clean is cross-platform compilation across Alpine Linux and macOS ARM64?

== Spikes & Prototypes
* *spike-concurrency-benchmark*: Simulate 50 concurrent reader threads executing 100k read operations against a 500MB database.
* *spike-container-build*: Verify automated static linking inside minimal Docker scratch image.

== Investigation Log & Findings
SQLite WAL mode achieved 1.4ms p99 read latency across 50 threads with 42MB RSS memory.

== Final Decision & Rationale
*(Record outcome and justification when concluding the exploration).*

3. The 5-Step Exploration Protocol¶

Step 1: Scaffold the Exploration¶

Create the exploration document using the Eidos CLI:

eidos spec create exploration EXP-2026-001 \
  --title "Evaluate SQLite in WAL Mode for Edge Node Caching"

Validate the new spec:

eidos validate eidos

Step 2: Branch & Prototype¶

Create an isolated Git branch:

git checkout -b spike/exp-2026-001

Implement the prototype or benchmark in a scratch directory (e.g., scratch/). Never mix exploratory prototype code into production source directories (src/).

Step 3: Record Empirical Data¶

Document concrete, verifiable numbers directly in the AsciiDoc body: * Throughput (requests/sec) * Latency percentiles (p50, p95, p99) * Memory footprint and CPU utilization * Operational friction (build complexity, dependencies)

Step 4: Resolve Outcome¶

Path A: Hypothesis Proven -> Graduate to SpecDelta & Embedded Decision¶

1. Update frontmatter status: adopted and populate the decision: block:

   status: adopted
   decision:
     outcome: adopted
     resulting_deltas:
       - DLT-2026-010
   

2. Scaffold a formal SpecDelta to plan the production integration:

   eidos delta create DLT-2026-010 \
     --title "Integrate SQLite WAL Cache Engine" \
     --kind feature
   

3. Record the enduring architectural trade-off in the target container's decisions: block with graduated_from: EXP-2026-001, and implement clean, production-ready code under the SpecDelta checklist.

Path B: Hypothesis Disproven -> Archive Institutional Knowledge¶

1. Update frontmatter status: discarded and populate decision.outcome:

   status: discarded
   decision:
     outcome: discarded
   

2. Document the exact reasons for failure in the Final Decision & Rationale section.
3. Merge the exploration document to main. The failure is preserved as permanent architectural knowledge, preventing future team members and AI agents from repeating the experiment.

Step 5: Verification Gate¶

Always ensure the repository passes validation before merging:

eidos validate eidos
eidos drift

4. How AI Agents Use Explorations¶

When using the eidos-plan agent skill: * Autonomous coding agents will refuse to make sweeping architectural gambles without first proposing an EXP-*. * Agents run benchmark scripts in isolated workspaces, capture output into the exploration log, and summarize findings for human review. * If a human rejects an approach, the agent marks the exploration discarded and registers the negative constraint so future agent prompts never re-propose it.