# Condensate - Memory, Retrieval & Governance for AI Agents

> Condensate is open-source infrastructure for autonomous AI agents, delivered as three complementary projects (Apache 2.0, built in Melbourne, Australia): agents that remember with provenance, retrieve with speed, and ship with proof.

## The Condensate Ecosystem
Condensate is three complementary open-source projects:

| Project | Role | Repository |
|---------|------|------------|
| **Condensate Core** | Memory Operating System - semantic graph, provenance, consolidation | github.com/condensate-io/core |
| **TurboQuant Qdrant** | Condensate's Qdrant fork - Google TurboQuant + FastScan SIMD, QJL correction, in-kernel thresholding | github.com/condensate-io/qdrant |
| **Verified Agentic Development (VAD)** | Control-system model - intent, proof, policy, orchestration, audit | github.com/condensate-io/verified-agentic-development |

- **Core** answers: *What does the agent know, and why?*
- **TurboQuant Qdrant** answers: *How fast can we retrieve and score embeddings at scale?*
- **VAD** answers: *What did the agent change, under what policy, with what proof, and who approved it?*

Stack (top to bottom): Agent Hosts (Cursor, Claude Code, Codex, VS Code, Windsurf) -> VAD Control Plane (intent, proof, policy, work queue, MCP gate, evidence, dashboard) -> Condensate Core (graph, provenance, consolidation, MCP) -> TurboQuant Qdrant (Condensate integration, FastScan, AVX2, HNSW).

---

# Condensate Core - The Memory Operating System

> Condensate Core is an open-source Memory Operating System and data protocol designed for AI Agent memory structures. It utilizes deterministic Merkle-DAGs for state tracking and SEC (Strong Eventual Consistency) to ensure conflict-free convergence across decentralized, offline-capable environments. It shifts AI memory from "similarity search over text" to a verifiable, multi-agent causal graph with a Structured Memory Ontology (Entities, Assertions, Relations, Events, Learnings, Policies).

## Core Properties
- **Context Rot & Token Optimization**: Eliminates unmanaged context accumulation in long-running multi-agent swarms and development loops. Natively slashes coding agent token bills by 80-85% through Hebbian path pruning and semantic distillation.
- **Recursive Codebase Onboarding**: Ingests project repository structures via high-performance walking filters that skip binaries, lockfiles, and virtual environments, indexing repository topology directly into a deterministic memory graph.
- **Decentralized Concurrency**: Multiple agents can mutate local memory state simultaneously without coordination.
- **Cryptography-First**: Every state change is hashed and signed, forming an immutable provenance chain.
- **Deterministic Merge**: Concurrent divergent states represent multiple realities that merge without conflict upon data sync (CRDT-like behavior).
- **Offline-First**: Local reads and writes have zero network latency.
- **Data Sovereignty & Portability**: You own the memory state. Agents on GCP, AWS, or OpenAI can read the same graph; no vendor-specific memory silo.

## Architecture & Data Structures
The core data structure is an immutable Directed Acyclic Graph (DAG). Nodes represent memory snapshots; edges represent delta operations.

```typescript
interface CondensateNode {
  hash: string;         // SHA-256 hash of the node payload
  parents: string[];    // Array of parent hashes
  payload: Operation[]; // The semantic diff applied
  signature: string;    // Ed25519 signature of the author
}
```

### Cognitive Processing
- **Hebbian Learning**: "Neurons that fire together, wire together." Memories co-retrieved during successful agent actions form stronger semantic connections automatically.
- **Long-Term Potentiation**: Frequently accessed pathways are reinforced, moving critical knowledge from "transient" to "permanent" storage tiers.
- **Entity Extraction**: Integrated GLiNER NER models identify People, Organizations, and Locations in real-time, populating the Graph automatically.
- **Spreading Activation**: Queries don't just hit the index. They trigger a "wave" of activation through the knowledge graph to uncover 2nd-order relationships.

## Threat Model & Security
- **Byzantine Peers**: Peers can send maliciously structured DAG segments. Condensate relies on deterministic hash-chaining; peers cannot forge history without the private key.
- **Human-in-the-Loop Review**: Condensate includes configurable guardrails. Every assertion extracted passes through instruction injection detection before entering long-term memory.
- **Encryption**: AES-256-GCM for at-rest and transport encryption. Synchronization connections use X25519 elliptic curve Diffie-Hellman.

## SDK & Agent Integration

### Model Context Protocol (MCP) Binding & Exposed Tools

Condensate supports a dual-MCP architecture:
1. **Stdio Bridge Server (`@condensate/core`)**: Zero-config Stdio server mapping workspace-level agent actions (Cursor, Windsurf, Claude Desktop) to memory.
2. **Native Fast HTTP Router (`/mcp` endpoint)**: High-performance FastAPI server running in your backend cluster for data orchestration and analytics.

#### 1. Configuration (Cursor, Windsurf, Claude Desktop)
Add the following to your configuration file (e.g. `claude_desktop_config.json`):

```json
{
  "mcpServers": {
    "condensate": {
      "command": "npx",
      "args": ["-y", "@condensate/core"],
      "env": {
        "CONDENSATE_URL": "http://localhost:8000",
        "CONDENSATE_API_KEY": "sk-your-api-key"
      }
    }
  }
}
```

#### 2. Exposed MCP Tools Directory

##### A. Stdio Bridge Tools (`@condensate/core`)
- **`add_memory`**: Ingest episodic logs and observations.
  - *Parameters*: `text` (string, required), `source` (string), `project_id` (string).
- **`retrieve_memory`**: Query relevant knowledge via the causal graph index.
  - *Parameters*: `query` (string, required).
- **`start_task_session`**: Set task boundaries (e.g. Linear ticket ID) and pre-fetch matching project policies.
  - *Parameters*: `task_id` (string, required), `agent_id` (string), `agent_role` (string).
- **`record_assertion`**: Commit a structured fact triplet directly into the Merkle-DAG.
  - *Parameters*: `subject_text` (string, required), `predicate` (string, required), `object_text` (string, required), `confidence` (number).
- **`checkpoint_state`**: Dump active agent status (open files, cursor position, state variables) for resilient crash recovery.
  - *Parameters*: `state_dump` (object, required).

##### B. Native Fast HTTP Server Tools (`http://localhost:8000/mcp`)
- **`store_memory`**: Store episodic memory and trigger async Hebbian graph condensation.
  - *Parameters*: `content` (string, required), `type` (string, enum: `["episodic", "semantic"]`).
- **`add_data_source`**: Register dynamic web URLs, local files, or **recursive codebase repositories** with high-signal directory walking.
  - *Parameters*: `name` (string, required), `source_type` (string, enum: `["url", "file", "api", "codebase"]`), `configuration` (object containing `path`, `max_file_size`, `allowed_extensions`, `ignore_patterns`).
- **`trigger_data_source`**: Trigger a background crawler and ingestion job.
  - *Parameters*: `source_id` (string, required).
- **`query_graph`**: Search the causal graph for verified, conflict-resolved entities and chronological assertions.
  - *Parameters*: `query` (string, required), `limit` (integer).
- **`get_context_analytics`**: Retrieve graph PageRank centrality weights, Louvain community clusters, and graph betweenness attention bottlenecks.
  - *Parameters*: `limit` (integer).

#### 3. Causal Graph Attention Budget Optimization ($O(1)$ scaling)
Traditional Vector RAG scales linearly with context length ($O(N)$), leading to **unmanaged context rot** and unneeded attention decay. By tracking topological retrieval paths and utilizing Hebbian pruning alongside Louvain semantic cluster consolidation (via `get_context_analytics`), Condensate keeps the context size at a stable, pristine $O(1)$ ceiling. This results in an **80-85% absolute prompt token footprint reduction** over flat vector spaces.


### Codebase Local Onboarding (CLI)
You can recursively ingest your local repository layout and index it into the memory graph with a single command:

```bash
# Ingest the source folder, excluding typical node_modules, binaries, and virtual envs
condensate onboard ./src --max-size=64k
```

### Python SDK
```python
pip install condensate

from condensate.client import CondensateClient

client = CondensateClient("http://localhost:8000", "api_key")
client.add_item(
    project_id="proj_1",
    source="api",
    text="User requested scheduling"
)
```

### TypeScript SDK
```typescript
npm install @condensate-io/sdk

import { CondensatesClient } from '@condensate-io/sdk';

const client = new CondensatesClient("http://localhost:8000", "api_key");
await client.addItem({
    project_id: "proj_1",
    source: "api",
    text: "User requested scheduling"
});
```

## Astrocyte Memory
Regulated long-horizon recall - not bigger RAG. Four layers:
1. **Recall Gate** - question typing + retrieval-mode routing before context assembly
2. **Temporal hierarchy** - supersession, validity windows, memory strata (assertions, events, policies, summaries)
3. **Evidence Verifier** - post-retrieval support, temporal validity, abstention
4. **Answer-aware feedback** - reinforce retrieval paths that matched gold evidence (eval loop)

Product brief: https://condensate.io/astrocyte-memory.html
Technical doc: `docs/architecture/astrocyte_memory.md` in repository.

## LoCoMo-10 (June 2026 fair run)
Dataset: 10 conversations, 1,986 questions. Per-conversation fresh ingest, isolated sessions, shared scoring for all baselines. Artifact: `locomo10_condensate_v53_fair.json`.

| Metric | Condensate | Full transcript | Industry ref. |
|--------|------------|-----------------|---------------|
| Recall (answer in retrieved memory) | **83.6%** | 80.4% | ~92.5% |
| Tokens per question | **~1,647** | ~20,476 | ~7,000 |
| Supersession on user correction | **Yes** | No | Varies |

Notable category recall: open-domain 92.5% (ref 76.0%), temporal 95.6% (ref 92.8%), multi-hop 81.2%, adversarial 58.3% (LOC-018 in progress). +1.4 pts to 85% internal target. Native answer match (not recall-only): 72.6%. Peak fair run: 85.4% (June 2026).

Regenerate: `make test-locomo-v53-fair` then `make test-locomo-report` (WSL + Docker). Reports: https://condensate.io/benchmarks.html · https://condensate.io/locomo10_comparative_report.html

## Testing (Makefile / Docker)
- `make test`: Python unit, TS/Go SDKs, MCP bridge
- `make test-integration`: Postgres + Qdrant
- `make test-benchmarks`: harness + baselines
- `make test-locomo-v53-fair`: Fair full LoCoMo-10 (canonical JSON)
- `make test-locomo-report`: Comparative MD/HTML + failure analysis
- `make test-locomo-full`: Full LoCoMo-10 with all baselines in one report
- `make test-contradiction`: 50 supersession cases (`full_context` fails, structured passes)

Do not compare to QA-only partial LoCoMo runs (~50.5%).

## Core SDKs
| SDK | Install | Package |
|-----|---------|---------|
| Python | `pip install condensate` | sdks/python |
| TypeScript | `npm install @condensate-io/sdk` | sdks/ts |
| MCP Bridge | `npx -y @condensate-io/core` | sdks/mcp-bridge |
| Rust | `cargo add condensate` | sdks/rust |
| Go | `go get github.com/condensate/condensate-go-sdk` | sdks/go |

## Comparisons (Core)
- **vs CRDTs**: Optimized for AI Memory Graphs and Semantic Distillation. Cryptography-first.
- **vs Relational DBs**: P2P Decentralized, Deterministic Merge instead of Last-Write-Wins.
- **vs Vector DBs**: Complete causal state tracking over time, not just append-mostly semantic search.

---

# TurboQuant Qdrant - Condensate's Vector Retrieval Engine

> github.com/condensate-io/qdrant is Condensate engineering work: a fork of Qdrant v1.18.2 that integrates Google's TurboQuant extreme quantization (4-bit, 2-bit, 1-bit modes), plus SIMD FastScan kernels, QJL residual correction, and in-kernel threshold filtering. This is NOT upstream Qdrant - it is our performance path for agent-scale embedding retrieval.

## What We Implemented
- **FastScan block-transposed layout**: groups vectors into 32-vector blocks for AVX2 cache locality and coalesced memory loads.
- **SIMD multi-query scoring**: parallel distance accumulation in 256-bit registers instead of scalar loops.
- **QJL residual correction**: 1-bit Quantized Johnson-Lindenstrauss projection to recover quantization accuracy.
- **SIMD thresholding and in-kernel filtering**: fuses priority-queue threshold checks into AVX2 kernels to reduce branch overhead during HNSW candidate evaluation.
- **Dynamic density fallback**: falls back to point-wise scoring when block density is low, protecting sparse/random batch access patterns.

## Benchmark Highlights
Against the Qdrant v1.18.2 baseline on 10,000 vectors with 4-bit TurboQuant (`Bits4`), Criterion.rs, AVX2:

| Access pattern | Best result | Notes |
|----------------|-------------|-------|
| Dense (contiguous) batch scoring | 1.82x speedup @ dim 128, batch 32 | Up to 1.65x at batch 256; strong gains on plain-index / linear scan paths |
| Sparse (random) batch scoring | overhead without fallback | Density heuristic recommended before FastScan |

Representative dense results: dim 128 / batch 32 = 1.82x (-45% latency); dim 128 / batch 1024 = 1.61x (-38%); dim 768 / batch 32 = 1.15x (-13%).

## Get Qdrant Running
```bash
git clone https://github.com/condensate-io/qdrant
cd qdrant
docker run -p 6333:6333 qdrant/qdrant

# Build and benchmark locally (see docs/DEVELOPMENT.md)
cargo test -p quantization
cargo bench -p quantization --bench turboquant_bench
```

TurboQuant Qdrant is Condensate Core's performance path for embedding retrieval at scale - especially when collections use TurboQuant compression and workloads include dense plain-index scans or HNSW traversal with in-kernel threshold filtering. Docs: docs/BENCHMARKS.md, docs/DEVELOPMENT.md, implementation_tracker.md.

---

# Verified Agentic Development (VAD) - The Governance Loop

> VAD is a control-system model for enterprise software delivery under agentic acceleration. It treats delivery as a closed loop of intent -> proof -> construction -> verification -> release -> feedback, with separation of duties between builder, verifier, and release-guardian roles. The current reference implementation is a local Level 4 orchestrator: an operator-owned control plane on localhost, not hosted SaaS or a cloud control plane.

## What VAD Implements Today
- **Executable Intent Packages (EIP)**: structured intent, invariants, proof obligations, and evidence bundles.
- **Verified agentic loop**: ask assessment, proof mapping, guarded execution, MEES effort scoring, and release gates.
- **Local control-plane server**: lifecycle, readiness, SQLite event ledger, replay dashboard, and approval routing.
- **Active orchestration**: durable work items, scheduler assignment, run/task state projection, stale-client recovery with optional auto-reassignment.
- **Governed MCP gateway**: role-aware tool visibility, high-risk denial, client attribution, and redacted summaries over stdio and local HTTP JSON-RPC.
- **Multi-client packages**: local connector artifacts for Codex, Claude Code, VS Code, Cursor, Windsurf, OpenCode, Generic MCP/A2A, and Antigravity fallback.
- **Plugin operations**: review-only dry-runs, persisted inventory, local apply/uninstall/rollback writers, event-derived dashboard status.
- **Change control**: diff proposal persistence, verifier and release-guardian approval before sandboxed apply, operator intent records.
- **Deterministic verification**: Docker test gate with 570+ passing tests and no paid model calls in default flows.

VAD remains intentionally local and operator-owned. It is not a cloud control plane, managed marketplace, or automatic production deployment system unless you explicitly opt in outside the default reference boundary.

## Get VAD Running
```bash
git clone https://github.com/condensate-io/verified-agentic-development
cd verified-agentic-development
docker build -t vad-test:local .
docker run --rm vad-test:local

# Start the local control plane and dashboard
vad control-plane serve
vad local-os demo
```

## VAD + Core Together
VAD generates structured delivery evidence - verification events, policy decisions, EIP compliance, and retrospective learnings. Condensate Core is the natural long-term store for that evidence as typed Events and Learnings, queryable across projects and agents. See the Condensate Integration Guide in the VAD repo.

## Ecosystem Compatibility
- **Model Providers**: OpenAI, Anthropic, Azure OpenAI, Google Gemini, Mistral
- **Local Inference**: Ollama, LM Studio, LocalAI
- **Agent Frameworks**: LangChain, LlamaIndex, AutoGen, CrewAI
- **Agent Hosts (Core)**: Claude Desktop, Cursor, Windsurf, Codeium
- **Agent Hosts (VAD packages)**: Codex, Claude Code, VS Code, Cursor, Windsurf, OpenCode, Generic MCP/A2A, Antigravity

## Relevant Links
- Definitive Technical Reference: https://condensate.io/
- The Ecosystem: https://condensate.io/#ecosystem
- Verified Agentic Development: https://condensate.io/#vad
- TurboQuant Qdrant: https://condensate.io/#turboquant
- Astrocyte Memory: https://condensate.io/astrocyte-memory.html
- LoCoMo Benchmark Report: https://condensate.io/benchmarks.html
- LoCoMo Comparative Report (HTML): https://condensate.io/locomo10_comparative_report.html
- GitHub Organization: https://github.com/condensate-io
- Condensate Core: https://github.com/condensate-io/core
- TurboQuant Qdrant: https://github.com/condensate-io/qdrant
- Verified Agentic Development: https://github.com/condensate-io/verified-agentic-development
- LinkedIn: https://www.linkedin.com/company/condensate-io/
- Security: https://condensate.io/#security
- Implementation: https://condensate.io/#implementation