BDH Graph Harness

⚠️ Disclaimer This is an experimental research project implementing biological neural network analogies (Hebbian plasticity, neurogenesis, sleep-cycle consolidation) on Obsidian vault graphs. It is not production software — API endpoints, configuration, and data formats may change without notice between versions.

The theoretical foundation comes from the Dragon Hatchling paper (Kosowski et al., 2025). This implementation is an independent exploration of those ideas, not an official implementation of the paper.

If you're looking for a stable knowledge management tool, consider Obsidian + Dataview or a mature RAG solution. This project is for people who want to experiment with bio-inspired graph learning.

What it does

Turns an Obsidian vault into a living knowledge graph where:

• Notes → neurons — each note embedded with nomic-embed-text-v2-moe (768d, via Ollama) and stored in ChromaDB with OllamaEmbeddingFunction
• Wikilinks → synapses — graph edges from [[wikilinks]]
• Hebbian learning — co-activated notes strengthen their synaptic weight over time (frequency + recency + activation correlation)
• Vector + lexical retrieval — semantic search via Chroma embeddings with optional BM25 Hybrid scoring (hybrid_search: true, α=0.7 / β=0.3)
• Adaptive thresholding — max(Q75, mean+1std, 0.15) to filter noise dynamically
• Neurogenesis — LLM extracts new concepts from queries and creates notes in the vault, filtered by a 3-layer signal system (prompt engineering + regex blocklist + semantic dedup) to prevent noise; generation provenance is kept in frontmatter so it does not pollute embeddings
• Node quality scoring — composite score (strong edges + mean weight + frequency) auto-prunes dormant nodes from visualization; re-activates on strong re-encounter
• Sleep-cycle consolidation — periodic synaptic downscaling (×0.9), structural pruning below weight floor, and stale dormant node removal — mirrors biological sleep consolidation
• Server-side file watcher — source-aware polling detects vault and configured external Markdown changes, debounces editor bursts, and triggers incremental graph/embedding updates with source-safe pruning
• LLM responses — any OpenAI-compatible provider (OpenRouter, Ollama Cloud, local Ollama), with citations back to source notes
• Real-time visualization — WebGL force-graph showing nodes activating, edges pulsing as Hebbian weights update during queries
• Multi-vault isolation — optional vaults: configuration creates one graph state, watcher, BM25 index, lock, and ChromaDB collection per vault; requests select a vault explicitly without cross-contaminating embeddings or state
• Federated Markdown sources — optional read-only external_sources merge selected Markdown trees into the primary graph with source-aware IDs, cross-source wikilinks, and configurable include/exclude globs

For the theory behind these choices — why Hebbian plasticity, why Obsidian, why not just RAG — see docs/philosophy.md.

Neurogenesis Signal Filtering

Neurogenesis creates new notes from concepts the LLM identifies in its response. Without filtering, this generates ~50% noise (model names, internal plumbing, generic process words). Three layers ensure signal-first neurogenesis:

1. Prompt engineering — the system prompt explicitly instructs the LLM what to extract (algorithms, architectures, patterns, lessons) vs what to reject (model names, API providers, internal plumbing, generic words)
2. Regex blocklist — deterministic post-LLM filter catches model names (glm-*, gemma*, mistral-*, etc.), BDH plumbing (graph-refresh, hebbian-update, etc.), generic process words, and too-short slugs
3. Semantic dedup — ChromaDB cosine similarity (threshold 0.65) catches spelling variants and semantic duplicates that exact-string matching misses (e.g. sleepcycle-consolidation vs sleep-cycle-consolidation)

See bdh_graph_harness/neurogenesis/creator.py and dedupe.py for implementation.

Architecture

Obsidian Vault → Embed (Ollama) → ChromaDB + Graph
                                    ↓
Query → Vector Search → Attention Spread (max_hop=2)
                                    ↓
Hebbian Update (co-activation strengthening) → LLM Response (OpenAI-compatible)
                                    ↓
WebSocket → force-graph (WebGL) (nodes light up, synapses pulse)

Sleep Cycle (periodic):
  Synaptic Downscaling (×0.9) → Stale-Weak Pruning → Prune (< floor) → Quality Re-eval → Stale Removal

Package structure

bdh_graph_harness/
├── __main__.py              # CLI entry point (--serve, --mcp, --query, --refresh)
├── config.py                # Config loading, env var expansion, retry logic
├── vaults.py                # VaultConfig, VaultContext, VaultRegistry (multi-vault isolation)
├── mcp_server.py            # MCP server (FastMCP, stdio + HTTP transport)
├── graph/
│   ├── parser.py            # Frontmatter + wikilink parsing
│   ├── builder.py           # Legacy graph construction + incremental cache
│   ├── sources.py           # Vault/external Markdown source scanners
│   ├── federated.py         # Source-aware IDs + federated graph builder
│   └── cache.py             # Graph cache serialization
├── retrieval/
│   ├── embeddings.py        # Ollama embedding client
│   ├── chroma_store.py      # ChromaDB vector store
│   ├── bm25.py              # BM25 lexical index (optional, disabled by default)
│   ├── hybrid.py            # Vector + BM25 fusion (optional, disabled by default)
│   └── attention.py         # Seed selection + k-hop spread + adaptive threshold
├── memory/
│   ├── hebbian.py           # Synaptic weight update + decay
│   ├── quality.py           # Node quality scoring + dormant pruning
│   ├── consolidation.py     # Sleep-cycle: downscaling + pruning + stale removal
│   └── state_store.py       # Persistent state (file-locked)
├── llm/
│   ├── providers.py         # LLM factory + payload builder
│   ├── ollama.py            # Ollama backend
│   ├── openrouter.py        # OpenRouter backend (OpenAI-compatible)
│   └── prompt.py            # System prompt + context formatting
├── neurogenesis/
│   ├── creator.py           # Concept extraction + note creation + noise filtering + source provenance IDs
│   └── dedupe.py            # Exact + semantic duplicate detection (ChromaDB cosine similarity)
├── api/
│   ├── server.py            # aiohttp app setup + WebSocket
│   ├── routes.py            # REST endpoints
│   ├── ws.py                # WebSocket handlers
│   └── watcher.py           # Source-aware polling watcher with debounce
└── visualization/
    └── templates/           # 3D force-graph (WebGL) real-time graph UI + controls + WebSocket

harness.py is a compatibility shim that re-exports from the package — tests use import harness.

Setup

1. Ollama running locally with nomic-embed-text-v2-moe pulled (for embeddings)
2. LLM provider — any OpenAI-compatible endpoint:
   • OpenRouter: set OPENROUTER_API_KEY env var (default config uses openrouter/free)
   • Ollama Cloud: set OLLAMA_API_KEY and point openrouter_url to https://ollama.com/v1/chat/completions
   • Local Ollama: switch llm_provider: ollama in config
   • Any other OpenAI-compatible API works — just set openrouter_url, openrouter_key, and llm_model
3. Python 3.11+ with dependencies:

pip install -r requirements.txt

4. Configure the vault path:

cp bdh-config.yaml bdh-config.local.yaml
# Edit vault_path to point at your Obsidian vault

Usage

# Start server
python -m bdh_graph_harness --serve

# Single query (CLI)
python -m bdh_graph_harness --query "come funziona l'apprendimento Hebbian?"

# Force graph rebuild
python -m bdh_graph_harness --refresh

# Read-only source scan (no ChromaDB, embeddings, LLM, or writes)
python -m bdh_graph_harness --config bdh-config.local.yaml --scan-sources

# Open visualization
open http://localhost:8643

# List configured vaults (multi-vault mode)
python -m bdh_graph_harness --list-vaults

# Target a configured vault from the CLI
python -m bdh_graph_harness --vault-id research --stats

Multi-vault API

Keep the legacy vault_path config for one vault, or use the vaults: list shown in bdh-config.yaml. Each entry is isolated: it has its own graph, Hebbian state, watcher, BM25 index, lock, and ChromaDB collection.

# Query one vault explicitly
curl -X POST http://localhost:8643/api/query \
  -H 'Content-Type: application/json' \
  -d '{"vault_id":"research","query":"How does retrieval work?"}'

# Read stats for one vault, or discover configured vaults
curl 'http://localhost:8643/api/stats?vault_id=research'
curl http://localhost:8643/api/vaults

vault_id is also accepted by MCP tools such as query(question="...", vault_id="research"). Omitting it selects default_vault (or the first configured vault).

Running as a service (macOS)

# Install launchd service
cp ai.bdh.graph-harness.plist ~/Library/LaunchAgents/
launchctl load ~/Library/LaunchAgents/ai.bdh.graph-harness.plist

The service auto-restarts on crash (KeepAlive: true). Logs at ~/.hermes/logs/bdh-server.log. The start-server.sh wrapper exports OPENROUTER_API_KEY from ~/.hermes/.env before launching.

Config

See bdh-config.yaml for all parameters. Key ones:

Tests

# Install runtime + test tooling
pip install -r requirements-dev.txt

# Full suite
python -m pytest -q

# Include statement and branch coverage
python -m pytest -q --cov=bdh_graph_harness --cov-branch --cov-report=term-missing

The current develop baseline is 214 passing tests and 50% package branch coverage. The mobile/provenance branch has been verified with 222 passing tests. The target is 100%, without excluding application modules just to manufacture a prettier number. Regression coverage is already complete for state persistence, consolidation, node quality, BM25, mobile visualization layout, and neurogenesis provenance; the remaining work focuses on API/WebSocket, CLI/MCP, graph/cache, embeddings, and provider failure paths.

See docs/testing.md for the coverage policy, exact commands, and multi-vault regression requirements. docs/coverage.md records the current versioned baseline; GitHub Actions keeps the XML and JSON report for every later develop or main run.

Visualization

The web UI at :8643 shows a real-time force-graph (WebGL) with: The web UI at :8643 shows a real-time 3D force-graph (WebGL via 3d-force-graph) with:

• 3D node rendering — Three.js objects with custom geometries, label sprites, rings, and dashed semantic links
• Nodes colored by activation state or by Obsidian tags (toggle); neurogenesis nodes use aqua (#00E5FF)
• Wikilink edges + Hebbian synapses + Phantom links + Counterpart/project-context edges + Neurogenesis source edges with hover tooltips (weight, type, connected notes)
• Live neurogenesis — new edges appear in real-time as concepts are created; provenance edges connect newborn notes to their source nodes
• Hover highlighting — node hover shows 1-hop subgraph, edge hover highlights the edge
• Node drag, viewport-preserving updates, camera-preserving structural updates, and touch-first mobile graph interactions
• Z-order — wikilinks (bottom) → phantom (middle) → hebbian (top)
• Orphan nodes toggle, tag legend overlay, dark theme, mobile responsive (iPhone safe area, touch dismiss, tab-based layout)
• Node quality — dormant nodes dimmed (gray, 30% opacity) with 💤 tooltip; stats bar shows dormant count and Hebbian tail metrics
• Persisted controls — slider values saved in localStorage, restored on refresh
• Hebbian tail metrics — hebbian_strong_synapses, hebbian_weak_synapses, hebbian_stale_weak_synapses exposed via /api/stats
• Query-response WebSocket event — final ordered event after LLM response so plugin-launched queries are visible in the graph UI

See docs/visualization.md for full details on controls, tooltips, and mobile support. See docs/visualization-3d-migration.md for the 3D migration architecture, lifecycle, and validation details.

MCP Server

The harness includes a Model Context Protocol server that exposes the Hebbian graph as tools to any MCP-compatible client (Claude Desktop, Cursor, Windsurf, Continue).

# stdio mode (Claude Desktop, Cursor)
python -m bdh_graph_harness --mcp

# HTTP mode (web clients)
python -m bdh_graph_harness --mcp --mcp-transport http --mcp-port 8644

Tools: query (grounded Q&A with citations), stats (graph overview), hebbian (learned synapses), graph (full network), refresh (rebuild embeddings).

The MCP server imports the package directly — no dependency on the HTTP API server. Both can run independently or simultaneously.

See docs/mcp-server.md for client configuration (Claude Desktop, Cursor, etc.).

Hermes Agent integration

bdh-hermes-bridge plugin (recommended)

The bdh-hermes-bridge plugin provides bidirectional integration between Hermes Agent and BDH:

• Write path — every Hermes response (>200 chars) is fed to BDH, triggering Hebbian reinforcement and neurogenesis from real usage
• Read path — bdh_query tool lets Hermes pull context from the knowledge graph on demand
• Echo-loop dampening — assistant responses are flagged with source: "assistant_response" to prevent feedback amplification
• User context capture — original user prompt is included in write payloads, enabling proper question→answer synaptic associations

# Install
git clone https://github.com/albidev/bdh-hermes-bridge.git ~/.hermes/plugins/bdh-hermes-bridge

# Enable in ~/.hermes/config.yaml
plugins:
  enabled:
    - bdh-hermes-bridge

Hermes skill (CLI)

The harness also ships with a Hermes Agent skill that lets you query the graph from chat. The skill definition is in docs/hermes-skill.md — copy it to ~/.hermes/skills/research/bdh-graph-harness/SKILL.md to activate it.

Once installed, your Hermes agent can:

• Query the graph via natural language ("bdh query: how does Hebbian learning work?")
• Show graph stats and Hebbian synaptic state
• Start/stop the API server
• Present answers with source citations

Obsidian Sync Plugin

Auto-sync vault changes to the BDH server via an Obsidian plugin — no manual refresh needed.

Obsidian edit → Plugin detects → Debounce 1s → POST /api/node-update
    → Server diffs graph → WebSocket broadcast → Viz updates in real-time

Setup:

1. Build the plugin: cd plugins/obsidian && npm install && npm run build
2. Copy manifest.json + main.js to your vault's .obsidian/plugins/bdh-graph-harness-sync/
3. Enable "BDH Graph Harness Sync" in Obsidian Settings → Community Plugins

Features:

• Debounced updates (1s, configurable) — no server spam
• Status bar indicator (○ idle, ◎ syncing, ● ok, ✗ error)
• Ignores non-.md files and .obsidian/ directory
• Configurable server URL, debounce delay, enable/disable

Pulse animation: Hebbian edges get animated particles during query — color transitions from green through blue, particle count scales with weight gain. Activation state is managed via external Maps to avoid mutating force-graph's live objects.

Sleep-Cycle Consolidation

Periodic graph maintenance that mirrors biological sleep consolidation. Run it manually or schedule it (e.g. nightly via cron):

# Trigger a consolidation cycle
curl -X POST http://localhost:8643/api/consolidate

# Dry run (see what would change without committing)
curl -X POST http://localhost:8643/api/consolidate -H "Content-Type: application/json" -d '{"dry_run": true}'

# View config and cycle count
curl http://localhost:8643/api/consolidation-stats

Cycle steps:

1. Synaptic downscaling — multiply all Hebbian weights by consolidation_downscale_factor (default 0.90). Prevents runaway strengthening.
2. Stale-weak pruning — delete synapses that are below consolidation_weak_weight_threshold (default 0.15), have frequency ≤ consolidation_weak_max_frequency (default 1.0), and were last co-activated more than consolidation_weak_min_age_hours (default 48) ago. Fresh and frequently reinforced weak synapses survive.
3. Structural pruning — delete synapses with weight below consolidation_prune_weight_floor (default 0.02) after downscaling.
4. Quality re-evaluation — recalculate node quality scores and update dormant state.
5. Stale removal — delete nodes dormant for more than consolidation_dormant_persist_cycles (default 3) consecutive cycles, if consolidation_prune_dormant_nodes is true.

No tokens consumed — pure algorithmic operation on the local graph state. Safe to run while the server is serving queries (file-locked state access).

Graph Quality Audit

A read-only audit script is available to check structural and Hebbian invariants without mutating state:

python scripts/audit_graph_quality.py --url http://localhost:8643

Reports: node count, structural edges, invalid endpoints, self-loops, duplicate edges, Hebbian tail classification (strong/weak/stale-weak), generated edge types, and source distribution.

License

MIT