Pre-v1.0 Beta — Cognithor is under active development. APIs may change between releases. Not recommended for production customer-facing deployments. Bug reports and feedback welcome via Issues.
While the test suite is extensive (17,000+ test functions, 89% coverage gate), the project has not been battle-tested in production environments. Expect rough edges, breaking changes between versions, and some German-language strings in system prompts and error messages. See Status & Maturity for details. For non-technical users, wait until version 1.0.0 for stable long-term support.
Vibe-Engineered, not vibe-coded. Cognithor is not a weekend hack held together by AI-generated spaghetti. Every module follows a deliberate architecture (PGE-Trinity, 6-phase gateway init, 3-layer security, TRUST-1..10 operational-trust stack), backed by 17,000+ test functions, structured plans, spec compliance reviews, and code quality gates. The AI writes the code — but a human engineers the system. There's a difference.
Why Cognithor?
Most AI assistants send your data to the cloud. Cognithor runs entirely on your machine — with Ollama or LM Studio, no API keys required. Cloud providers are optional, not mandatory.
It replaces a patchwork of tools with one integrated system: 17 channels, 141 MCP tools across 30 modules, 6-tier memory with 4-channel hybrid search, knowledge vault, voice, browser automation, Computer Use, cross-platform social listening, signed audit receipts (TRUST-1..10), resumable batch workflows (CRWE), TUF-Light-signed pack registry, and more — all wired together from day one. The test suite is extensive (17,000+ test functions, 89% coverage gate). See Status & Maturity for what that does and does not guarantee.
In plain terms: Cognithor is an AI assistant that lives entirely on your computer. You talk to it through your terminal, a web UI, Telegram, Discord, or any of 18 supported channels — and it talks back, remembers what you said last week, and acts on your behalf. It can search the web, write and edit files, run shell commands, control your browser, automate your desktop (clicking, typing, reading windows), manage your calendar, and learn new skills over time. Think of it as a local, private, self-improving Jarvis.
Unlike cloud-based assistants, Cognithor keeps all your data on your machine. Your conversations, memories, documents, and credentials never leave your hardware unless you explicitly configure a cloud LLM provider. It works fully offline with Ollama or LM Studio, and it encrypts everything at rest with SQLCipher (AES-256). If privacy matters to you — and it should — this is the architecture you want.
What makes it different from other local AI tools is that Cognithor is not just a chatbot. It is an agent operating system: it plans multi-step tasks, evaluates its own results, learns from mistakes, and improves autonomously. It can control your desktop through Computer Use (screenshots, clicks, keystrokes, window automation), compete in ARC-AGI-3 reasoning benchmarks, and manage a marketplace of community-contributed skills. It is built to grow with you.
Status & Maturity
Cognithor is Beta / Experimental software. It is under rapid, active development.
| Aspect | Status |
|---|---|
| Core agent loop (PGE) | Stable — well-tested and functional |
| Memory system | Stable — 6-tier architecture works reliably |
| CLI channel | Stable — primary development interface |
| Flutter Command Center | Beta — Sci-Fi aesthetic, cross-platform, GEPA pipeline visualization, Robot Office pathfinding, 20+ config pages, chat, voice, learning dashboard |
| Messaging channels (Telegram, Discord, etc.) | Beta — basic flows work, edge cases may break |
| Voice mode / TTS | Alpha — experimental, hardware-dependent |
| Browser automation | Stable — Playwright-based, CAPTCHA solving, stealth mode |
| Computer Use | Stable — 6 phases (Vision, Agent Loop, Planner Intelligence, Security, Robustness, UI Automation) |
| ARC-AGI-3 Benchmark | Beta — 13/25 games solved (24 levels), 4 solver families incl. SmartExplorer |
| Skill Marketplace | Stable — GitHub registry, 5-check validation, publisher verification |
| Windows UI Automation | Beta — pywinauto UIA for exact element coordinates |
| Deployment (Docker, bare-metal) | Beta — tested on limited configurations |
| SSH Remote Execution | Beta — tested against Docker containers, key-based auth |
| Evolution Engine | Stable — all 6 phases complete, autonomous deep learning with quality self-examination, GDPR-compliant |
| Autonomous Task Framework | Beta — task decomposition, self-evaluation, recurring scheduling |
| Background Process Manager | Beta — 6 MCP tools, 5-method ProcessMonitor, SQLite persistence |
| Multi-Agent System | Beta — 5 specialized agents with model/temperature/top_p overrides |
| Audit & Compliance | Beta — HMAC + Ed25519 signatures, RFC 3161 TSA, GDPR Art. 15/33, WORM-ready, hash-chained prev_hash over canonical NFC-JSON, dedicated AuditCategory.REFLECTION channel for autonomous learning (Compliance-Spring v0.98.0) |
| Resilient Workflow Engine (CRWE) | Stable — cognithor task <manifest> with JSONL streaming, atomic checkpoint, file-lock, SIGINT/SIGTERM emergency-checkpoint between tasks, manifest-tamper detection on --resume, audit-chain integration. Crash-recovery integration test passes on Windows under SIGKILL (v0.99.0). |
| Pack Registry Signing | Beta — TUF-Light Ed25519 (offline Root + online Targets) + SHA-256 verifier; marketplace dormant by default until owner mints Root keypair |
| Video Composition & Rendering | Beta — cognithor.video package with pluggable RendererABC; default HyperFrames backend (Apache-2.0); 5 MCP tools (video_compose GREEN, video_render ORANGE, raw HTML RED at Gatekeeper); render-receipt linked to TRUST-1 via run_id; composer prompts + skill templates ship in cognithor.video.skills |
| Enterprise features (GDPR, A2A, Governance) | Stable — GDPR 100% user rights, consent management, SQLCipher encryption, audit trail |
| Encryption at Rest | Stable — SQLCipher (AES-256) for all databases, Fernet for files, OS Keyring key management |
| Cross-Platform Social Listening | Beta — Reddit + Hacker News + Discord scanning, LLM-scored leads, unified MCP tools |
| Hierarchical Document Reasoning | Beta — Tree-based retrieval for PDF/DOCX/HTML/Markdown, LLM-navigated section selection |
| CAG Layer (Cache-Augmented Generation) | Beta — Deterministic prefix generation for LLM KV-cache reuse, prefix + native builders |
| CLI Config TUI | Stable — Interactive terminal config editor (rich + prompt_toolkit), model discovery |
| AST-Based Security | Stable — Python AST + bashlex shell analysis replacing regex-based guards |
| OSINT / HIM Module | Beta — person/project/org investigation with trust scoring |
| Observer Audit Layer | Stable — post-response LLM quality check across 4 dimensions (hallucinations, sycophancy, laziness, tool-ignorance); triggers regeneration or full PGE re-loop; fails open |
What the test suite covers: Unit tests, integration tests, property-based tests (Hypothesis), audit-completeness burn-ins (nightly CI), real-life scenario tests, and live Ollama tests for all modules. The 17,000+ test functions verify code correctness in controlled environments.
What the test suite does NOT cover: Real-world deployment scenarios, network edge cases, long-running stability, multi-user load, hardware-specific voice/GPU issues, or actual LLM response quality.
Important notes for users:
- This project is developed by a solo developer with AI assistance. Code is human-reviewed, but the pace is fast.
- Breaking changes may occur between minor versions. Pin your version if stability matters.
- The default language is German, switchable to English via the Flutter Command Center or
config.yaml. See Language & Internationalization. - For production use, thorough testing in your specific environment is strongly recommended.
- Bug reports and contributions are welcome — see Issues.
Cognithor is a fully local, Ollama/LM Studio-powered, autonomous agent operating system that acts as your personal AI assistant. All data stays on your machine — no cloud, no mandatory API keys, full GDPR compliance. It supports tasks ranging from research, project management, and knowledge organization to file management and automated workflows. Optional cloud LLM providers (OpenAI, Anthropic, Gemini, and 12 more) can be enabled with a single API key. Users can add custom skills and rules to tailor the agent to their needs.
Table of Contents
- Why Cognithor?
- Status & Maturity
- Highlights
- Architecture
- LLM Providers
- Channels
- Quick Start (under 5 minutes)
- Configuration
- Security
- Operational Trust (TRUST-1..10)
- MCP Tools
- Tests
- Deployment
- Language & Internationalization
- License
- What's New
Highlights
- 19 LLM Providers — Ollama (local), LM Studio (local), vLLM (local), llama-cpp-python (local), OpenAI, Anthropic, Google Gemini, Groq, DeepSeek, Mistral, Together AI, OpenRouter, xAI (Grok), Cerebras, GitHub Models, AWS Bedrock, Hugging Face, Moonshot/Kimi, Claude Code — plus any custom OpenAI-compatible endpoint
- 17 Communication Channels — CLI, Web UI, REST API, Telegram, Discord, Slack, WhatsApp, Signal, iMessage, Microsoft Teams, Matrix, Google Chat, Mattermost, Feishu/Lark, IRC, Twitch, Voice (STT/TTS)
- 6-Tier Cognitive Memory — Core identity, episodic logs, semantic knowledge graph, procedural skills, working memory, tactical memory
- 4-Channel Hybrid Search — BM25 full-text + vector embeddings + knowledge graph traversal + hierarchical document reasoning with score fusion
- PGE Architecture — Planner (LLM) -> Gatekeeper (deterministic policy engine) -> Executor (sandboxed)
- Crew-Layer (v0.93.0) — high-level Multi-Agent API (
from cognithor import Crew, CrewAgent, CrewTask) on top of PGE-Trinity. Async kickoff with parallel fan-out, idempotent replay, guardrails, 5 templates (research, customer-support, data-analyst, content, versicherungs-vergleich). - 8-page Quickstart —
docs/quickstart/— frompip installto your first Crew in under 10 minutes, bilingual (DE primary + EN). Includes runnable examples for first-crew, first-tool, first-skill, guardrails, and PKV report. - Security — Platform-adaptive sandbox (bubblewrap on Linux, subprocess+timeout fallback), AST-based Python/Shell code analysis (Python
ast.NodeVisitor+bashlexparser), SHA-256 audit chain, credential vault, runtime token encryption (Fernet AES-256), Gatekeeper policy engine with GREEN/YELLOW/ORANGE/RED risk classification (not independently audited — see Status & Maturity) - Knowledge Vault — Obsidian-compatible Markdown vault with YAML frontmatter, tags,
[[backlinks]], full-text search - Document Analysis — LLM-powered structured analysis of PDF/DOCX/HTML (summary, risks, action items, decisions)
- Video Input — Attach local videos (
.mp4/.webm/.mov/.mkv/.avi) or paste direct video URLs; Qwen3.6-27B (or any video-capable VLM) analyzes them end-to-end via vLLM's nativevideo_urlcontent type. Adaptive frame sampling (fps=3 for short clips,num_frames=32for long) viaffprobe. Single video per turn, served from a 127.0.0.1-only HTTP file server, 24h auto-cleanup. Requires vLLM backend — Windows installer bundles LGPL ffmpeg. Seedocs/vllm-user-guide.md. - VLM Router — Three-tier profile system (
fast/balanced/premium) for video and image understanding. Heuristic classifier inspects the user's prompt and clip metadata, picks the right VLM (Qwen3-VL-8B-Instruct → Qwen3-VL-8B-Thinking → Qwen3.6-27B-NVFP4), exposes the routing decision via TRUST-2 (rule_id+matched_pattern) so a receipt reviewer can replay why a particular model was chosen. Override viawith router.quality_scope("premium"):orconfig.vllm.quality_default. ContextVar-isolated for async safety. Seesrc/cognithor/core/vlm_router.py. - Video Composition & Rendering — Cognithor doesn't only read videos, it can make them. The
cognithor.videopackage ships a thinRendererABCabstraction with HyperFrames (Apache-2.0) as the default backend; future renderers (Remotion, cloud) can be swapped without touching the MCP-tool layer. Five MCP tools cover the workflow:video_compose(GREEN — pure-function, builds a self-contained HTML composition from a structured spec, no subprocess, no FS write),video_compose_explainer(16:9 title-card + body + CTA preset),video_compose_social_cut(vertical 9:16 hook + fast-cut beats + outro),video_caption_overlay(parallel caption track), andvideo_render(ORANGE — renders composition HTML to MP4 / MOV / WebM under~/.cognithor/render/<run_id>/). Raw user-supplied HTML is RED at the Gatekeeper — only structured specs reach the renderer. Render output is linked to the agent run via the samerun_idthe streaming EventEmitter and TRUST-1 receipt API use, so every frame is provenance-attributable end-to-end. Composer prompts + reusable templates live incognithor.video.skills(HF-5). VLM video-read can drive composition directly — see VLM-4 smoke attests/test_video/. - Model Context Protocol (MCP) — 141 tools across 30 modules (filesystem, shell, memory, web, browser, media, vault, synthesis, code, skills, documents, social, kanban, identity, evolution, computer-use, sevDesk, A2A and more) — counts auto-generated into
docs/integrations/catalog.jsonon every release - Computer Use — Complete desktop automation: screenshots, clicking, typing, scrolling, dragging, Windows UI Automation via pywinauto for exact element coordinates, 3-layer security, adaptive wait
- ARC-AGI-3 Benchmark Agent — Compete in ARC Prize 2026: 13/25 games solved (24 levels), 4 solver families (ClusterClick, SequenceClick+SimA*, KeyboardDFS, SmartExplorer), persistent game profiles, multimodal vision (qwen3-vl)
- Distributed Locking — Redis-backed (with file-based fallback) locks for multi-instance deployments
- Durable Message Queue — SQLite-backed persistent queue with priorities, DLQ, and automatic retry
- Prometheus Metrics — /metrics endpoint with Grafana dashboard for production observability
- Skill Marketplace — SQLite-persisted skill marketplace with ratings, search, and REST API
- Community Skill Marketplace — GitHub-hosted registry with publisher verification (4 trust levels), 5-check validation pipeline, ToolEnforcer runtime sandboxing, async install/search/report
- Telegram Webhook — Polling + webhook mode with sub-100ms latency
- Auto-Dependency Loading — Missing optional packages detected and installed at startup
- Agent-to-Agent Protocol (A2A) — Linux Foundation RC v1.0 with full JSON-RPC 2.0 server/client, Planner-level delegation via MCP tools, auto-discovery, SSE streaming
- Integrated Chat — Full chat page in the Flutter Command Center with WebSocket streaming, tool indicators, canvas panel, approval banners, and voice mode
- Flutter Command Center — Cross-platform UI (Flutter 3.41, Web/Desktop/Mobile) with Sci-Fi aesthetic, GEPA pipeline visualization, Robot Office pathfinding, 20+ editable config pages, Observe panel, Knowledge Graph, Voice Mode, Learning Dashboard, Light/Dark theme, 4-language i18n
- Active Learning & Curiosity — CuriosityEngine detects knowledge gaps, KnowledgeConfidenceManager with time decay and feedback, ActiveLearner processes files in background during idle time
- Human Feel — Personality Engine (warmth, humor, greetings), sentiment detection (frustrated/urgent/confused/positive), user preference learning, real-time status callbacks, user-friendly German error messages
- Auto-Detect Channels — Channels activate automatically when tokens are present in
.env— no manual config flags needed - Knowledge Synthesis — Meta-analysis across Memory + Vault + Web with LLM fusion:
knowledge_synthesize(full synthesis with confidence ratings),knowledge_contradictions(fact-checking),knowledge_timeline(causal chains),knowledge_gaps(completeness score + research suggestions) - Adaptive Context Pipeline — Automatic context enrichment before every Planner call: BM25 memory search + vault full-text search + recent episodes, injected into WorkingMemory in <50ms
- Security Hardening — Runtime token encryption (Fernet AES-256) across all channels, TLS support for webhook servers, file-size limits on all upload/processing paths, persistent session mappings in SQLite
- One-Click Start — Double-click
start_cognithor.bat-> browser opens -> click Power On -> done - Enhanced Web Research — 4-provider search fallback (SearXNG -> Brave -> Google CSE -> DuckDuckGo), Jina AI Reader for JS-heavy sites, domain filtering, source cross-checking
- Procedural Learning — Reflector auto-synthesizes reusable skills from successful sessions
- DAG Workflow Engine — Directed acyclic graph execution with parallel branches, conditional edges, cycle detection, automatic retry. Now wired into the Executor for parallel tool execution
- Distributed Workers — Capability-based job routing, health monitoring, failover, dead-letter queue
- Multi-Agent Collaboration — Debate, voting, and pipeline patterns for agent teams
- i18n Language Packs — JSON-based internationalization with SHA-256 integrity verification, German and English included, extensible to any language
- Tool Sandbox Hardening — Per-tool resource limits, network guards, escape detection (8 attack categories)
- GDPR Compliance Toolkit — Data processing logs (Art. 30), retention enforcement, right-to-erasure (Art. 17), audit export
- ARC-AGI-3 Benchmark — Compete in the ARC Prize 2026 ($2M+ prize pool) via the
arc/module: hybrid agent (algorithmic + LLM + CNN), 3 MCP tools (arc_play,arc_status,arc_replay), CLI mode, swarm mode.pip install cognithor[arc] - Multi-Agent Benchmark Scaffold — Reproducible benchmark harness in the top-level
cognithor_bench/package: ownpyproject.toml,cognithor-benchCLI, Cognithor + AutoGen adapters - Deterministic Replay — Record and replay agent executions with what-if analysis and diff comparison
- Agent SDK — Decorator-based agent registration (
@agent,@tool,@hook), project scaffolding - Plugin Remote Registry — Remote manifests with SHA-256 checksums, dependency resolution, install/update/rollback
- Cross-Platform Social Listening — Reddit + Hacker News (Firebase/Algolia) + Discord (REST API v10) scanning with LLM lead scoring. 2 unified MCP tools (
social_scan,social_leads), configurable per-platform intervals, auto-post opt-in - Hierarchical Document Reasoning — Tree-based retrieval (4th search channel): 5 parsers (Markdown, PDF, DOCX, HTML, plaintext) build heading trees, LLM-navigated section selection, no embedding required
- CAG Layer (Cache-Augmented Generation) — Deterministic prefix generation for LLM KV-cache reuse. Prefix builder (Ollama-compatible) + native llama.cpp builder, content normalization, hit-rate metrics
- CLI Config TUI — Interactive terminal config editor (rich + prompt_toolkit): model discovery from live providers, section navigation, validation, save/discard
- AST-Based Security Guards — Python
ast.NodeVisitor+bashlexshell parser replacing regex-based code analysis. Detects imports, subprocess calls, eval, network access, file operations at the syntax tree level _safe_call()Pattern — Unified error handling replacing silentexcept Exception: pass. Failure registry with per-function tracking, async variant, circuit-breaker integration- uv Installer Support — Automatic uv detection for 10x faster installs, transparent pip fallback
- Observer Audit Layer — Every response audited against 4 quality dimensions (hallucination, sycophancy, laziness, tool-ignorance) with differentiated retry strategies. Runs locally with qwen3:32b.
- Operational Trust (TRUST-1..10) — Signed run-receipts (
cognithor receipt show / verify / list / export-all / diff), structured Gatekeeper decision-explanations (rule_id+rule_source+matched_pattern), 15-valueFailureModetaxonomy, pack rollback, six append-only ledgers (Provenance, Permission-Scopes, Tool-Fingerprints, Cloud-Escalation, Cost in micro-USD, Migration). REST:GET /api/crew/trace/{trace_id}/receipt. See Operational Trust anddocs/operational_trust.md - Resilient Workflow Engine (CRWE, v0.99.0) —
cognithor task <manifest.json>runs declarative batch workflows with JSONL streaming, per-taskflush() + os.fsync(), atomic modulo-N.checkpoint.json,.checkpoint.lock(POSIXfcntl.flock/ Windowsmsvcrt.locking), SIGINT/SIGTERM emergency-checkpoint between tasks, manifest-tamper detection on--resume, and audit-chain integration (workflow_resumed/system_checkpoint_createdevents). Crash-recovery usesresults.jsonlline count as source-of-truth. - Hash-Chained Audit Log — Every audit event is HMAC-SHA-256 chained to the previous via
prev_hashover canonical NFC-normalized JSON. The Reflector ships through a dedicatedAuditCategory.REFLECTIONchannel (Compliance-Spring v0.98.0) with nine event types covering causal sequences, weight snapshots, episodic appends, semantic facts, and procedure auto-creation — verifiable end-to-end withcognithor audit verify. Property-based Hypothesis tests + a nightly burn-in CI keep the chain intact. - TUF-Light Pack Registry Signing — Community pack registry uses Ed25519 + SHA-256 (offline Root key, online Targets key) per the TUF-Light spec. Operator runbook:
docs/runbooks/registry_key_rotation.md. Marketplace dormant by default until owner mints Root keypair. - Integrations Catalog — Auto-generated from
src/cognithor/mcp/— seedocs/integrations/catalog.json. DACH-specific: sevDesk REST connector (accounting). - Competitive Analysis —
docs/competitive-analysis/— Cognithor vs AutoGen / MAF / LangGraph / CrewAI. - 17,000+ test functions · 89% coverage gate · 0 lint errors · 0 CodeQL alerts · mypy --strict clean across the new TRUST + CRWE surface
Architecture
flowchart TB
%% ===== ENTRY POINTS =====
subgraph ENTRY["Entry Points"]
UI["Flutter UI / Web UI
(Mobile app available)"]
API["REST API"]
CHANNELS["Messaging Channels
(17 channels — chat, voice, REST)"]
end
%% ===== CORE LOOP =====
subgraph CORE["Autonomous Agent Loop"]
GATEWAY["Gateway
Session + Routing + State"]
CONTEXT["Context Pipeline
Memory + Vault + Episodes"]
subgraph PGE["PGE Core Engine"]
direction LR
PLANNER["Planner
Task Understanding
+ Planning"]
GATEKEEPER["Gatekeeper
Policy + Risk Validation"]
EXECUTOR["Executer
Sandbox + Action"]
end
MEMORY["Cognitive Memory
6-Tier Learning System"]
end
%% ===== EXECUTION =====
subgraph EXEC["Execution Stack"]
WORKFLOW["DAG Engine
Parallel + Retry Logic"]
TOOLS["MCP Tools
(141 tools / 30 modules)"]
LLM["LLM Backends
Local + Cloud Models
(22 API endpoints)"]
end
%% ===== FLOW =====
%% Entry into loop
UI --> GATEWAY
API --> GATEWAY
CHANNELS --> GATEWAY
%% Core loop
GATEWAY --> CONTEXT
CONTEXT --> PLANNER
PLANNER --> GATEKEEPER --> EXECUTOR
%% Execution path
EXECUTOR --> WORKFLOW --> TOOLS --> LLM --> MEMORY
%% Feedback loop
MEMORY --> CONTEXT
%% Visibility (important fix)
EXECUTOR --> UI
PGE Trinity (Planner -> Gatekeeper -> Executor)
Every user request passes through three stages:
- Planner — LLM-based understanding and planning. Analyzes the request, searches memory for relevant context, creates structured action plans with tool calls. Supports re-planning on failures.
- Gatekeeper — Deterministic policy engine. Validates every planned tool call against security rules (risk levels GREEN/YELLOW/ORANGE/RED, sandbox policy, parameter validation). No LLM, no hallucinations, no exceptions.
- Executor — Executes approved actions via DAG-based parallel scheduling (independent actions run concurrently in waves). Shell commands run isolated (Process -> Namespace -> Container), file access restricted to allowed paths.
6-Tier Cognitive Memory
| Tier | Name | Persistence | Purpose |
|---|---|---|---|
| 1 | Core | CORE.md |
Identity, rules, personality |
| 2 | Episodic | Daily log files | What happened today/yesterday |
| 3 | Semantic | Knowledge graph + SQLite | Customers, products, facts, relations |
| 4 | Procedural | Markdown + frontmatter | Learned skills and workflows |
| 5 | Working | RAM (volatile) | Active session context |
| 6 | Tactical | SQLite | Active goals, pending actions, rollback |
Memory search uses a 4-channel hybrid approach: BM25 (full-text search with FTS5, optimized for German compound words) + Vector Search (Ollama embeddings, cosine similarity) + Graph Traversal (entity relations) + Hierarchical Document Reasoning (tree-based section navigation for long documents). Score fusion with configurable weights and recency decay.
Knowledge Vault
In addition to the 6-tier memory, Cognithor includes an Obsidian-compatible Knowledge Vault (~/.cognithor/vault/) for persistent, human-readable notes:
- Folder structure:
recherchen/,meetings/,wissen/,projekte/,daily/ - Obsidian format: YAML frontmatter (title, tags, sources, dates),
[[backlinks]] - 6 tools:
vault_save,vault_search,vault_list,vault_read,vault_update,vault_link - Open the vault folder directly in Obsidian for graph visualization
Reflection & Procedural Learning
After completed sessions, the Reflector evaluates results, extracts facts for semantic memory, and identifies repeatable patterns as procedure candidates. Learned procedures are automatically suggested for future similar requests.
Computer Use Pipeline
User Goal → CUTaskDecomposer → Sub-Tasks
→ UIA (exact coords) / Vision (fallback)
→ 3-Layer Tool Enforcement
→ Adaptive Wait → Next Iteration
Cognithor can control your desktop like a human: take screenshots, analyze them with a vision model, click at pixel coordinates, type text, scroll, and drag. Phase 3 adds Windows UI Automation via pywinauto, which reads the OS accessibility tree to get exact element coordinates without relying on vision alone. A 3-layer security model (allowlist + gatekeeper + tool enforcer) ensures destructive operations require explicit approval.
Architecture Decision Records
docs/adr/0001-pge-trinity-vs-group-chat.md— Why Cognithor uses Planner/Gatekeeper/Executor instead of conversational GroupChat patterns.
Comparison with Other Frameworks
docs/competitive-analysis/— Cognithor vs AutoGen, Microsoft Agent Framework, LangGraph, CrewAI.
LLM Providers
Cognithor auto-detects your backend from API keys. Set one key and models are configured automatically:
| Provider | Backend Type | Config Key | Models (Planner / Executor) |
|---|---|---|---|
| Ollama (local) | ollama |
(none needed) | qwen3:32b / qwen3:8b |
| LM Studio (local) | lmstudio |
(none needed) | (your loaded models) |
| vLLM (local, NVIDIA GPU) | vllm |
(none needed) | qwen3.6:27b NVFP4 (RTX 5090 / 32 GB VRAM target) |
| llama-cpp-python (local) | llama_cpp |
llama_cpp_base_url |
(GGUF model loaded by llama.cpp server) |
| OpenAI | openai |
openai_api_key |
gpt-5.2 / gpt-5-mini |
| Anthropic | anthropic |
anthropic_api_key |
claude-opus-4-6 / claude-haiku-4-5 |
| Google Gemini | gemini |
gemini_api_key |
gemini-2.5-pro / gemini-2.5-flash |
| Groq | groq |
groq_api_key |
llama-4-maverick / llama-3.1-8b-instant |
| DeepSeek | deepseek |
deepseek_api_key |
deepseek-chat (V3.2) |
| Mistral | mistral |
mistral_api_key |
mistral-large-latest / mistral-small-latest |
| Together AI | together |
together_api_key |
Llama-4-Maverick / Llama-4-Scout |
| OpenRouter | openrouter |
openrouter_api_key |
claude-opus-4.6 / gemini-2.5-flash |
| xAI (Grok) | xai |
xai_api_key |
grok-4-1-fast-reasoning / grok-4-1-fast |
| Cerebras | cerebras |
cerebras_api_key |
gpt-oss-120b / llama3.1-8b |
| GitHub Models | github |
github_api_key |
gpt-4.1 / gpt-4.1-mini |
| AWS Bedrock | bedrock |
bedrock_api_key |
claude-opus-4-6 / claude-haiku-4-5 |
| Hugging Face | huggingface |
huggingface_api_key |
Llama-3.3-70B / Llama-3.1-8B |
| Moonshot/Kimi | moonshot |
moonshot_api_key |
kimi-k2.5 / kimi-k2-turbo |
| Claude Code | claude-code |
(uses Claude Code CLI) | (Claude Code model) |
# ~/.cognithor/config.yaml — just set one key, everything else is auto-configured
gemini_api_key: "AIza..."
# That's it. Backend, models, and operation mode are auto-detected.
# Or use LM Studio (local, no API key needed):
llm_backend_type: "lmstudio"
# lmstudio_base_url: "http://localhost:1234/v1" # default
Channels
| Channel | Protocol | Features |
|---|---|---|
| CLI | Terminal REPL | Rich formatting, streaming, /commands, status feedback |
| Web UI | WebSocket | Real-time streaming, voice recording, file upload, dark theme, status events |
| REST API | FastAPI + SSE | Programmatic access, server-sent events |
| Telegram | Bot API (poll + webhook) | Text, voice messages (Whisper STT), photos, documents, webhook mode (<100ms), typing indicator |
| Discord | Gateway + REST | Embeds, reactions, thread support, typing indicator |
| Slack | Socket Mode | Block Kit, interactive buttons, thread support |
| Meta Cloud API | Text, media, location, contacts | |
| Signal | signal-cli bridge | Encrypted messaging, attachments |
| iMessage | PyObjC (macOS) | Native macOS integration |
| Microsoft Teams | Bot Framework v4 | Adaptive cards, approvals |
| Matrix | matrix-nio | Federated, encrypted rooms |
| Google Chat | Chat API | Workspace integration |
| Mattermost | REST API | Self-hosted team chat |
| Feishu/Lark | Bot API | ByteDance enterprise messaging |
| IRC | IRC protocol | Classic internet relay chat |
| Twitch | TwitchIO | Live stream chat integration |
| Voice | Whisper + Piper + ElevenLabs | STT, TTS, wake word (Levenshtein), Konversationsmodus, Piper TTS (Thorsten Emotional) |
Quick Start
Under 5 minutes from download to running agent.
Option A: Windows Installer (recommended for Windows users)
- Download
CognithorSetup-{version}.exefrom the latest release - Run the installer — it bundles Python, Ollama, Flutter UI, and all dependencies
- Launch via Desktop shortcut or Start Menu
- Done — browser opens at
http://localhost:8741
The installer auto-detects your GPU, downloads the right models, and runs a health check. No prerequisites needed.
Option B: Linux / macOS / Source Install
# Clone
git clone https://github.com/Alex8791-cyber/cognithor.git
cd cognithor
# Linux/macOS: Interactive installer (venv, Ollama, models, systemd)
chmod +x install.sh && ./install.sh
# Windows (from source): Double-click install.bat
# Or manual: pip install
pip install -e ".[all,dev]"
Option C: PyPI (any platform)
pip install cognithor # Core only: PGE, CLI, Ollama
pip install cognithor[telegram] # + Telegram channel
pip install cognithor[web] # + Web UI (Flutter Command Center)
pip install cognithor[all] # Everything (large install)
ollama pull qwen3:8b # Pull a model
cognithor # Start
# Upgrade to a specific version:
pip install cognithor[all]==0.99.0
# Upgrade to latest:
pip install --upgrade cognithor[all]
Prerequisites (Option B/C only)
- Python >= 3.12
- Ollama (local, free) or any cloud LLM API key (OpenAI, Anthropic, Gemini, etc.)
After Installation: Pull Models
ollama pull qwen3:32b # Planner (20 GB VRAM)
ollama pull qwen3:8b # Executor (6 GB VRAM)
ollama pull qwen3-embedding:0.6b # Embeddings (500 MB VRAM)
No GPU? Use qwen3:8b for both, or set a cloud API key — models are auto-configured.
Uninstall: Windows: run
uninstall.bat. Linux:./install.sh --uninstall. PyPI:pip uninstall cognithor.
Start (~10 sec)
Option A: One-Click (Windows) — includes a pre-built Web UI, no Node.js needed
Double-click start_cognithor.bat -> Browser opens -> Click "Power On" -> Done.
The launcher auto-installs Python and Ollama via winget if missing. Node.js is only needed for UI development (
npm run dev).
Option B: CLI
cognithor # Interactive CLI
python -m cognithor # Same thing (always works, no PATH needed)
python -m cognithor --lite # Lite mode: qwen3:8b only (6 GB VRAM)
python -m cognithor --no-cli # Headless mode (API only)
COGNITHOR_HOME=~/my-cognithor cognithor # Custom home directory
Windows: If
cognithoris not recognized afterpip install, usepython -m cognithorinstead — this always works regardless of PATH configuration. Alternatively, add Python'sScriptsdirectory to your PATH (typically%APPDATA%\Python\PythonXY\Scriptsor theScriptsfolder inside your venv).
Option C: Flutter Command Center (Development)
cd flutter_app
flutter pub get
flutter run # Desktop, or:
flutter run -d chrome # Web
The Flutter Command Center connects to the Python backend on port 8741. Start the backend first (python -m cognithor --no-cli), then launch the Flutter app. The Chat page opens as the default start page — start talking to Cognithor immediately, or activate Voice Mode for hands-free conversation. The Sci-Fi aesthetic features dark translucent panels, neon accents, and GEPA pipeline visualization.
All configuration — agents, prompts, cron jobs, MCP servers, A2A settings — can be edited and saved through the dashboard. Changes persist to YAML files under ~/.cognithor/.
Legacy React UI (deprecated): The old React + Vite UI in
ui/is deprecated and will be removed in a future release. Use the Flutter Command Center instead.
Windows users: A desktop shortcut named Cognithor is included for convenience.
Channel Auto-Detection
Channels start automatically when their tokens are found in ~/.cognithor/.env:
# ~/.cognithor/.env — just add your tokens, channels activate automatically
COGNITHOR_TELEGRAM_TOKEN=your-bot-token
COGNITHOR_TELEGRAM_ALLOWED_USERS=123456789
COGNITHOR_DISCORD_TOKEN=your-discord-token
COGNITHOR_SLACK_TOKEN=xoxb-your-slack-token
No need to set telegram_enabled: true in the config — the presence of the token is sufficient.
Directory Structure (Auto-Created on First Start)
~/.cognithor/
├── config.yaml # User configuration
├── CORE.md # Identity and rules
├── memory/
│ ├── episodes/ # Daily log files
│ ├── knowledge/ # Knowledge graph files
│ ├── procedures/ # Learned skills
│ └── sessions/ # Session snapshots
├── vault/ # Knowledge Vault (Obsidian-compatible)
│ ├── recherchen/ # Web research results
│ ├── meetings/ # Meeting notes
│ ├── wissen/ # Knowledge articles
│ ├── projekte/ # Project notes
│ ├── daily/
No comments yet
Be the first to share your take.