kb-with-receipts

A knowledge base that shows its receipts — every answer cites the source it came from, down to the file and the section.

Turn scattered product docs, design files, and tutorial videos into a citation-grounded, queryable knowledge base: one Markdown source of truth, two outputs, and a QA/audit workflow that keeps the assistant from making things up.

🌏 Languages: English (this file) · 繁體中文版 → README.zh-TW.md

🚀 New here? The hands-on, beginner-friendly walkthrough is TUTORIAL.md — install the skill and let it coach you through building your own KB.

The problem, in one picture

Looking up product knowledge in most teams is like finding one sentence in a library with no catalog: the books (docs) are scattered across a notes app, a design tool, and a pile of screen recordings; there's no single entrance; and the only real "index" is the one senior teammate everyone keeps interrupting.

This repo is a manual for training a digital librarian — one that answers by typing and, crucially, always points back to exactly which document and section an answer came from. It is the training manual, not anyone's actual library — and installed as an agent skill, it works like a coach that guides you through building the KB yourself, step by step. You bring your own books; the method does the rest.

Distilled and fully anonymized from a real internal deployment that took ~300 scattered source files to a 97%-correct, zero-hallucination, fully-traceable Q&A knowledge base (a representative 30-question audit went 67% → 97%). Every company-, product-, and person-specific detail has been removed.

Why it works (it's not guesswork)

The method maps to two published Anthropic methods, not vibes:

  • Contextual Retrieval — prepend each chunk with its source/context. Anthropic measured this cutting retrieval failures by roughly ~35%. That's the basis for the "tag every section with where it came from" rule below.
  • Agent Skills best practices — progressive disclosure: a lean router file, details loaded on demand. That's the basis for this repo's SKILL.md + references/ shape.

What the reference deployment actually produced, with real (anonymized) numbers:

Result Number
Scattered sources consolidated ~300 source files
Answer accuracy (30-question audit) 67% → 97%, zero fabrication
Cross-file contradictions reconciled 11 of 12
Adversarial full-KB audit 108 files, only 1 must-fix
Concept-page adoption (maker/verifier loop) 8 of 8

Quickstart (one line)

git clone https://github.com/DennisWei9898/kb-with-receipts.git ~/.claude/skills/kb-with-receipts

Then, in any Claude Code session, say "build me a sourced knowledge base" or "consolidate these docs into a queryable KB" — the skill walks you through it. You supply the sources; the template supplies the method, the QA/audit gates, and the upload tooling. Prefer to just read? Open examples/ for a three-minute before/after you can skim without installing anything.

Works with your tools

The method is tool-agnostic. The references/ are plain Markdown you can drop into Claude Code, Cursor, Codex, Copilot CLI, or any assistant that reads a repo. NotebookLM is only the default no-code Q&A output (picked for click-back-to-source citations and a zero learning curve for non-technical teammates) — swap it for Claude Projects or your own backend and the method holds.

The one idea: one source, two mouths

Keep ONE clean Single Source of Truth (SSOT) in Markdown, and feed it to TWO outputs:

  1. a no-code Q&A tool (e.g. NotebookLM) for non-technical teammates, and
  2. a reusable Claude Skill (SKILL.md + references/) for advanced queries & automation.

Edit one place → both stay in sync, and they can never contradict each other.

Four iron rules (this is where the quality comes from)

  1. Copy verbatim, don't rewrite. Numbers, amounts, dates, rules, field names — moved word-for-word. Paraphrasing reads nicely but makes assistants hallucinate. Facts beat fluency.
  2. Tag every section with "where I came from." Each section opens with > Source: {file} | {date}. This is what makes answers retrievable and citable.
  3. Newest wins. When a topic has many versions, the latest is authoritative; older ones only backfill gaps, with the change annotated.
  4. "Faithful to source ≠ correct." Sources contain drafts, mis-pastes, and unbuilt "vision" items. Only a domain expert sampling answers catches these — and every catch becomes a regression test.

The pipeline (like training a new librarian)

Stage What
0 Inventory & taxonomy — drop empties/drafts/dupes; mutually-exclusive buckets
1 Consolidate text — dedupe, newest-wins, copy verbatim, source-tag each section
2 Videos → text — scene-change frames + transcript → timestamped illustrated steps
3 Design files → text — REST-API render → read visually → text specs
4 Publish to the Q&A tool — programmatic upload, split by audience
5 QA loop — ask likely questions, grade, fix, re-test
6 Integrity audit — three-way cross-check to kill "source has it, KB missing it" gaps
∞ Monthly update — machine does the heavy lifting; humans only arbitrate & sample

The single biggest gotcha

NotebookLM silently drops fenced code blocks from retrieval. Any SQL/JSON/command inside a ``` fence becomes un-answerable — and the assistant may hallucinate fake SQL instead. In one audit, un-fencing lifted a 109-question run from ~54% → ~84% and cleared hallucinations to zero. Fix: run any file with code through scripts/make_nlm_upload.py before uploading (content stays byte-identical). Full list in references/05-qa-tool-gotchas.md.

What's inside (and what's new)

The core method is SKILL.md + references/01–05. Six field-tested chapters go beyond "consolidate docs" into keeping a KB honest, current, and grounded in reality:

  • 06 · Consensus pages — resolve cross-file contradictions with a single authoritative "concept page" + bidirectional backlinks, verified "add-only, never break" by checksum.
  • 07 · Adversarial audit — a guilty-until-proven-innocent auditor that lists what to fix with evidence, but never edits your files. Only "must-fix" blocks a release.
  • 08 · Correction flywheel — turn your KB into a chat bot, then turn every "👎 wrong answer" into a queued fix that makes the KB smarter over time.
  • 09 · Audience-adaptive answers — one KB that gives ops a plain-language conclusion and engineers the exact field names, routed by how the question was asked.
  • 10 · Schema grounding — let the KB answer "does our database actually support feature X?" by mapping features to tables and drift-checking against the live schema.
  • 11 · Maker/verifier loop — how an agent can safely edit a KB overnight: a different-model, fresh-context verifier checks only objective facts, with a checksum safety net.

Once your KB is live: give it a LINE face

Once your KB is on NotebookLM and answering real questions correctly, you can give it a chat-app face: people @-mention a bot in a LINE group (or DM it) and get answers straight from your notebook. Every reply carries a source count and a disclaimer, and it costs nothing extra — no self-hosted RAG, no paid LLM, because the brain stays the one NotebookLM notebook you already built. It fits a non-technical team that already lives in LINE, and the same shape swaps cleanly to Slack or Telegram. Because this only pays off after the Q&A is trustworthy, the Skill never pushes it on you — it confirms the KB works, then asks whether you want a chat-app face, and proceeds only if you say yes. Full walkthrough: 12 · Give your KB a LINE face; the de-identified, deployable service lives in templates/line-bot-service/.

Repository layout

SKILL.md                              # the method skill (router + the four rules)
references/
  01-method-overview.md               # plain-language "why" + tool selection
  02-technical-pipeline.md            # architecture + condensing pipeline
  03-qa-and-audit.md                  # QA rubric + four-layer integrity audit
  04-monthly-update-sop.md            # monthly refresh + CI-style coverage diff
  05-qa-tool-gotchas.md               # NotebookLM behaviors that silently wreck quality
  06-consensus-pages.md               # concept pages resolve cross-file contradictions
  07-adversarial-audit.md             # guilty-until-proven auditor, read-only
  08-correction-flywheel.md           # chat bot + user-driven correction loop
  09-audience-adaptive-answers.md     # one KB, two audiences, routed by question
  10-schema-grounding.md              # map features to DB tables; drift-check the schema
  11-maker-verifier-loop.md           # safe agentic KB edits, verified across models
  12-notebooklm-line-bot.md           # give the KB a LINE chat-bot face (opt-in, after QA)
scripts/
  make_nlm_upload.py                  # un-fence code blocks so the Q&A tool can index SQL
examples/                             # 3-minute before/after on a fictional app
templates/line-bot-service/           # deployable NotebookLM-backed LINE bot; see reference 12

FAQ

How is this different from just asking Claude to read my docs? Ad-hoc "read this folder" works until the folder disagrees with itself and the model quietly picks a side. This method's value is the discipline around the docs: verbatim extraction, per-section source tags, newest-wins conflict resolution, an adversarial audit, and a monthly regression test. That's what turns "it answers" into "it answers correctly, completely, and with receipts."

Do I have to use NotebookLM? No. It's the default no-code output because of its click-back-to-source citations. The method is tool-agnostic; only the Stage 4/5 commands and the gotchas in 05 are NotebookLM-specific.

Is it really free to try? Yes — it's MIT-licensed Markdown. Clone it, or just read examples/ for the before/after with no install.

Methodology sources

Decisions & history

See docs/adr/ — this repo records its own design decisions as ADRs (why a Claude Skill, why anonymized, why English-first).

Author & contact

Dennis Wei

License

MIT — free to use, adapt, and share.