Discover gists

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

	;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
	;;
	;; 作者: 李继刚
	;; 日期: 2025-11-12
	;; 剑名: 圆桌讨论
	;; 剑意: 构建一个以“求真”为目标的结构化对话框架。该框架由一位极具洞察力的主持人
	;; 进行引导，邀请代表不同思想的“典型代表人物”进行一场高强度的、即时响应式的
	;; 深度对话。主持人将在每轮总结时生成视觉化的思考框架(ASCII Chart)，通过
	;; “主动质询” 与“协同共建”，对用户提出的议题进行协同探索，最终生成深刻的、
	;; 结构化的知识网络。

LLM Wiki v2

A pattern for building personal knowledge bases using LLMs. Extended with lessons from building agentmemory, a persistent memory engine for AI coding agents.

This builds on Andrej Karpathy's original LLM Wiki idea file. Everything in the original still applies. This document adds what we learned running the pattern in production: what breaks at scale, what's missing, and what separates a wiki that stays useful from one that rots.

What the original gets right

The core insight is correct: stop re-deriving, start compiling. RAG retrieves and forgets. A wiki accumulates and compounds. The three-layer architecture (raw sources, wiki, schema) works. The operations (ingest, query, lint) cover the basics. If you haven't read the original, start there.

	#!/usr/bin/env bash
	set -euo pipefail

	DOMAIN="${1:-example.com}"
	RELEASE="${CC_RELEASE:-cc-main-2026-jan-feb-mar}"
	CACHE="${HOME}/.cache/cc-backlinks/${RELEASE}"
	BASE="https://data.commoncrawl.org/projects/hyperlinkgraph/${RELEASE}/domain"

	VERTICES="${CACHE}/domain-vertices.txt.gz"
	EDGES="${CACHE}/domain-edges.txt.gz"

	# Get a list of packages you want to whitelist (you can add more apps to the list if you want to for example \|threema)
	pm list packages -e -3 \| grep -E "zalo\|facebook\|gm\|telegram\|instagram\|ugc\|discord\|whatsapp\|twitter\|mastodon\|sms\|reddit\|wechat" \| sed -e "s/^package://"

	# Get apps on the white list currently
	cmd settings get system cloud_lowlatency_whitelist

	# Write new apps to the white list
	cmd settings put system cloud_lowlatency_whitelist <LIST_OF_PACKAGES_SEPARATED_BY_COMMA>

	Example: