Miscellaneous documentation that doesn’t fit elsewhere.
Documentation for the Paperless-NGX document ingestion pipeline, AI tagging, and knowledge management workflow.
Miscellaneous documentation that doesn’t fit elsewhere.
Documentation for the Paperless-NGX document ingestion pipeline, AI tagging, and knowledge management workflow.
The personal knowledge pipeline β document ingestion via Paperless-NGX, semantic search and knowledge graph via the Life Archive RAG system, and structured knowledge management in Tana.
Last updated: May 2026. Previous versions referenced old infrastructure (TrueNAS, Proxmox at .230, paperless-ai) β all decommissioned.
Source documents (PDFs, email attachments, Evernote exports, magazines)
β
Paperless-NGX (CT100 Β· paperless.edmd.me)
OCR, full-text index, tagging, archival
β
Life Archive RAG Pipeline (Mac Studio :8900)
gte-Qwen2-7B embeddings β LanceDB vectors β knowledge graph
Multi-strategy retrieval: dense + SPLADE + QA pairs + KG + HyDE
β
Query interfaces
Claude MCP tools (life_archive_search, entity_lookup, etc.)
HTTP API (:8900) Β· KG web explorer (:1313/kg/)
β
Tana β structured knowledge workspace
394+ contacts, 85+ plant species, homelab docs, tasks
Runs on CT100 (hpve) via Docker at /opt/paperless, reachable at paperless.edmd.me (direct: http://192.168.8.100:8100). Migrated from the Mac Studio on 2026-06-21 β pinned paperless-ngx:2.20.7 with its OWN postgres:16 + redis:7 (not the shared PostgreSQL). Nightly pg_dump via paperless-pgdump.sh (00:30) + covered by the nvmepoolβBiggest vms backup.
Start/stop:
ssh ct100
cd /opt/paperless
docker compose up -d
docker compose down
Key volumes (on CT100, under /opt/paperless):
| Path | Purpose |
|---|---|
paperless-postgres volume |
PostgreSQL 16 database |
/opt/paperless/media/ |
Stored documents |
/opt/paperless/consume/ |
Drop files here to ingest (polls every 10s) |
/opt/paperless/export/ |
Bulk export output |
Ingest a file: Drop it in /opt/paperless/consume/ on CT100 β Paperless OCRs, tags, and indexes automatically within ~10 seconds.
Paperless API:
curl -H "Authorization: Token <see SECRETS.md>" \
https://paperless.edmd.me/api/documents/
Gotcha: redis proto-max-bulk-len is raised to 2gb in the compose file β the django-treenode tag-tree cache blob (~526MB, 10k+ tags) exceeds redis’s 512MB default and crash-loops startup otherwise.
Note: the paperless-ai auto-tagger stays on the Mac Studio (needs local LM Studio at :1234), pointed at this instance.
See the full Life Archive page for complete documentation. Summary:
| Metric | Value |
|---|---|
| Total documents | ~74K in LanceDB |
| Paragraphs indexed | ~2.69M |
| Knowledge graph entities | ~276K |
| Sources | Evernote, emails, magazines, Tana nodes, Paperless docs |
| Embedding model | gte-Qwen2-7B on Apple MPS (port 1235) |
| Query API | FastAPI at port 8900 |
| MCP server | Streamable HTTP at port 8901 |
Check service status:
launchctl list | grep beedifferent
Quick search:
curl -X POST http://localhost:8900/search \
-H "Content-Type: application/json" \
-d '{"query": "your question here"}'
Evernote exports (155 notebooks) were converted using Yarle and are staged at ~/Sync/ED/life_archive/:
| Directory | Contents |
|---|---|
Evernote/ |
Source ENEX exports |
EmailAttachments/ |
~9,490 extracted email attachments |
Yarle configs:
~/paperless-ngx/yarle_config_tana.json β Tana Internal Format output~/paperless-ngx/yarle_config_paperless.json β HTML/MD for Paperless ingestionBoth passes were completed. Evernote notes are fully indexed in both Paperless and the Life Archive.
Tana is the structured knowledge layer β everything that needs relationships, fields, and queries rather than just search.
Two workspaces:
MCP integration: tana-local MCP server connects Claude directly to both workspaces. See MCP Servers.
Key supertags: Contact, Place, Thing, Event, Activity, Resource, Concept, Plant (55 fields), Habitat Zone, Observation.
Ingest a document into Paperless:
scp /path/to/document.pdf ct100:/opt/paperless/consume/
Search the Life Archive:
Ask Claude directly β life_archive_search MCP tool is connected. Or use the API at http://192.168.8.180:8900/docs.
Check Life Archive services:
launchctl list | grep beedifferent
curl http://localhost:8900/health
curl http://localhost:8901/mcp
Restart embed server:
launchctl kickstart -k gui/$(id -u)/com.beedifferent.embed-server
View logs:
tail -f ~/Sync/ED/life_archive/http_api.stdout.log
tail -f ~/Sync/ED/life_archive/http_api.stderr.log