Misc Miscellaneous references

Miscellaneous documentation that doesn’t fit elsewhere.

Second Brain Workflow

Documentation for the Paperless-NGX document ingestion pipeline, AI tagging, and knowledge management workflow.

View Second Brain Workflow β†’

Second Brain Workflow

The personal knowledge pipeline β€” document ingestion via Paperless-NGX, semantic search and knowledge graph via the Life Archive RAG system, and structured knowledge management in Tana.

Last updated: May 2026. Previous versions referenced old infrastructure (TrueNAS, Proxmox at .230, paperless-ai) β€” all decommissioned.

Architecture Overview
Source documents (PDFs, email attachments, Evernote exports, magazines)
    ↓
Paperless-NGX (CT100 Β· paperless.edmd.me)
    OCR, full-text index, tagging, archival
    ↓
Life Archive RAG Pipeline (Mac Studio :8900)
    gte-Qwen2-7B embeddings β†’ LanceDB vectors β†’ knowledge graph
    Multi-strategy retrieval: dense + SPLADE + QA pairs + KG + HyDE
    ↓
Query interfaces
    Claude MCP tools (life_archive_search, entity_lookup, etc.)
    HTTP API (:8900)  Β·  KG web explorer (:1313/kg/)
    ↓
Tana β€” structured knowledge workspace
    394+ contacts, 85+ plant species, homelab docs, tasks
Paperless-NGX

Runs on CT100 (hpve) via Docker at /opt/paperless, reachable at paperless.edmd.me (direct: http://192.168.8.100:8100). Migrated from the Mac Studio on 2026-06-21 — pinned paperless-ngx:2.20.7 with its OWN postgres:16 + redis:7 (not the shared PostgreSQL). Nightly pg_dump via paperless-pgdump.sh (00:30) + covered by the nvmepool→Biggest vms backup.

Start/stop:

ssh ct100
cd /opt/paperless
docker compose up -d
docker compose down

Key volumes (on CT100, under /opt/paperless):

Path Purpose
paperless-postgres volume PostgreSQL 16 database
/opt/paperless/media/ Stored documents
/opt/paperless/consume/ Drop files here to ingest (polls every 10s)
/opt/paperless/export/ Bulk export output

Ingest a file: Drop it in /opt/paperless/consume/ on CT100 β€” Paperless OCRs, tags, and indexes automatically within ~10 seconds.

Paperless API:

curl -H "Authorization: Token <see SECRETS.md>" \
  https://paperless.edmd.me/api/documents/

Gotcha: redis proto-max-bulk-len is raised to 2gb in the compose file β€” the django-treenode tag-tree cache blob (~526MB, 10k+ tags) exceeds redis’s 512MB default and crash-loops startup otherwise.

Note: the paperless-ai auto-tagger stays on the Mac Studio (needs local LM Studio at :1234), pointed at this instance.

Life Archive

See the full Life Archive page for complete documentation. Summary:

Metric Value
Total documents ~74K in LanceDB
Paragraphs indexed ~2.69M
Knowledge graph entities ~276K
Sources Evernote, emails, magazines, Tana nodes, Paperless docs
Embedding model gte-Qwen2-7B on Apple MPS (port 1235)
Query API FastAPI at port 8900
MCP server Streamable HTTP at port 8901

Check service status:

launchctl list | grep beedifferent

Quick search:

curl -X POST http://localhost:8900/search \
  -H "Content-Type: application/json" \
  -d '{"query": "your question here"}'
Evernote Source Material

Evernote exports (155 notebooks) were converted using Yarle and are staged at ~/Sync/ED/life_archive/:

Directory Contents
Evernote/ Source ENEX exports
EmailAttachments/ ~9,490 extracted email attachments

Yarle configs:

  • ~/paperless-ngx/yarle_config_tana.json β€” Tana Internal Format output
  • ~/paperless-ngx/yarle_config_paperless.json β€” HTML/MD for Paperless ingestion

Both passes were completed. Evernote notes are fully indexed in both Paperless and the Life Archive.

Tana

Tana is the structured knowledge layer β€” everything that needs relationships, fields, and queries rather than just search.

Two workspaces:

  • Main / BeeDifferent β€” contacts (~394), general knowledge, tasks
  • Brownsville β€” 93-acre property management: 85+ plant species, 9 habitat zones, 10 custom supertags, ecological tracking

MCP integration: tana-local MCP server connects Claude directly to both workspaces. See MCP Servers.

Key supertags: Contact, Place, Thing, Event, Activity, Resource, Concept, Plant (55 fields), Habitat Zone, Observation.

Common Tasks

Ingest a document into Paperless:

scp /path/to/document.pdf ct100:/opt/paperless/consume/

Search the Life Archive: Ask Claude directly β€” life_archive_search MCP tool is connected. Or use the API at http://192.168.8.180:8900/docs.

Check Life Archive services:

launchctl list | grep beedifferent
curl http://localhost:8900/health
curl http://localhost:8901/mcp

Restart embed server:

launchctl kickstart -k gui/$(id -u)/com.beedifferent.embed-server

View logs:

tail -f ~/Sync/ED/life_archive/http_api.stdout.log
tail -f ~/Sync/ED/life_archive/http_api.stderr.log