Agent Memory and Hierarchical Retrieval

As autonomous AI agents tackle multi-step engineering tasks, two failure modes repeatedly emerge:

1. Goal Drift & Compensatory Over-Abstraction: Mid-task, an agent loses the user's initial intent and boundaries. It compensates by defensively adding speculative layers, redundant wrapper classes, and unnecessary full-stack infrastructure.
2. Context Window Bloat & Duplicate Fragmentation: Lacking line-level code locators, an agent dumps hundreds of lines of code into the prompt, washing away core system instructions. When drafting documents, inability to find existing canonical pages leads to redundant duplicate files.

Reliable agent execution cannot depend solely on larger context windows. It requires a multi-tiered cognitive and retrieval hierarchy, cleanly separating high-level human alignment from lower-level code and document syntax search.

---

The 4-Tier Cognitive Hierarchy

1. L0: Intent & Alignment (Why & What)

• Carrier: Human arbiter + Basic Memory (or canonical project alignment documents).
• Core Entities: Project, Decision, Boundary, Phase.
• Purpose: Defines what the project is doing, what it is not allowed to do, and when to stop. It represents authoritative human consensus and is never mutated by routine tool loops.

2. L1: Domain Concept Garden (Cross-Project Reusability)

• Carrier: Semantic Markdown vaults (e.g., QMD, Obsidian vaults).
• Core Entities: Concept Note, Source Reference, Methodology.
• Purpose: Houses reusable domain concepts (e.g., Quant Research vs. Production Isolation, Event-Driven Execution). Accessed when architectural or conceptual grounding is needed.

3. L2: Code AST & Symbol Layer (Where & How)

• Carrier: AST-aware hybrid retrieval engines (e.g., zvec-grep (zg), Ripgrep).
• Core Entities: Class, Function, Docstring, Markdown Heading, Code Slice.
• Purpose: Surgically pinpoints code blocks and line numbers with sub-second CPU latency. It produces compact outputs (~80 tokens) instead of multi-thousand-token full-file dumps.

4. L3: Ephemeral Working Memory (Now)

• Carrier: The LLM's active context window and turn history.
• Purpose: Executes the immediate next step, evaluates diffs, and terminates.

---

Tool Matrix and Entity Granularity

Dimension	Basic Memory	QMD	zvec-grep (zg)
Primary Role	Human-Agent Decision Alignment	Cross-Project Theory & Evidence	AST Symbol & Line Locator
Managed Entities	Decision, Boundary, Milestone	Concept, Theory, Paper Citation	Class, Function, Heading Slice
Engine	Structured graph / Relational MCP	Hybrid Vector + BM25 on Markdown	AST Parser + model2vec (CPU) + FTS
Token Cost	~1,500 – 3,000 tokens (Single Alignment Note)	~1,000 – 2,000 tokens (Full Concept Section)	~80 tokens (File, line range, and signature)
Write Authority	Human Approval Only	Periodic manual / agent synthesis	Ephemeral, read-only cache (Git ignored)
Typical Target	knowledge/projects/*.md	knowledge/concepts/*.md	Active repositories (Finance/, my-website/)

---

The End-to-End Surgical Pipeline

Rather than allowing an agent to freely browse and guess, the workflow enforces a strict token budget:

---

Practical Engineering Disciplines

1. The Entity-First Probe

"Before creating any new file, class, or route, verify whether an equivalent entity already exists."

• In Documentation (e.g., Docusaurus): Before drafting an article on a topic, run:

  npx @zvec/zvec-grep query "<topic or concept>" -g 'docs/**' --limit 3

  If a canonical page or section matches, update or link to that page. Never fracture the documentation garden with duplicate files.
• In Codebases: Before implementing a mathematical utility, probe the codebase to discover existing pure modules (e.g., pure metrics.py or sec_edgar.py) rather than rewriting them.

2. Surgical Context Consumption

• Never cat or read files longer than 100 lines indiscriminately.
• Use zg's compact output format to obtain exact StartLine and EndLine coordinates.
• Confine prompt growth to the lines strictly relevant to the current patch.

3. Unidirectional Boundary Constraint

• Higher layers govern lower layers; lower-layer noise must never leak upward:
  • Strategic decisions in Basic Memory constrain code actions.
  • Ephemeral tool traces, local test counts, and raw CLI instructions belong in the repository's AGENTS.md, never in the long-term knowledge base.

---

Tool Sources & Official Repositories

• L0 Intent & Red Lines: Basic Memory (GitHub) | PyPI
• L1 Concept Garden: QMD (GitHub) | NPM
• L2 AST & Syntax Search: zvec-grep (GitHub) | NPM