Skip to content

feat: add persistent auto-memory system#3241

Open
mlzoo wants to merge 1 commit into
ultraworkers:mainfrom
mlzoo:feat/persistent-auto-memory
Open

feat: add persistent auto-memory system#3241
mlzoo wants to merge 1 commit into
ultraworkers:mainfrom
mlzoo:feat/persistent-auto-memory

Conversation

@mlzoo

@mlzoo mlzoo commented Jun 9, 2026

Copy link
Copy Markdown

Summary

Add a file-based persistent memory system that allows the AI to retain knowledge across sessions. This implements the same auto-memory pattern used by Claude Code, adapted for Claw Code's Rust architecture.

  • MemoryStore data layer: FNV-1a workspace fingerprinting, YAML frontmatter parser, MEMORY.md index with 200-line cap
  • MemoryRead/MemoryWrite tools: security-hardened (path traversal, symlink, absolute path, 10MB size limit protections)
  • System prompt injection: MEMORY.md content loaded at session start with prompt injection mitigation
  • Config: autoMemoryEnabled option (defaults to true)
  • Doctor: auto-memory health check reporting directory state
  • /memory command: enhanced to show persistent memory info alongside instruction files

Architecture

~/.claw/projects/<workspace-hash>/memory/
├── MEMORY.md          ← index, auto-loaded into system prompt
├── user_role.md       ← memory files (YAML frontmatter + markdown)
└── ...

Memory types: user, feedback, project, reference

Security

  • Path traversal blocked (.. components rejected)
  • Absolute paths rejected
  • Symlink targets refused via symlink_metadata()
  • 10MB file size cap on read/write
  • Prompt injection mitigation: content in fenced code block with trust-lowering notice

Testing

  • 19 unit tests, 700+ workspace tests pass, Clippy clean
  • End-to-end verified with Zhipu GLM-4-Flash via OpenAI-compatible routing:
    • ✅ System prompt shows memory section
    • ✅ MemoryWrite creates files / MemoryRead retrieves files
    • ✅ Cross-session persistence (new session loads MEMORY.md)
    • ✅ Path traversal and absolute path attacks blocked

🤖 Generated with Claude Code

Implement a file-based persistent memory system that allows the AI to
retain knowledge about users, feedback, projects, and references across
sessions. Memory files use YAML frontmatter with markdown body, stored
at ~/.claw/projects/<workspace-hash>/memory/.

Key components:
- MemoryStore data layer with FNV-1a workspace fingerprinting
- MemoryRead/MemoryWrite tools with security hardening (path traversal,
  symlink, absolute path, and size limit protections)
- System prompt injection of MEMORY.md index with prompt injection
  mitigation (fenced code block + trust-lowering notice)
- autoMemoryEnabled config option (defaults to true)
- Doctor health check for auto-memory state
- /memory command enhancement showing persistent memory info

Tested end-to-end with Zhipu GLM-4-Flash via OpenAI-compatible routing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@1716775457damn

Copy link
Copy Markdown

This is a solid implementation. The FNV-1a workspace fingerprinting approach is clean and avoids complex state tracking. The security hardening is comprehensive -- blocking path traversal, symlinks, and absolute paths covers the main attack vectors, and the prompt injection mitigation with fenced code blocks is a pragmatic touch.

One consideration: with the 200-line cap on MEMORY.md, how does the system handle index overflow? Does it use an LRU eviction strategy, or does it simply stop appending once the cap is reached? This matters if users accumulate significant project-specific memory over time. It may be worth exposing the cap as a configurable option (e.g., �utoMemoryIndexMaxLines) so repos with different needs can tune it.

The per-workspace hash directory structure under ~/.claw/projects/ is a clean design choice -- isolates memories by project and avoids cross-contamination.

@mlzoo

mlzoo commented Jun 10, 2026

Copy link
Copy Markdown
Author

Thanks for the careful review and the kind words on the security hardening!

On the 200-line cap: it's a system-prompt budget (~30KB at ~150 chars/line), not a storage limit. The on-disk MEMORY.md is never modified by the runtime — lines beyond the cap are still on disk and reachable via MemoryRead. The individual memory entries also live in separate frontmatter-annotated .md files that aren't subject to the cap at all (a two-tier index/entry architecture).

Curation is delegated to the model itself: the system prompt instructs it to keep entries terse (one line, <150 chars), organize semantically by topic, consolidate related entries, and remove stale ones. When overflow does occur, the rendered index includes a [... N more lines truncated] marker visible to the model, which provides explicit feedback to compact on the next write. So in practice the index stays well under the cap because the curator runs continuously.

We deliberately avoid LRU or automatic eviction — mechanical eviction would silently rewrite a user-visible file the model curates, and access-time signals are a poor proxy for "still relevant" (that judgment is semantic, which the model handles better than any heuristic we'd encode).

On making the cap configurable (autoMemoryIndexMaxLines): rather than expose it now — which risks users setting it too high (prompt bloat, degraded performance) or too low (silent context loss) without a clear "correct" non-default value — I'd prefer to land this as-is and open a follow-up issue to gather real usage data first. If workspaces consistently hit the ceiling in practice, a configurable cap becomes a much better-informed change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants