[Future] Optimize Context Assembly Order for LLM Cache Hits

[frankbria]

Important

Refreshed 2026-06-05 — the original body below targets the obsolete v1 tiered-memory system. The opportunity survives in v2 form: the ReAct loop is already append-only but no cache_control breakpoints exist in the Anthropic adapter. See the refreshed assessment in the comments for the current plan of record.

---

Summary

Audit and optimize context assembly order to maximize LLM provider cache hits, potentially achieving up to 4x cost reduction through prompt caching.

Background: State of the Art

From Philipp Schmid's 5 Practical Tips for Context Engineering:

"Context Ordering Matters: Try to use 'append-only' context, adding new information to the end. This maximizes cache hits reducing cost (4x) and latency."

LLM providers (Anthropic, OpenAI) implement prompt caching where repeated prefixes are cached. If your context window looks like:

[System Prompt] + [Project Context] + [Task History] + [Current Request]

And only [Current Request] changes between calls, the prefix can be cached. But if you reorder or modify earlier sections, the cache is invalidated.

Key principle: Static content first, dynamic content last.

Current State in CodeFRAME

The tiered memory system assembles context, but the ordering of that assembly is unclear:

• Is system prompt consistently first?
• Does tier promotion/demotion cause reordering?
• Are tool definitions stable in position?
• Is task-specific context appended at the end?

With Claude API's prompt caching (available since late 2024), improper ordering directly impacts costs.

Investigation Tasks

1. Map current context assembly order

   • Document the exact sequence: system prompt → X → Y → Z → user message
   • Identify what components are static vs. dynamic per-call
   • Check if tier changes cause mid-context insertions

2. Identify cache-breaking patterns

   • Log context assembly across multiple agent calls in a session
   • Diff consecutive prompts to find what's changing and where
   • Quantify how much prefix is stable vs. changing

3. Implement append-only assembly

   • Restructure context builder to enforce: static_prefix + append_only_dynamic
   • Move all changing content to end of context
   • Ensure tool definitions don't shift position

4. Measure cache hit rates

   • Enable cache metrics from Anthropic API (if using Claude)
   • Compare before/after cost and latency

Success Criteria

• Documented context assembly sequence
• Identified cache-breaking patterns in current implementation
• Refactored to append-only pattern (if beneficial)
• Measured improvement in cache hit rate and cost reduction

Cost Impact Estimate

If CodeFRAME averages 50 LLM calls per task with 10K tokens of static context:

• Without caching: 50 × 10K = 500K input tokens billed
• With caching: 10K + (49 × cached rate) = potentially 75% reduction
• At Claude Sonnet rates: significant $ savings at scale

References

• Anthropic Prompt Caching Documentation
• Context Engineering Tips - Philipp Schmid
• CodeFRAME cost tracking already exists - leverage for measurement

[traycerai]

Warning

Ticket Assist will be discontinued on May 1, 2026. After that date, the feature will no longer be available. We suggest installing the Traycer IDE Extension for continued AI-powered, spec-driven development. Thank you for choosing Traycer!

Plan

Optimize Context Assembly Order for LLM Cache Hits

Observations

The CodeFRAME system currently has:

1. No prompt caching implementation - The file:codeframe/providers/anthropic.py provider doesn't use Anthropic's prompt caching API features (cache_control markers)
2. No system prompt in conversation assembly - The file:codeframe/agents/lead_agent.py chat method (lines 288-361) only sends user/assistant messages without system prompts
3. Tiered context system (HOT/WARM/COLD) in file:codeframe/lib/context_manager.py but unclear ordering during assembly
4. Multiple context assembly points - Lead agent chat, worker agent code generation, and SDK client wrapper each handle context differently
5. No cache_control markers - Messages don't include cache_control: {"type": "ephemeral"} blocks required for Anthropic caching
6. Dynamic context ordering - Context items are loaded from database without stable sorting, causing potential cache invalidation

Approach

The plan implements append-only context assembly with Anthropic prompt caching to maximize cache hit rates and reduce costs. The strategy involves: (1) creating a dedicated context builder that enforces static content first, dynamic content last, (2) adding system prompts to all conversation flows, (3) implementing cache_control markers at optimal boundaries, (4) stabilizing context item ordering to prevent cache invalidation, (5) enabling cache metrics collection to measure improvement, and (6) implementing gradual rollout with comprehensive testing. This aligns with Philipp Schmid's context engineering principle: static content first, dynamic content last, maximizing prefix stability for cache hits.

Implementation Steps

Phase 1: Create Append-Only Context Builder Architecture

1.1 Create Context Builder Module

• Create new file: file:codeframe/lib/context_builder.py
• Implement AppendOnlyContextBuilder class with three sections:
  • static_prefix: System prompt, tool definitions, project context (never changes)
  • stable_context: Tiered context items (changes only on tier updates, but in stable order)
  • dynamic_suffix: Conversation history, current request (changes every call)
• Implement methods:
  • add_system_prompt(prompt: str) - Add system prompt to static prefix
  • add_tool_definitions(tools: List[Dict]) - Add tool definitions to static prefix
  • add_project_context(context: str) - Add project context to static prefix
  • add_context_items(items: List[Dict], tier: str) - Add tiered context items in stable order
  • add_conversation_history(messages: List[Dict]) - Add conversation to dynamic suffix
  • add_user_message(message: str) - Add current user message to dynamic suffix
  • build() - Return assembled context as list of messages
  • build_with_cache_markers() - Return assembled context with cache_control markers
  • get_cache_boundary_position() - Return token count of cacheable prefix
  • get_section_sizes() - Return token counts for each section

1.2 Implement Cache Control Marker Placement

• Add CacheMarkerConfig dataclass with options:
  • enable_caching: bool - Enable/disable caching
  • cache_boundary: str - Where to place marker ("after_system", "after_tools", "after_project_context")
  • cache_ttl: int - Time-to-live in seconds (default: 300)
  • min_cache_tokens: int - Minimum tokens to cache (default: 1024)
• Implement marker placement logic:
  • Place cache_control: {"type": "ephemeral"} after static_prefix
  • Ensure marker is placed before any dynamic content
  • Add configuration option to adjust cache boundary

1.3 Implement Stable Context Item Ordering

• Modify file:codeframe/lib/context_manager.py to add sorting method:
  • Create get_stable_sort_key(item: Dict) -> Tuple method
  • Sort by: tier (HOT → WARM → COLD), then by stable identifier (file path, symbol name), then by insertion order
  • Ensure same items always appear in same order regardless of tier changes
• Add method to context items to provide stable ordering key

1.4 Add System Prompt to All Flows

• Create system prompt templates for each agent type:
  • Lead agent system prompt (discovery, task management, coordination)
  • Worker agent system prompt (code generation, file operations)
  • Create file:codeframe/lib/system_prompts.py with prompt templates
• Modify file:codeframe/agents/lead_agent.py:
  • Add _get_system_prompt() method
  • Update chat() method to use context builder with system prompt
  • Ensure system prompt is consistent across all calls for same agent
• Modify file:codeframe/agents/backend_worker_agent.py:
  • Update generate_code() to use context builder with system prompt
  • Ensure system prompt is consistent across all code generation calls

Phase 2: Integrate Prompt Caching into Anthropic Provider

2.1 Update Anthropic Provider

• Modify file:codeframe/providers/anthropic.py:
  • Update send_message() method signature to accept optional system: str parameter
  • Update send_message() to accept AppendOnlyContextBuilder or raw messages
  • Build context with cache markers if builder provided
  • Pass system prompt via system= parameter to messages.create()
  • Include cache_control markers in message content
  • Add method extract_cache_metrics(response) to parse cache usage from API response

2.2 Enable Cache Metrics Collection

• Capture from Anthropic API response:
  • usage.cache_creation_input_tokens: Tokens written to cache
  • usage.cache_read_input_tokens: Tokens read from cache
  • usage.input_tokens: Non-cached input tokens
• Store metrics in response dictionary for tracking
• Add to file:codeframe/lib/metrics_tracker.py:
  • cache_creation_tokens: Track cache writes
  • cache_read_tokens: Track cache hits
  • cache_hit_rate: Calculate percentage of cached tokens
  • cache_boundary_position: Track where cache marker was placed

2.3 Add Cache Configuration

• Create file:codeframe/lib/cache_config.py with:
  • CacheConfig dataclass with options:
    • enable_caching: bool - Enable/disable caching
    • cache_boundary: str - Where to place cache_control marker
    • cache_ttl: int - Time-to-live for cache
    • min_cache_tokens: int - Minimum tokens to cache
  • get_default_cache_config() - Return default configuration
  • load_cache_config_from_env() - Load from environment variables

Phase 3: Refactor Lead Agent Context Assembly

3.1 Update Lead Agent Chat Method

• Modify file:codeframe/agents/lead_agent.py chat() method (lines 288-361):
  • Create AppendOnlyContextBuilder instance
  • Add system prompt to static_prefix via _get_system_prompt()
  • Add conversation history to dynamic_suffix
  • Add user message to dynamic_suffix
  • Build context with cache markers
  • Pass to Anthropic provider with system prompt parameter
  • Update send_message() call to use new signature

3.2 Stabilize Conversation History

• Ensure conversation history is always loaded in same order (by timestamp or index)
• Modify get_conversation_history() method to:
  • Sort messages by index (user_0, assistant_1, user_2, etc.)
  • Validate message order before returning
  • Add logging for conversation assembly

3.3 Add Cache Metrics to Lead Agent

• Track cache hit rate per conversation
• Log cache metrics after each API call
• Store in database for analysis
• Add method _log_cache_metrics(response) to extract and log metrics

Phase 4: Refactor Worker Agent Context Assembly

4.1 Update Worker Agent Code Generation

• Modify file:codeframe/agents/backend_worker_agent.py generate_code() method (lines 270-376):
  • Create AppendOnlyContextBuilder instance
  • Add system prompt to static_prefix via _get_system_prompt()
  • Add related files/symbols to stable_context (sorted by path)
  • Add task context to dynamic_suffix
  • Build context with cache markers
  • Pass to Anthropic provider with system prompt parameter

4.2 Stabilize Related Files/Symbols Order

• Sort related files by file path (alphabetically)
• Sort symbols within each file by line number
• Add get_stable_sort_key() method to file/symbol objects
• Ensure same files/symbols always appear in same order

4.3 Add Cache Metrics to Worker Agent

• Track cache hit rate per code generation task
• Log cache metrics after each API call
• Correlate cache hits with task complexity

Phase 5: Update SDK Client Integration

5.1 Verify SDK Cache Support

• Check if ClaudeAgentOptions supports cache_control markers
• Review file:codeframe/providers/sdk_client.py for caching support
• If not supported, use direct API path for caching
• Document SDK limitations for caching

5.2 Add Cache Configuration to SDK Client

• Update SDK client to accept CacheConfig
• Pass cache settings to ClaudeAgentOptions if supported
• Fall back to direct API if SDK doesn't support caching
• Add logging for cache configuration

Phase 6: Implement Tiered Context Caching Strategy

6.1 Design Tier-Aware Cache Boundaries

• HOT tier items: Include in cache (frequently accessed, stable)
• WARM tier items: Include in cache (occasionally accessed, stable)
• COLD tier items: Exclude from cache (rarely accessed, may change)
• This reduces cache invalidation from tier changes

6.2 Implement Tier Change Detection

• Add method to detect when tier assignments change
• If tier changes affect cached items, invalidate cache
• Log tier changes for analysis

6.3 Optimize Cache Boundary for Tier Changes

• Place cache boundary after HOT tier items only
• This way, WARM/COLD tier changes don't invalidate cache
• Reduces cache invalidation frequency

Phase 7: Add Comprehensive Logging and Debugging

7.1 Add Context Assembly Logging

• Log context assembly process with:
  • Static prefix size (tokens)
  • Stable context size (tokens)
  • Dynamic suffix size (tokens)
  • Cache boundary position
  • Total context size
• Use debug level for detailed logging

7.2 Add Cache Metrics Logging

• Log after each API call:
  • Cache creation tokens (new cache writes)
  • Cache read tokens (cache hits)
  • Input tokens (non-cached)
  • Cache hit rate percentage
  • Estimated cost savings

7.3 Add Context Diff Logging

• Compare consecutive prompts to identify changes
• Log what changed and where in context
• Identify cache-breaking patterns
• Add method _log_context_diff(prev_context, new_context) to lead agent

Phase 8: Implement Cache Hit Measurement and Reporting

8.1 Extend Metrics Database Schema

• Add columns to token_usage table:
  • cache_creation_tokens: Tokens written to cache
  • cache_read_tokens: Tokens read from cache
  • cache_hit_rate: Percentage of cached tokens
  • cache_boundary_position: Where cache marker was placed
• Create migration for schema update

8.2 Create Cache Metrics Dashboard

• Add metrics collection to file:codeframe/lib/metrics_tracker.py:
  • Total cache creation tokens (cumulative)
  • Total cache read tokens (cumulative)
  • Overall cache hit rate (weighted average)
  • Cost savings from caching
  • Cache hit rate trend over time

8.3 Implement Cache Performance Report

• Create report showing:
  • Cache hit rate by agent type
  • Cache hit rate by task type
  • Cost reduction achieved
  • Comparison to baseline (pre-caching)
  • Recommendations for further optimization

8.4 Add Cache Metrics to API Response

• Include cache metrics in agent response metadata
• Allow clients to see cache performance
• Enable per-task cache analysis

Phase 9: Testing and Validation

9.1 Unit Tests for Context Builder

• Create tests/lib/test_context_builder.py:
  • Test static_prefix assembly
  • Test stable_context ordering
  • Test dynamic_suffix assembly
  • Test cache marker placement
  • Test cache boundary calculation
  • Test section size calculations

9.2 Integration Tests for Cache Behavior

• Create tests/providers/test_anthropic_cache.py:
  • Test cache hit on repeated prompts
  • Test cache invalidation on context changes
  • Test cache metrics collection
  • Test cache boundary placement
  • Test system prompt consistency

9.3 End-to-End Tests for Cache Effectiveness

• Create tests/integration/test_cache_effectiveness.py:
  • Run representative tasks with caching enabled
  • Verify cache hits occur as expected
  • Measure actual cost reduction
  • Compare to baseline metrics

9.4 Performance Tests

• Create tests/performance/test_cache_performance.py:
  • Measure latency impact of caching
  • Verify no performance degradation
  • Measure cache hit rate stability
  • Benchmark context assembly speed

Phase 10: Documentation and Rollout

10.1 Create Implementation Documentation

• Create docs/cache-optimization.md:
  • Document append-only context assembly architecture
  • Document cache control marker placement strategy
  • Document cache configuration options
  • Document cache metrics and reporting
  • Include examples and best practices

10.2 Create Migration Guide

• Document changes to context assembly
• Provide examples of new context builder usage
• Document breaking changes (if any)
• Provide rollback procedure

10.3 Create Operational Guide

• Document how to monitor cache performance
• Document how to troubleshoot cache issues
• Document cache configuration best practices
• Document cost tracking and reporting

10.4 Implement Gradual Rollout

• Phase 1: Enable caching for worker agents (lower risk)
  • Deploy with feature flag ENABLE_WORKER_CACHE
  • Monitor cache metrics for 1 week
  • Verify no performance degradation
• Phase 2: Enable caching for lead agent (higher impact)
  • Deploy with feature flag ENABLE_LEAD_CACHE
  • Monitor cache metrics for 1 week
  • Verify cost reduction
• Phase 3: Enable caching for SDK client (if supported)
  • Deploy with feature flag ENABLE_SDK_CACHE
  • Monitor cache metrics for 1 week
  • Verify compatibility

Success Metrics

Metric	Target	Measurement Method
Cache Hit Rate	70%+	API metrics (cache_read_tokens / total_input_tokens)
Cost Reduction	50%+	Compare pre/post caching costs
Context Assembly Order	100% stable	Verify same order across calls
Cache Invalidation Rate	<5%	Track cache invalidations per 100 calls
Implementation Completion	100%	All phases completed and tested
Performance Impact	<5% latency increase	Measure end-to-end latency

Deliverables

1. AppendOnlyContextBuilder - New context builder class with cache support
2. Updated Anthropic Provider - Integrated prompt caching support
3. Updated Lead Agent - System prompt and cache integration
4. Updated Worker Agent - Stable context ordering and cache integration
5. Cache Configuration - CacheConfig dataclass and configuration options
6. Cache Metrics - Extended database schema and metrics collection
7. Cache Dashboard - Metrics reporting and analysis
8. Unit Tests - Comprehensive test coverage for context builder
9. Integration Tests - Cache behavior validation
10. Documentation - Implementation, migration, and operational guides
11. Rollout Plan - Phased deployment strategy with feature flags

Files to Modify/Create

File	Changes Required
file:codeframe/lib/context_builder.py	NEW - AppendOnlyContextBuilder class
file:codeframe/lib/cache_config.py	NEW - Cache configuration
file:codeframe/lib/system_prompts.py	NEW - System prompt templates
file:codeframe/providers/anthropic.py	Add cache support, cache metrics extraction
file:codeframe/agents/lead_agent.py	Add system prompt, use context builder
file:codeframe/agents/backend_worker_agent.py	Stable ordering, use context builder
file:codeframe/lib/context_manager.py	Add stable ordering, tier-aware caching
file:codeframe/lib/metrics_tracker.py	Add cache metrics collection
file:codeframe/persistence/database.py	Extend schema with cache metrics
file:codeframe/providers/sdk_client.py	Add cache configuration support
tests/lib/test_context_builder.py	NEW - Unit tests for context builder
tests/providers/test_anthropic_cache.py	NEW - Integration tests for caching
tests/integration/test_cache_effectiveness.py	NEW - E2E tests for caching
tests/performance/test_cache_performance.py	NEW - Performance tests
docs/cache-optimization.md	NEW - Implementation documentation

Cost Impact Estimate (Projected)

Current State (No Caching):

• Average task: 50 LLM calls
• Average static context: 10K tokens per call
• Total input tokens: 50 × 10K = 500K tokens
• Cost at Claude Sonnet rates ($3/MTok): $1.50 per task

With Caching (70% hit rate - Target):

• First call: 10K tokens (full cost)
• Remaining 49 calls: 10K tokens cached (90% discount)
• Cached tokens: 49 × 10K = 490K tokens
• Cost: $0.30 (first call) + $0.147 (cached calls) = $0.447 per task
• Projected savings: 70% cost reduction ($1.05 per task)

At Scale (1000 tasks/month - Projected):

• Current cost: $1,500/month
• With caching: $447/month
• Projected monthly savings: $1,053

Timeline

• Phase 1 (Architecture): 2-3 days - Create context builder and cache markers
• Phase 2 (Provider Integration): 2-3 days - Update Anthropic provider
• Phase 3 (Lead Agent): 1-2 days - Refactor lead agent
• Phase 4 (Worker Agent): 1-2 days - Refactor worker agent
• Phase 5 (SDK Integration): 1 day - Update SDK client
• Phase 6 (Tier Caching): 1-2 days - Implement tier-aware caching
• Phase 7 (Logging): 1 day - Add comprehensive logging
• Phase 8 (Metrics): 2-3 days - Implement metrics and reporting
• Phase 9 (Testing): 3-4 days - Unit, integration, and E2E tests
• Phase 10 (Documentation): 2-3 days - Create guides and rollout plan
• Total: 17-25 days for complete implementation and testing

Import In IDE

🤖 Prompt for AI Agents

# Optimize Context Assembly Order for LLM Cache Hits

## Observations

The CodeFRAME system currently has:
1. **No prompt caching implementation** - The `file:codeframe/providers/anthropic.py` provider doesn't use Anthropic's prompt caching API features (cache_control markers)
2. **No system prompt in conversation assembly** - The `file:codeframe/agents/lead_agent.py` chat method (lines 288-361) only sends user/assistant messages without system prompts
3. **Tiered context system** (HOT/WARM/COLD) in `file:codeframe/lib/context_manager.py` but unclear ordering during assembly
4. **Multiple context assembly points** - Lead agent chat, worker agent code generation, and SDK client wrapper each handle context differently
5. **No cache_control markers** - Messages don't include `cache_control: {"type": "ephemeral"}` blocks required for Anthropic caching
6. **Dynamic context ordering** - Context items are loaded from database without stable sorting, causing potential cache invalidation

## Approach

The plan implements append-only context assembly with Anthropic prompt caching to maximize cache hit rates and reduce costs. The strategy involves: (1) creating a dedicated context builder that enforces static content first, dynamic content last, (2) adding system prompts to all conversation flows, (3) implementing cache_control markers at optimal boundaries, (4) stabilizing context item ordering to prevent cache invalidation, (5) enabling cache metrics collection to measure improvement, and (6) implementing gradual rollout with comprehensive testing. This aligns with Philipp Schmid's context engineering principle: static content first, dynamic content last, maximizing prefix stability for cache hits.

## Implementation Steps

### Phase 1: Create Append-Only Context Builder Architecture

**1.1 Create Context Builder Module**
- Create new file: `file:codeframe/lib/context_builder.py`
- Implement `AppendOnlyContextBuilder` class with three sections:
  - `static_prefix`: System prompt, tool definitions, project context (never changes)
  - `stable_context`: Tiered context items (changes only on tier updates, but in stable order)
  - `dynamic_suffix`: Conversation history, current request (changes every call)
- Implement methods:
  - `add_system_prompt(prompt: str)` - Add system prompt to static prefix
  - `add_tool_definitions(tools: List[Dict])` - Add tool definitions to static prefix
  - `add_project_context(context: str)` - Add project context to static prefix
  - `add_context_items(items: List[Dict], tier: str)` - Add tiered context items in stable order
  - `add_conversation_history(messages: List[Dict])` - Add conversation to dynamic suffix
  - `add_user_message(message: str)` - Add current user message to dynamic suffix
  - `build()` - Return assembled context as list of messages
  - `build_with_cache_markers()` - Return assembled context with cache_control markers
  - `get_cache_boundary_position()` - Return token count of cacheable prefix
  - `get_section_sizes()` - Return token counts for each section

**1.2 Implement Cache Control Marker Placement**
- Add `CacheMarkerConfig` dataclass with options:
  - `enable_caching: bool` - Enable/disable caching
  - `cache_boundary: str` - Where to place marker ("after_system", "after_tools", "after_project_context")
  - `cache_ttl: int` - Time-to-live in seconds (default: 300)
  - `min_cache_tokens: int` - Minimum tokens to cache (default: 1024)
- Implement marker placement logic:
  - Place `cache_control: {"type": "ephemeral"}` after static_prefix
  - Ensure marker is placed before any dynamic content
  - Add configuration option to adjust cache boundary

**1.3 Implement Stable Context Item Ordering**
- Modify `file:codeframe/lib/context_manager.py` to add sorting method:
  - Create `get_stable_sort_key(item: Dict) -> Tuple` method
  - Sort by: tier (HOT → WARM → COLD), then by stable identifier (file path, symbol name), then by insertion order
  - Ensure same items always appear in same order regardless of tier changes
- Add method to context items to provide stable ordering key

**1.4 Add System Prompt to All Flows**
- Create system prompt templates for each agent type:
  - Lead agent system prompt (discovery, task management, coordination)
  - Worker agent system prompt (code generation, file operations)
  - Create `file:codeframe/lib/system_prompts.py` with prompt templates
- Modify `file:codeframe/agents/lead_agent.py`:
  - Add `_get_system_prompt()` method
  - Update `chat()` method to use context builder with system prompt
  - Ensure system prompt is consistent across all calls for same agent
- Modify `file:codeframe/agents/backend_worker_agent.py`:
  - Update `generate_code()` to use context builder with system prompt
  - Ensure system prompt is consistent across all code generation calls

### Phase 2: Integrate Prompt Caching into Anthropic Provider

**2.1 Update Anthropic Provider**
- Modify `file:codeframe/providers/anthropic.py`:
  - Update `send_message()` method signature to accept optional `system: str` parameter
  - Update `send_message()` to accept `AppendOnlyContextBuilder` or raw messages
  - Build context with cache markers if builder provided
  - Pass system prompt via `system=` parameter to `messages.create()`
  - Include cache_control markers in message content
  - Add method `extract_cache_metrics(response)` to parse cache usage from API response

**2.2 Enable Cache Metrics Collection**
- Capture from Anthropic API response:
  - `usage.cache_creation_input_tokens`: Tokens written to cache
  - `usage.cache_read_input_tokens`: Tokens read from cache
  - `usage.input_tokens`: Non-cached input tokens
- Store metrics in response dictionary for tracking
- Add to `file:codeframe/lib/metrics_tracker.py`:
  - `cache_creation_tokens`: Track cache writes
  - `cache_read_tokens`: Track cache hits
  - `cache_hit_rate`: Calculate percentage of cached tokens
  - `cache_boundary_position`: Track where cache marker was placed

**2.3 Add Cache Configuration**
- Create `file:codeframe/lib/cache_config.py` with:
  - `CacheConfig` dataclass with options:
    - `enable_caching: bool` - Enable/disable caching
    - `cache_boundary: str` - Where to place cache_control marker
    - `cache_ttl: int` - Time-to-live for cache
    - `min_cache_tokens: int` - Minimum tokens to cache
  - `get_default_cache_config()` - Return default configuration
  - `load_cache_config_from_env()` - Load from environment variables

### Phase 3: Refactor Lead Agent Context Assembly

**3.1 Update Lead Agent Chat Method**
- Modify `file:codeframe/agents/lead_agent.py` `chat()` method (lines 288-361):
  - Create `AppendOnlyContextBuilder` instance
  - Add system prompt to static_prefix via `_get_system_prompt()`
  - Add conversation history to dynamic_suffix
  - Add user message to dynamic_suffix
  - Build context with cache markers
  - Pass to Anthropic provider with system prompt parameter
  - Update `send_message()` call to use new signature

**3.2 Stabilize Conversation History**
- Ensure conversation history is always loaded in same order (by timestamp or index)
- Modify `get_conversation_history()` method to:
  - Sort messages by index (user_0, assistant_1, user_2, etc.)
  - Validate message order before returning
  - Add logging for conversation assembly

**3.3 Add Cache Metrics to Lead Agent**
- Track cache hit rate per conversation
- Log cache metrics after each API call
- Store in database for analysis
- Add method `_log_cache_metrics(response)` to extract and log metrics

### Phase 4: Refactor Worker Agent Context Assembly

**4.1 Update Worker Agent Code Generation**
- Modify `file:codeframe/agents/backend_worker_agent.py` `generate_code()` method (lines 270-376):
  - Create `AppendOnlyContextBuilder` instance
  - Add system prompt to static_prefix via `_get_system_prompt()`
  - Add related files/symbols to stable_context (sorted by path)
  - Add task context to dynamic_suffix
  - Build context with cache markers
  - Pass to Anthropic provider with system prompt parameter

**4.2 Stabilize Related Files/Symbols Order**
- Sort related files by file path (alphabetically)
- Sort symbols within each file by line number
- Add `get_stable_sort_key()` method to file/symbol objects
- Ensure same files/symbols always appear in same order

**4.3 Add Cache Metrics to Worker Agent**
- Track cache hit rate per code generation task
- Log cache metrics after each API call
- Correlate cache hits with task complexity

### Phase 5: Update SDK Client Integration

**5.1 Verify SDK Cache Support**
- Check if ClaudeAgentOptions supports cache_control markers
- Review `file:codeframe/providers/sdk_client.py` for caching support
- If not supported, use direct API path for caching
- Document SDK limitations for caching

**5.2 Add Cache Configuration to SDK Client**
- Update SDK client to accept `CacheConfig`
- Pass cache settings to ClaudeAgentOptions if supported
- Fall back to direct API if SDK doesn't support caching
- Add logging for cache configuration

### Phase 6: Implement Tiered Context Caching Strategy

**6.1 Design Tier-Aware Cache Boundaries**
- HOT tier items: Include in cache (frequently accessed, stable)
- WARM tier items: Include in cache (occasionally accessed, stable)
- COLD tier items: Exclude from cache (rarely accessed, may change)
- This reduces cache invalidation from tier changes

**6.2 Implement Tier Change Detection**
- Add method to detect when tier assignments change
- If tier changes affect cached items, invalidate cache
- Log tier changes for analysis

**6.3 Optimize Cache Boundary for Tier Changes**
- Place cache boundary after HOT tier items only
- This way, WARM/COLD tier changes don't invalidate cache
- Reduces cache invalidation frequency

### Phase 7: Add Comprehensive Logging and Debugging

**7.1 Add Context Assembly Logging**
- Log context assembly process with:
  - Static prefix size (tokens)
  - Stable context size (tokens)
  - Dynamic suffix size (tokens)
  - Cache boundary position
  - Total context size
- Use debug level for detailed logging

**7.2 Add Cache Metrics Logging**
- Log after each API call:
  - Cache creation tokens (new cache writes)
  - Cache read tokens (cache hits)
  - Input tokens (non-cached)
  - Cache hit rate percentage
  - Estimated cost savings

**7.3 Add Context Diff Logging**
- Compare consecutive prompts to identify changes
- Log what changed and where in context
- Identify cache-breaking patterns
- Add method `_log_context_diff(prev_context, new_context)` to lead agent

### Phase 8: Implement Cache Hit Measurement and Reporting

**8.1 Extend Metrics Database Schema**
- Add columns to token_usage table:
  - `cache_creation_tokens`: Tokens written to cache
  - `cache_read_tokens`: Tokens read from cache
  - `cache_hit_rate`: Percentage of cached tokens
  - `cache_boundary_position`: Where cache marker was placed
- Create migration for schema update

**8.2 Create Cache Metrics Dashboard**
- Add metrics collection to `file:codeframe/lib/metrics_tracker.py`:
  - Total cache creation tokens (cumulative)
  - Total cache read tokens (cumulative)
  - Overall cache hit rate (weighted average)
  - Cost savings from caching
  - Cache hit rate trend over time

**8.3 Implement Cache Performance Report**
- Create report showing:
  - Cache hit rate by agent type
  - Cache hit rate by task type
  - Cost reduction achieved
  - Comparison to baseline (pre-caching)
  - Recommendations for further optimization

**8.4 Add Cache Metrics to API Response**
- Include cache metrics in agent response metadata
- Allow clients to see cache performance
- Enable per-task cache analysis

### Phase 9: Testing and Validation

**9.1 Unit Tests for Context Builder**
- Create `tests/lib/test_context_builder.py`:
  - Test static_prefix assembly
  - Test stable_context ordering
  - Test dynamic_suffix assembly
  - Test cache marker placement
  - Test cache boundary calculation
  - Test section size calculations

**9.2 Integration Tests for Cache Behavior**
- Create `tests/providers/test_anthropic_cache.py`:
  - Test cache hit on repeated prompts
  - Test cache invalidation on context changes
  - Test cache metrics collection
  - Test cache boundary placement
  - Test system prompt consistency

**9.3 End-to-End Tests for Cache Effectiveness**
- Create `tests/integration/test_cache_effectiveness.py`:
  - Run representative tasks with caching enabled
  - Verify cache hits occur as expected
  - Measure actual cost reduction
  - Compare to baseline metrics

**9.4 Performance Tests**
- Create `tests/performance/test_cache_performance.py`:
  - Measure latency impact of caching
  - Verify no performance degradation
  - Measure cache hit rate stability
  - Benchmark context assembly speed

### Phase 10: Documentation and Rollout

**10.1 Create Implementation Documentation**
- Create `docs/cache-optimization.md`:
  - Document append-only context assembly architecture
  - Document cache control marker placement strategy
  - Document cache configuration options
  - Document cache metrics and reporting
  - Include examples and best practices

**10.2 Create Migration Guide**
- Document changes to context assembly
- Provide examples of new context builder usage
- Document breaking changes (if any)
- Provide rollback procedure

**10.3 Create Operational Guide**
- Document how to monitor cache performance
- Document how to troubleshoot cache issues
- Document cache configuration best practices
- Document cost tracking and reporting

**10.4 Implement Gradual Rollout**
- Phase 1: Enable caching for worker agents (lower risk)
  - Deploy with feature flag `ENABLE_WORKER_CACHE`
  - Monitor cache metrics for 1 week
  - Verify no performance degradation
- Phase 2: Enable caching for lead agent (higher impact)
  - Deploy with feature flag `ENABLE_LEAD_CACHE`
  - Monitor cache metrics for 1 week
  - Verify cost reduction
- Phase 3: Enable caching for SDK client (if supported)
  - Deploy with feature flag `ENABLE_SDK_CACHE`
  - Monitor cache metrics for 1 week
  - Verify compatibility

## Success Metrics

| Metric | Target | Measurement Method |
|--------|--------|-------------------|
| Cache Hit Rate | 70%+ | API metrics (cache_read_tokens / total_input_tokens) |
| Cost Reduction | 50%+ | Compare pre/post caching costs |
| Context Assembly Order | 100% stable | Verify same order across calls |
| Cache Invalidation Rate | <5% | Track cache invalidations per 100 calls |
| Implementation Completion | 100% | All phases completed and tested |
| Performance Impact | <5% latency increase | Measure end-to-end latency |

## Deliverables

1. **AppendOnlyContextBuilder** - New context builder class with cache support
2. **Updated Anthropic Provider** - Integrated prompt caching support
3. **Updated Lead Agent** - System prompt and cache integration
4. **Updated Worker Agent** - Stable context ordering and cache integration
5. **Cache Configuration** - CacheConfig dataclass and configuration options
6. **Cache Metrics** - Extended database schema and metrics collection
7. **Cache Dashboard** - Metrics reporting and analysis
8. **Unit Tests** - Comprehensive test coverage for context builder
9. **Integration Tests** - Cache behavior validation
10. **Documentation** - Implementation, migration, and operational guides
11. **Rollout Plan** - Phased deployment strategy with feature flags

## Files to Modify/Create

| File | Changes Required |
|------|-----------------|
| `file:codeframe/lib/context_builder.py` | **NEW** - AppendOnlyContextBuilder class |
| `file:codeframe/lib/cache_config.py` | **NEW** - Cache configuration |
| `file:codeframe/lib/system_prompts.py` | **NEW** - System prompt templates |
| `file:codeframe/providers/anthropic.py` | Add cache support, cache metrics extraction |
| `file:codeframe/agents/lead_agent.py` | Add system prompt, use context builder |
| `file:codeframe/agents/backend_worker_agent.py` | Stable ordering, use context builder |
| `file:codeframe/lib/context_manager.py` | Add stable ordering, tier-aware caching |
| `file:codeframe/lib/metrics_tracker.py` | Add cache metrics collection |
| `file:codeframe/persistence/database.py` | Extend schema with cache metrics |
| `file:codeframe/providers/sdk_client.py` | Add cache configuration support |
| `tests/lib/test_context_builder.py` | **NEW** - Unit tests for context builder |
| `tests/providers/test_anthropic_cache.py` | **NEW** - Integration tests for caching |
| `tests/integration/test_cache_effectiveness.py` | **NEW** - E2E tests for caching |
| `tests/performance/test_cache_performance.py` | **NEW** - Performance tests |
| `docs/cache-optimization.md` | **NEW** - Implementation documentation |

## Cost Impact Estimate (Projected)

**Current State (No Caching):**
- Average task: 50 LLM calls
- Average static context: 10K tokens per call
- Total input tokens: 50 × 10K = 500K tokens
- Cost at Claude Sonnet rates ($3/MTok): $1.50 per task

**With Caching (70% hit rate - Target):**
- First call: 10K tokens (full cost)
- Remaining 49 calls: 10K tokens cached (90% discount)
- Cached tokens: 49 × 10K = 490K tokens
- Cost: $0.30 (first call) + $0.147 (cached calls) = $0.447 per task
- **Projected savings: 70% cost reduction ($1.05 per task)**

**At Scale (1000 tasks/month - Projected):**
- Current cost: $1,500/month
- With caching: $447/month
- **Projected monthly savings: $1,053**

## Timeline

- **Phase 1 (Architecture)**: 2-3 days - Create context builder and cache markers
- **Phase 2 (Provider Integration)**: 2-3 days - Update Anthropic provider
- **Phase 3 (Lead Agent)**: 1-2 days - Refactor lead agent
- **Phase 4 (Worker Agent)**: 1-2 days - Refactor worker agent
- **Phase 5 (SDK Integration)**: 1 day - Update SDK client
- **Phase 6 (Tier Caching)**: 1-2 days - Implement tier-aware caching
- **Phase 7 (Logging)**: 1 day - Add comprehensive logging
- **Phase 8 (Metrics)**: 2-3 days - Implement metrics and reporting
- **Phase 9 (Testing)**: 3-4 days - Unit, integration, and E2E tests
- **Phase 10 (Documentation)**: 2-3 days - Create guides and rollout plan
- **Total**: 17-25 days for complete implementation and testing

---

Execution Information

Branch: main
Commit: 7d5caab

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[traycerai]

I have updated the Implementation Plan to reflect the recent changes in the ticket.

Previous Plan

Observations

The CodeFRAME system currently has:

1. No prompt caching implementation - The file:codeframe/providers/anthropic.py provider doesn't use Anthropic's prompt caching API features
2. No system prompt in conversation assembly - The file:codeframe/agents/lead_agent.py chat method only sends user/assistant messages without system prompts
3. Tiered context system (HOT/WARM/COLD) in file:codeframe/lib/context_manager.py but unclear ordering during assembly
4. Multiple context assembly points - Lead agent chat, worker agent code generation, and SDK client wrapper each handle context differently
5. No cache_control markers - Messages don't include cache_control: {"type": "ephemeral"} blocks required for Anthropic caching

Approach

The plan focuses on implementing append-only context assembly with Anthropic's prompt caching to achieve up to 4x cost reduction. The strategy involves: (1) auditing current context assembly across all agent types to document exact ordering, (2) identifying cache-breaking patterns where dynamic content appears before static content, (3) refactoring to enforce static-first, dynamic-last ordering with proper cache_control markers, and (4) measuring cache hit rates through Anthropic API response headers. This aligns with Philipp Schmid's context engineering principle: static content first, dynamic content last, maximizing prefix stability for cache hits.

Implementation Steps

Phase 1: Audit Current Context Assembly Order

1.1 Document Lead Agent Context Assembly

• Map the exact sequence in file:codeframe/agents/lead_agent.py chat() method (lines 170-240)
• Current flow: conversation history → user message → API call
• Issue: No system prompt included in messages.create() call
• Issue: Conversation history may contain reordered messages from tier changes

1.2 Document Worker Agent Context Assembly

• Examine file:codeframe/agents/backend_worker_agent.py generate_code() method (lines 260-364)
• Current flow: system prompt (via system= parameter) → user prompt with task/files/symbols
• SDK path: system prompt in ClaudeAgentOptions → user message
• Direct API path: system prompt as separate parameter → user message
• Good: System prompt is separate and stable
• Issue: Related files/symbols order may vary between calls

1.3 Document Context Item Loading

• Review file:codeframe/agents/worker_agent.py load_context() method (lines 204-238)
• Loads context items filtered by tier (HOT/WARM/COLD)
• Issue: Tier changes via update_tiers_for_agent() can cause items to move between tiers, potentially reordering context

1.4 Create Context Assembly Sequence Diagram

sequenceDiagram
    participant User
    participant LeadAgent
    participant AnthropicProvider
    participant ClaudeAPI
    
    User->>LeadAgent: chat(message)
    LeadAgent->>LeadAgent: get_conversation_history()
    Note over LeadAgent: Loads user/assistant messages<br/>NO system prompt included
    LeadAgent->>LeadAgent: append user message
    LeadAgent->>AnthropicProvider: send_message(conversation)
    Note over AnthropicProvider: messages.create()<br/>NO system parameter<br/>NO cache_control markers
    AnthropicProvider->>ClaudeAPI: POST /messages
    ClaudeAPI-->>AnthropicProvider: response
    AnthropicProvider-->>LeadAgent: response
    LeadAgent->>LeadAgent: save to database
    LeadAgent-->>User: assistant response

Loading

Phase 2: Identify Cache-Breaking Patterns

2.1 Analyze Conversation History Ordering

• In file:codeframe/agents/lead_agent.py, conversation history is loaded from database via get_conversation()
• Messages are stored with timestamps but loaded in insertion order
• Cache-breaking pattern: If conversation history is modified or reordered, cache invalidates

2.2 Analyze Context Item Ordering

• In file:codeframe/lib/context_manager.py, update_tiers_for_agent() (lines 92-146) recalculates scores and reassigns tiers
• Items can move from HOT→WARM→COLD based on age/access patterns
• Cache-breaking pattern: If context items are loaded by tier and tier assignments change, the order changes

2.3 Analyze Tool Definition Positioning

• SDK client in file:codeframe/providers/sdk_client.py passes tools via ClaudeAgentOptions
• Tools are static per agent type but position in prompt is SDK-controlled
• Potential issue: SDK may inject tool definitions at varying positions

2.4 Analyze Related Files/Symbols Ordering

• In file:codeframe/agents/backend_worker_agent.py generate_code(), related files/symbols are added to prompt
• Order depends on codebase index query results
• Cache-breaking pattern: If index returns results in different order, prompt prefix changes

2.5 Create Cache-Breaking Pattern Table

Component	Current Behavior	Cache Impact	Severity
System Prompt	Not included in Lead Agent	No caching possible	Critical
Conversation History	Loaded in insertion order	Stable if append-only	Low
Context Items (Tiers)	Tier changes cause reordering	Cache invalidation	High
Related Files/Symbols	Order varies by index query	Cache invalidation	Medium
Tool Definitions	SDK-controlled position	Unknown	Low
User Message	Always at end	Stable (expected)	None

Phase 3: Implement Append-Only Context Assembly

3.1 Add System Prompt to AnthropicProvider

• Modify file:codeframe/providers/anthropic.py send_message() method
• Add system parameter to messages.create() call
• Accept system prompt as optional parameter in method signature
• Ensure system prompt is passed from Lead Agent

3.2 Enable Prompt Caching in AnthropicProvider

• Add anthropic-beta: prompt-caching-2024-07-31 header to API client initialization
• Modify message structure to include cache_control markers
• Add cache_control to system prompt: {"type": "text", "text": system_prompt, "cache_control": {"type": "ephemeral"}}
• Add cache_control to stable context sections (e.g., last message in conversation history before current turn)

3.3 Refactor Lead Agent Context Assembly

• Modify file:codeframe/agents/lead_agent.py chat() method
• Ensure system prompt is loaded and passed to provider
• Structure messages as: system prompt (cached) → conversation history (cached) → current user message (not cached)
• Add cache_control marker after conversation history to maximize cache hit

3.4 Stabilize Context Item Ordering

• Modify file:codeframe/agents/worker_agent.py load_context() method
• Load context items in deterministic order: sort by (tier, created_at, id)
• Ensure tier changes don't affect relative ordering within tiers
• Consider loading all tiers together and filtering in-memory to maintain stable order

3.5 Stabilize Related Files/Symbols Ordering

• Modify file:codeframe/agents/backend_worker_agent.py build_context() method
• Sort related files alphabetically before adding to prompt
• Sort related symbols by (file_path, line_number) for deterministic order
• Document that codebase index should return stable ordering

3.6 Create Context Assembly Configuration

• Add configuration class for cache control settings
• Define which sections should be cached (system prompt, conversation history, context items)
• Allow per-agent-type cache configuration
• Default: cache everything except current user message

3.7 Implement Context Assembly Order Enforcement

flowchart TD
    A[Start Context Assembly] --> B[Add System Prompt]
    B --> C[Mark System Prompt as Cacheable]
    C --> D[Add Tool Definitions]
    D --> E[Mark Tools as Cacheable]
    E --> F[Add Context Items - Sorted]
    F --> G[Mark Context Items as Cacheable]
    G --> H[Add Conversation History - Sorted]
    H --> I[Mark Last History Message as Cacheable]
    I --> J[Add Current User Message]
    J --> K[NO Cache Control on User Message]
    K --> L[Send to API]
    L --> M{Cache Hit?}
    M -->|Yes| N[4x Cost Reduction]
    M -->|No| O[Full Processing]
    N --> P[Return Response]
    O --> P

Loading

Phase 4: Measure Cache Hit Rates

4.1 Add Cache Metrics Collection

• Modify file:codeframe/providers/anthropic.py to capture cache metrics from API response
• Anthropic API returns cache hit info in response headers: anthropic-ratelimit-input-tokens-cache-hit
• Store metrics: input_tokens, output_tokens, cache_creation_tokens, cache_read_tokens

4.2 Extend MetricsTracker

• Modify file:codeframe/lib/metrics_tracker.py to track cache metrics
• Add fields to token usage records: cache_hit_tokens, cache_miss_tokens
• Calculate cache hit rate: cache_hit_tokens / (cache_hit_tokens + cache_miss_tokens)

4.3 Add Cache Metrics to Database

• Extend token_usage table schema to include cache metrics
• Add columns: cache_creation_tokens, cache_read_tokens, cache_hit_rate
• Create migration for schema update

4.4 Create Cache Metrics Dashboard

• Add API endpoint in file:codeframe/ui/server.py to retrieve cache metrics
• Calculate aggregate metrics: total cache hit rate, cost savings, tokens saved
• Display metrics per project, per agent type, per time period

4.5 Implement Before/After Comparison

• Add feature flag to enable/disable prompt caching
• Run A/B test: 50% of requests with caching, 50% without
• Compare: cost per request, latency, cache hit rate
• Document findings in metrics dashboard

Phase 5: Optimize for Maximum Cache Hits

5.1 Implement Cache Warmup Strategy

• On agent initialization, send a warmup request with full context
• Ensures cache is populated before real requests
• Reduces cold start latency for first user interaction

5.2 Implement Cache Invalidation Strategy

• Define when cache should be invalidated: tier changes, context updates, system prompt changes
• Add cache version identifier to track when context structure changes
• Invalidate cache by changing version identifier in system prompt

5.3 Optimize Context Item Selection

• Prioritize HOT tier items for inclusion in cached prefix
• Move WARM/COLD items to dynamic suffix (after cache boundary)
• Balance: cache hit rate vs. context relevance

5.4 Implement Adaptive Caching

• Monitor cache hit rates per agent type
• Adjust cache boundaries dynamically based on hit rates
• If hit rate < 50%, move more content to dynamic section
• If hit rate > 90%, move more content to cached section

5.5 Document Best Practices

• Create guide for developers on maintaining cache-friendly context assembly
• Rules: always append new content, never reorder existing content, keep system prompt stable
• Add linting rules to detect cache-breaking patterns in code

Success Metrics

Metric	Target	Measurement Method
Cache Hit Rate	> 70%	Track via Anthropic API response headers
Cost Reduction	> 50%	Compare token costs before/after
Latency Improvement	> 30%	Measure API response time
Context Stability	100%	Audit: no reordering between calls
Implementation Coverage	100%	All agent types use caching

Files to Modify

File	Changes Required
file:codeframe/providers/anthropic.py	Add system prompt parameter, enable caching header, add cache_control markers, capture cache metrics
file:codeframe/agents/lead_agent.py	Pass system prompt to provider, structure messages for caching
file:codeframe/agents/worker_agent.py	Stabilize context item ordering (sort by tier, created_at, id)
file:codeframe/agents/backend_worker_agent.py	Stabilize related files/symbols ordering (alphabetical/line number)
file:codeframe/lib/metrics_tracker.py	Add cache metrics tracking (cache_hit_tokens, cache_miss_tokens, hit_rate)
file:codeframe/lib/context_manager.py	Document tier change impact on cache, add cache invalidation hooks
file:codeframe/ui/server.py	Add cache metrics API endpoint, dashboard integration
file:codeframe/persistence/database.py	Extend token_usage schema with cache metrics columns

Cost Impact Estimate

Current State (No Caching):

• Average task: 50 LLM calls
• Average static context: 10K tokens per call
• Total input tokens: 50 × 10K = 500K tokens
• Cost at Claude Sonnet rates ($3/MTok): $1.50 per task

With Caching (70% hit rate):

• First call: 10K tokens (full cost)
• Remaining 49 calls: 10K tokens cached (90% discount)
• Cached tokens: 49 × 10K = 490K tokens
• Cost: $0.30 (first call) + $0.147 (cached calls) = $0.447 per task
• Savings: 70% cost reduction ($1.05 per task)

At Scale (1000 tasks/month):

• Current cost: $1,500/month
• With caching: $447/month
• Monthly savings: $1,053

Import In IDE

🤖 Prompt for AI Agents

## Observations

The CodeFRAME system currently has:
1. **No prompt caching implementation** - The `file:codeframe/providers/anthropic.py` provider doesn't use Anthropic's prompt caching API features
2. **No system prompt in conversation assembly** - The `file:codeframe/agents/lead_agent.py` chat method only sends user/assistant messages without system prompts
3. **Tiered context system** (HOT/WARM/COLD) in `file:codeframe/lib/context_manager.py` but unclear ordering during assembly
4. **Multiple context assembly points** - Lead agent chat, worker agent code generation, and SDK client wrapper each handle context differently
5. **No cache_control markers** - Messages don't include `cache_control: {"type": "ephemeral"}` blocks required for Anthropic caching

## Approach

The plan focuses on implementing append-only context assembly with Anthropic's prompt caching to achieve up to 4x cost reduction. The strategy involves: (1) auditing current context assembly across all agent types to document exact ordering, (2) identifying cache-breaking patterns where dynamic content appears before static content, (3) refactoring to enforce static-first, dynamic-last ordering with proper cache_control markers, and (4) measuring cache hit rates through Anthropic API response headers. This aligns with Philipp Schmid's context engineering principle: static content first, dynamic content last, maximizing prefix stability for cache hits.

## Implementation Steps

### Phase 1: Audit Current Context Assembly Order

**1.1 Document Lead Agent Context Assembly**
- Map the exact sequence in `file:codeframe/agents/lead_agent.py` `chat()` method (lines 170-240)
- Current flow: conversation history → user message → API call
- **Issue**: No system prompt included in messages.create() call
- **Issue**: Conversation history may contain reordered messages from tier changes

**1.2 Document Worker Agent Context Assembly**
- Examine `file:codeframe/agents/backend_worker_agent.py` `generate_code()` method (lines 260-364)
- Current flow: system prompt (via `system=` parameter) → user prompt with task/files/symbols
- SDK path: system prompt in ClaudeAgentOptions → user message
- Direct API path: system prompt as separate parameter → user message
- **Good**: System prompt is separate and stable
- **Issue**: Related files/symbols order may vary between calls

**1.3 Document Context Item Loading**
- Review `file:codeframe/agents/worker_agent.py` `load_context()` method (lines 204-238)
- Loads context items filtered by tier (HOT/WARM/COLD)
- **Issue**: Tier changes via `update_tiers_for_agent()` can cause items to move between tiers, potentially reordering context

**1.4 Create Context Assembly Sequence Diagram**

```mermaid
sequenceDiagram
    participant User
    participant LeadAgent
    participant AnthropicProvider
    participant ClaudeAPI
    
    User->>LeadAgent: chat(message)
    LeadAgent->>LeadAgent: get_conversation_history()
    Note over LeadAgent: Loads user/assistant messages<br/>NO system prompt included
    LeadAgent->>LeadAgent: append user message
    LeadAgent->>AnthropicProvider: send_message(conversation)
    Note over AnthropicProvider: messages.create()<br/>NO system parameter<br/>NO cache_control markers
    AnthropicProvider->>ClaudeAPI: POST /messages
    ClaudeAPI-->>AnthropicProvider: response
    AnthropicProvider-->>LeadAgent: response
    LeadAgent->>LeadAgent: save to database
    LeadAgent-->>User: assistant response
```

### Phase 2: Identify Cache-Breaking Patterns

**2.1 Analyze Conversation History Ordering**
- In `file:codeframe/agents/lead_agent.py`, conversation history is loaded from database via `get_conversation()`
- Messages are stored with timestamps but loaded in insertion order
- **Cache-breaking pattern**: If conversation history is modified or reordered, cache invalidates

**2.2 Analyze Context Item Ordering**
- In `file:codeframe/lib/context_manager.py`, `update_tiers_for_agent()` (lines 92-146) recalculates scores and reassigns tiers
- Items can move from HOT→WARM→COLD based on age/access patterns
- **Cache-breaking pattern**: If context items are loaded by tier and tier assignments change, the order changes

**2.3 Analyze Tool Definition Positioning**
- SDK client in `file:codeframe/providers/sdk_client.py` passes tools via ClaudeAgentOptions
- Tools are static per agent type but position in prompt is SDK-controlled
- **Potential issue**: SDK may inject tool definitions at varying positions

**2.4 Analyze Related Files/Symbols Ordering**
- In `file:codeframe/agents/backend_worker_agent.py` `generate_code()`, related files/symbols are added to prompt
- Order depends on codebase index query results
- **Cache-breaking pattern**: If index returns results in different order, prompt prefix changes

**2.5 Create Cache-Breaking Pattern Table**

| Component | Current Behavior | Cache Impact | Severity |
|-----------|-----------------|--------------|----------|
| System Prompt | Not included in Lead Agent | No caching possible | **Critical** |
| Conversation History | Loaded in insertion order | Stable if append-only | Low |
| Context Items (Tiers) | Tier changes cause reordering | Cache invalidation | **High** |
| Related Files/Symbols | Order varies by index query | Cache invalidation | **Medium** |
| Tool Definitions | SDK-controlled position | Unknown | Low |
| User Message | Always at end | Stable (expected) | None |

### Phase 3: Implement Append-Only Context Assembly

**3.1 Add System Prompt to AnthropicProvider**
- Modify `file:codeframe/providers/anthropic.py` `send_message()` method
- Add `system` parameter to `messages.create()` call
- Accept system prompt as optional parameter in method signature
- Ensure system prompt is passed from Lead Agent

**3.2 Enable Prompt Caching in AnthropicProvider**
- Add `anthropic-beta: prompt-caching-2024-07-31` header to API client initialization
- Modify message structure to include `cache_control` markers
- Add cache_control to system prompt: `{"type": "text", "text": system_prompt, "cache_control": {"type": "ephemeral"}}`
- Add cache_control to stable context sections (e.g., last message in conversation history before current turn)

**3.3 Refactor Lead Agent Context Assembly**
- Modify `file:codeframe/agents/lead_agent.py` `chat()` method
- Ensure system prompt is loaded and passed to provider
- Structure messages as: system prompt (cached) → conversation history (cached) → current user message (not cached)
- Add cache_control marker after conversation history to maximize cache hit

**3.4 Stabilize Context Item Ordering**
- Modify `file:codeframe/agents/worker_agent.py` `load_context()` method
- Load context items in deterministic order: sort by (tier, created_at, id)
- Ensure tier changes don't affect relative ordering within tiers
- Consider loading all tiers together and filtering in-memory to maintain stable order

**3.5 Stabilize Related Files/Symbols Ordering**
- Modify `file:codeframe/agents/backend_worker_agent.py` `build_context()` method
- Sort related files alphabetically before adding to prompt
- Sort related symbols by (file_path, line_number) for deterministic order
- Document that codebase index should return stable ordering

**3.6 Create Context Assembly Configuration**
- Add configuration class for cache control settings
- Define which sections should be cached (system prompt, conversation history, context items)
- Allow per-agent-type cache configuration
- Default: cache everything except current user message

**3.7 Implement Context Assembly Order Enforcement**

```mermaid
flowchart TD
    A[Start Context Assembly] --> B[Add System Prompt]
    B --> C[Mark System Prompt as Cacheable]
    C --> D[Add Tool Definitions]
    D --> E[Mark Tools as Cacheable]
    E --> F[Add Context Items - Sorted]
    F --> G[Mark Context Items as Cacheable]
    G --> H[Add Conversation History - Sorted]
    H --> I[Mark Last History Message as Cacheable]
    I --> J[Add Current User Message]
    J --> K[NO Cache Control on User Message]
    K --> L[Send to API]
    L --> M{Cache Hit?}
    M -->|Yes| N[4x Cost Reduction]
    M -->|No| O[Full Processing]
    N --> P[Return Response]
    O --> P
```

### Phase 4: Measure Cache Hit Rates

**4.1 Add Cache Metrics Collection**
- Modify `file:codeframe/providers/anthropic.py` to capture cache metrics from API response
- Anthropic API returns cache hit info in response headers: `anthropic-ratelimit-input-tokens-cache-hit`
- Store metrics: input_tokens, output_tokens, cache_creation_tokens, cache_read_tokens

**4.2 Extend MetricsTracker**
- Modify `file:codeframe/lib/metrics_tracker.py` to track cache metrics
- Add fields to token usage records: cache_hit_tokens, cache_miss_tokens
- Calculate cache hit rate: cache_hit_tokens / (cache_hit_tokens + cache_miss_tokens)

**4.3 Add Cache Metrics to Database**
- Extend token_usage table schema to include cache metrics
- Add columns: cache_creation_tokens, cache_read_tokens, cache_hit_rate
- Create migration for schema update

**4.4 Create Cache Metrics Dashboard**
- Add API endpoint in `file:codeframe/ui/server.py` to retrieve cache metrics
- Calculate aggregate metrics: total cache hit rate, cost savings, tokens saved
- Display metrics per project, per agent type, per time period

**4.5 Implement Before/After Comparison**
- Add feature flag to enable/disable prompt caching
- Run A/B test: 50% of requests with caching, 50% without
- Compare: cost per request, latency, cache hit rate
- Document findings in metrics dashboard

### Phase 5: Optimize for Maximum Cache Hits

**5.1 Implement Cache Warmup Strategy**
- On agent initialization, send a warmup request with full context
- Ensures cache is populated before real requests
- Reduces cold start latency for first user interaction

**5.2 Implement Cache Invalidation Strategy**
- Define when cache should be invalidated: tier changes, context updates, system prompt changes
- Add cache version identifier to track when context structure changes
- Invalidate cache by changing version identifier in system prompt

**5.3 Optimize Context Item Selection**
- Prioritize HOT tier items for inclusion in cached prefix
- Move WARM/COLD items to dynamic suffix (after cache boundary)
- Balance: cache hit rate vs. context relevance

**5.4 Implement Adaptive Caching**
- Monitor cache hit rates per agent type
- Adjust cache boundaries dynamically based on hit rates
- If hit rate < 50%, move more content to dynamic section
- If hit rate > 90%, move more content to cached section

**5.5 Document Best Practices**
- Create guide for developers on maintaining cache-friendly context assembly
- Rules: always append new content, never reorder existing content, keep system prompt stable
- Add linting rules to detect cache-breaking patterns in code

## Success Metrics

| Metric | Target | Measurement Method |
|--------|--------|-------------------|
| Cache Hit Rate | > 70% | Track via Anthropic API response headers |
| Cost Reduction | > 50% | Compare token costs before/after |
| Latency Improvement | > 30% | Measure API response time |
| Context Stability | 100% | Audit: no reordering between calls |
| Implementation Coverage | 100% | All agent types use caching |

## Files to Modify

| File | Changes Required |
|------|-----------------|
| `file:codeframe/providers/anthropic.py` | Add system prompt parameter, enable caching header, add cache_control markers, capture cache metrics |
| `file:codeframe/agents/lead_agent.py` | Pass system prompt to provider, structure messages for caching |
| `file:codeframe/agents/worker_agent.py` | Stabilize context item ordering (sort by tier, created_at, id) |
| `file:codeframe/agents/backend_worker_agent.py` | Stabilize related files/symbols ordering (alphabetical/line number) |
| `file:codeframe/lib/metrics_tracker.py` | Add cache metrics tracking (cache_hit_tokens, cache_miss_tokens, hit_rate) |
| `file:codeframe/lib/context_manager.py` | Document tier change impact on cache, add cache invalidation hooks |
| `file:codeframe/ui/server.py` | Add cache metrics API endpoint, dashboard integration |
| `file:codeframe/persistence/database.py` | Extend token_usage schema with cache metrics columns |

## Cost Impact Estimate

**Current State (No Caching):**
- Average task: 50 LLM calls
- Average static context: 10K tokens per call
- Total input tokens: 50 × 10K = 500K tokens
- Cost at Claude Sonnet rates ($3/MTok): $1.50 per task

**With Caching (70% hit rate):**
- First call: 10K tokens (full cost)
- Remaining 49 calls: 10K tokens cached (90% discount)
- Cached tokens: 49 × 10K = 490K tokens
- Cost: $0.30 (first call) + $0.147 (cached calls) = $0.447 per task
- **Savings: 70% cost reduction ($1.05 per task)**

**At Scale (1000 tasks/month):**
- Current cost: $1,500/month
- With caching: $447/month
- **Monthly savings: $1,053**

---

Execution Information

Branch: main
Commit: 08bbb8e

New Plan

Observations

The CodeFRAME system currently has:

1. No prompt caching implementation - The file:codeframe/providers/anthropic.py provider doesn't use Anthropic's prompt caching API features
2. No system prompt in conversation assembly - The file:codeframe/agents/lead_agent.py chat method only sends user/assistant messages without system prompts
3. Tiered context system (HOT/WARM/COLD) in file:codeframe/lib/context_manager.py but unclear ordering during assembly
4. Multiple context assembly points - Lead agent chat, worker agent code generation, and SDK client wrapper each handle context differently
5. No cache_control markers - Messages don't include cache_control: {"type": "ephemeral"} blocks required for Anthropic caching

Approach

The plan focuses on auditing and documenting the current context assembly order to establish a baseline for future optimization. The strategy involves: (1) mapping the exact sequence of context assembly across all agent types, (2) identifying cache-breaking patterns where dynamic content appears before static content, (3) creating detailed documentation and metrics to quantify the impact of improper ordering, and (4) establishing a foundation for future implementation of append-only context assembly with Anthropic's prompt caching. This aligns with Philipp Schmid's context engineering principle: static content first, dynamic content last, maximizing prefix stability for cache hits.

Implementation Steps

Phase 1: Audit Current Context Assembly Order

1.1 Document Lead Agent Context Assembly

• Map the exact sequence in file:codeframe/agents/lead_agent.py chat() method (lines 170-240)
• Current flow: conversation history → user message → API call
• Issue: No system prompt included in messages.create() call
• Issue: Conversation history may contain reordered messages from tier changes

1.2 Document Worker Agent Context Assembly

• Examine file:codeframe/agents/backend_worker_agent.py generate_code() method (lines 260-364)
• Current flow: system prompt (via system= parameter) → user prompt with task/files/symbols
• SDK path: system prompt in ClaudeAgentOptions → user message
• Direct API path: system prompt as separate parameter → user message
• Good: System prompt is separate and stable
• Issue: Related files/symbols order may vary between calls

1.3 Document Context Item Loading

• Review file:codeframe/agents/worker_agent.py load_context() method (lines 204-238)
• Loads context items filtered by tier (HOT/WARM/COLD)
• Issue: Tier changes via update_tiers_for_agent() can cause items to move between tiers, potentially reordering context

1.4 Create Context Assembly Sequence Diagram

sequenceDiagram
    participant User
    participant LeadAgent
    participant AnthropicProvider
    participant ClaudeAPI
    
    User->>LeadAgent: chat(message)
    LeadAgent->>LeadAgent: get_conversation_history()
    Note over LeadAgent: Loads user/assistant messages<br/>NO system prompt included
    LeadAgent->>LeadAgent: append user message
    LeadAgent->>AnthropicProvider: send_message(conversation)
    Note over AnthropicProvider: messages.create()<br/>NO system parameter<br/>NO cache_control markers
    AnthropicProvider->>ClaudeAPI: POST /messages
    ClaudeAPI-->>AnthropicProvider: response
    AnthropicProvider-->>LeadAgent: response
    LeadAgent->>LeadAgent: save to database
    LeadAgent-->>User: assistant response

Loading

Phase 2: Identify Cache-Breaking Patterns

2.1 Analyze Conversation History Ordering

• In file:codeframe/agents/lead_agent.py, conversation history is loaded from database via get_conversation()
• Messages are stored with timestamps but loaded in insertion order
• Cache-breaking pattern: If conversation history is modified or reordered, cache invalidates

2.2 Analyze Context Item Ordering

• In file:codeframe/lib/context_manager.py, update_tiers_for_agent() (lines 92-146) recalculates scores and reassigns tiers
• Items can move from HOT→WARM→COLD based on age/access patterns
• Cache-breaking pattern: If context items are loaded by tier and tier assignments change, the order changes

2.3 Analyze Tool Definition Positioning

• SDK client in file:codeframe/providers/sdk_client.py passes tools via ClaudeAgentOptions
• Tools are static per agent type but position in prompt is SDK-controlled
• Potential issue: SDK may inject tool definitions at varying positions

2.4 Analyze Related Files/Symbols Ordering

• In file:codeframe/agents/backend_worker_agent.py generate_code(), related files/symbols are added to prompt
• Order depends on codebase index query results
• Cache-breaking pattern: If index returns results in different order, prompt prefix changes

2.5 Create Cache-Breaking Pattern Table

Component	Current Behavior	Cache Impact	Severity
System Prompt	Not included in Lead Agent	No caching possible	Critical
Conversation History	Loaded in insertion order	Stable if append-only	Low
Context Items (Tiers)	Tier changes cause reordering	Cache invalidation	High
Related Files/Symbols	Order varies by index query	Cache invalidation	Medium
Tool Definitions	SDK-controlled position	Unknown	Low
User Message	Always at end	Stable (expected)	None

Phase 3: Measure Current Baseline (Without Caching)

3.1 Add Baseline Metrics Collection

• Modify file:codeframe/lib/metrics_tracker.py to capture baseline metrics
• Track: input tokens per call, output tokens, total tokens per task
• Record: number of calls per task, average context size
• Establish: baseline cost per task without caching

3.2 Extend Database Schema for Audit Data

• Add columns to token_usage table: context_components, component_order, tier_state
• Store: serialized context assembly metadata for analysis
• Create migration for schema update
• Enable detailed post-hoc analysis of context assembly patterns

3.3 Run Baseline Measurement Campaign

• Execute 100+ representative tasks with logging enabled
• Capture: context assembly logs, token usage, timing data
• Analyze: cache-breaking pattern frequency, context stability metrics
• Document: baseline metrics for future comparison

3.4 Create Baseline Metrics Report

• Calculate: average input tokens per call, per task
• Calculate: percentage of context that is static vs. dynamic
• Estimate: potential savings with 50%, 70%, 90% cache hit rates
• Project: cost reduction at different cache hit rate targets

Phase 4: Document Findings and Create Implementation Roadmap

4.1 Create Comprehensive Audit Report

• Consolidate findings from Phases 1-3
• Include: context assembly sequence, cache-breaking patterns, baseline metrics
• Add: visual diagrams, code references, detailed analysis
• Create: CONTEXT_ASSEMBLY_OPTIMIZATION_ROADMAP.md

4.2 Identify Quick Wins

• Document: changes that can be made with minimal effort
• Examples: stabilizing related files/symbols order, sorting context items
• Estimate: effort and impact for each quick win
• Prioritize: by impact/effort ratio

4.3 Design Future Implementation Strategy

• Outline: append-only context assembly architecture
• Design: cache_control marker placement strategy
• Plan: phased rollout (Lead Agent → Worker Agent → SDK Client)
• Estimate: effort and timeline for full implementation

4.4 Create Implementation Checklist for Future Sprint

• List: all changes required for append-only assembly
• Include: system prompt addition, cache_control markers, ordering stabilization
• Add: testing strategy, measurement approach, rollback plan
• Estimate: story points and sprint allocation

4.5 Document Decision Points

• Identify: key decisions needed before implementation
• Examples: cache boundary placement, tier change invalidation strategy
• Document: trade-offs for each decision
• Create: decision matrix for future team

Success Metrics

Metric	Target	Measurement Method
Audit Completion	100%	All phases documented
Cache-Breaking Patterns Identified	100%	All patterns documented with frequency
Baseline Metrics Established	100%	Metrics collected from 100+ tasks
Implementation Roadmap Created	100%	Detailed plan for future sprint
Quick Wins Identified	3+	Changes with high impact/effort ratio
Cost Reduction Potential Quantified	Yes	Estimated savings at different cache hit rates

Deliverables

1. CONTEXT_ASSEMBLY_AUDIT.md - Detailed audit of current context assembly order
2. CACHE_BREAKING_PATTERNS_ANALYSIS.md - Identified patterns with frequency and impact
3. BASELINE_METRICS_REPORT.md - Current token usage and cost baseline
4. CONTEXT_ASSEMBLY_OPTIMIZATION_ROADMAP.md - Implementation strategy for future sprint
5. Context Assembly Logging - Debug logging in anthropic.py provider
6. Baseline Metrics Database - Extended schema with audit data
7. Implementation Checklist - Detailed tasks for future implementation

Files to Modify (Audit Phase Only)

File	Changes Required
file:codeframe/providers/anthropic.py	Add debug logging for context assembly
file:codeframe/lib/metrics_tracker.py	Add baseline metrics collection
file:codeframe/persistence/database.py	Extend token_usage schema with audit data
Documentation	Create audit reports and roadmap documents

Cost Impact Estimate (Future Potential)

Current State (No Caching):

• Average task: 50 LLM calls
• Average static context: 10K tokens per call
• Total input tokens: 50 × 10K = 500K tokens
• Cost at Claude Sonnet rates ($3/MTok): $1.50 per task

With Caching (70% hit rate - Future Target):

• First call: 10K tokens (full cost)
• Remaining 49 calls: 10K tokens cached (90% discount)
• Cached tokens: 49 × 10K = 490K tokens
• Cost: $0.30 (first call) + $0.147 (cached calls) = $0.447 per task
• Potential savings: 70% cost reduction ($1.05 per task)

At Scale (1000 tasks/month - Future Potential):

• Current cost: $1,500/month
• With caching: $447/month
• Potential monthly savings: $1,053

Timeline

• Phase 1 (Audit): 2-3 days - Document current context assembly
• Phase 2 (Analysis): 2-3 days - Identify cache-breaking patterns
• Phase 3 (Measurement): 3-4 days - Collect baseline metrics
• Phase 4 (Documentation): 2-3 days - Create roadmap and reports
• Total: 9-13 days for complete audit and roadmap creation
• Future Implementation: Estimated 5-7 days (separate sprint)

Import In IDE

🤖 Prompt for AI Agents

## Observations

The CodeFRAME system currently has:
1. **No prompt caching implementation** - The `file:codeframe/providers/anthropic.py` provider doesn't use Anthropic's prompt caching API features
2. **No system prompt in conversation assembly** - The `file:codeframe/agents/lead_agent.py` chat method only sends user/assistant messages without system prompts
3. **Tiered context system** (HOT/WARM/COLD) in `file:codeframe/lib/context_manager.py` but unclear ordering during assembly
4. **Multiple context assembly points** - Lead agent chat, worker agent code generation, and SDK client wrapper each handle context differently
5. **No cache_control markers** - Messages don't include `cache_control: {"type": "ephemeral"}` blocks required for Anthropic caching

## Approach

The plan focuses on auditing and documenting the current context assembly order to establish a baseline for future optimization. The strategy involves: (1) mapping the exact sequence of context assembly across all agent types, (2) identifying cache-breaking patterns where dynamic content appears before static content, (3) creating detailed documentation and metrics to quantify the impact of improper ordering, and (4) establishing a foundation for future implementation of append-only context assembly with Anthropic's prompt caching. This aligns with Philipp Schmid's context engineering principle: static content first, dynamic content last, maximizing prefix stability for cache hits.

## Implementation Steps

### Phase 1: Audit Current Context Assembly Order

**1.1 Document Lead Agent Context Assembly**
- Map the exact sequence in `file:codeframe/agents/lead_agent.py` `chat()` method (lines 170-240)
- Current flow: conversation history → user message → API call
- **Issue**: No system prompt included in messages.create() call
- **Issue**: Conversation history may contain reordered messages from tier changes

**1.2 Document Worker Agent Context Assembly**
- Examine `file:codeframe/agents/backend_worker_agent.py` `generate_code()` method (lines 260-364)
- Current flow: system prompt (via `system=` parameter) → user prompt with task/files/symbols
- SDK path: system prompt in ClaudeAgentOptions → user message
- Direct API path: system prompt as separate parameter → user message
- **Good**: System prompt is separate and stable
- **Issue**: Related files/symbols order may vary between calls

**1.3 Document Context Item Loading**
- Review `file:codeframe/agents/worker_agent.py` `load_context()` method (lines 204-238)
- Loads context items filtered by tier (HOT/WARM/COLD)
- **Issue**: Tier changes via `update_tiers_for_agent()` can cause items to move between tiers, potentially reordering context

**1.4 Create Context Assembly Sequence Diagram**

```mermaid
sequenceDiagram
    participant User
    participant LeadAgent
    participant AnthropicProvider
    participant ClaudeAPI
    
    User->>LeadAgent: chat(message)
    LeadAgent->>LeadAgent: get_conversation_history()
    Note over LeadAgent: Loads user/assistant messages<br/>NO system prompt included
    LeadAgent->>LeadAgent: append user message
    LeadAgent->>AnthropicProvider: send_message(conversation)
    Note over AnthropicProvider: messages.create()<br/>NO system parameter<br/>NO cache_control markers
    AnthropicProvider->>ClaudeAPI: POST /messages
    ClaudeAPI-->>AnthropicProvider: response
    AnthropicProvider-->>LeadAgent: response
    LeadAgent->>LeadAgent: save to database
    LeadAgent-->>User: assistant response
```

### Phase 2: Identify Cache-Breaking Patterns

**2.1 Analyze Conversation History Ordering**
- In `file:codeframe/agents/lead_agent.py`, conversation history is loaded from database via `get_conversation()`
- Messages are stored with timestamps but loaded in insertion order
- **Cache-breaking pattern**: If conversation history is modified or reordered, cache invalidates

**2.2 Analyze Context Item Ordering**
- In `file:codeframe/lib/context_manager.py`, `update_tiers_for_agent()` (lines 92-146) recalculates scores and reassigns tiers
- Items can move from HOT→WARM→COLD based on age/access patterns
- **Cache-breaking pattern**: If context items are loaded by tier and tier assignments change, the order changes

**2.3 Analyze Tool Definition Positioning**
- SDK client in `file:codeframe/providers/sdk_client.py` passes tools via ClaudeAgentOptions
- Tools are static per agent type but position in prompt is SDK-controlled
- **Potential issue**: SDK may inject tool definitions at varying positions

**2.4 Analyze Related Files/Symbols Ordering**
- In `file:codeframe/agents/backend_worker_agent.py` `generate_code()`, related files/symbols are added to prompt
- Order depends on codebase index query results
- **Cache-breaking pattern**: If index returns results in different order, prompt prefix changes

**2.5 Create Cache-Breaking Pattern Table**

| Component | Current Behavior | Cache Impact | Severity |
|-----------|-----------------|--------------|----------|
| System Prompt | Not included in Lead Agent | No caching possible | **Critical** |
| Conversation History | Loaded in insertion order | Stable if append-only | Low |
| Context Items (Tiers) | Tier changes cause reordering | Cache invalidation | **High** |
| Related Files/Symbols | Order varies by index query | Cache invalidation | **Medium** |
| Tool Definitions | SDK-controlled position | Unknown | Low |
| User Message | Always at end | Stable (expected) | None |

### Phase 3: Measure Current Baseline (Without Caching)

**3.1 Add Baseline Metrics Collection**
- Modify `file:codeframe/lib/metrics_tracker.py` to capture baseline metrics
- Track: input tokens per call, output tokens, total tokens per task
- Record: number of calls per task, average context size
- Establish: baseline cost per task without caching

**3.2 Extend Database Schema for Audit Data**
- Add columns to token_usage table: context_components, component_order, tier_state
- Store: serialized context assembly metadata for analysis
- Create migration for schema update
- Enable detailed post-hoc analysis of context assembly patterns

**3.3 Run Baseline Measurement Campaign**
- Execute 100+ representative tasks with logging enabled
- Capture: context assembly logs, token usage, timing data
- Analyze: cache-breaking pattern frequency, context stability metrics
- Document: baseline metrics for future comparison

**3.4 Create Baseline Metrics Report**
- Calculate: average input tokens per call, per task
- Calculate: percentage of context that is static vs. dynamic
- Estimate: potential savings with 50%, 70%, 90% cache hit rates
- Project: cost reduction at different cache hit rate targets

### Phase 4: Document Findings and Create Implementation Roadmap

**4.1 Create Comprehensive Audit Report**
- Consolidate findings from Phases 1-3
- Include: context assembly sequence, cache-breaking patterns, baseline metrics
- Add: visual diagrams, code references, detailed analysis
- Create: `CONTEXT_ASSEMBLY_OPTIMIZATION_ROADMAP.md`

**4.2 Identify Quick Wins**
- Document: changes that can be made with minimal effort
- Examples: stabilizing related files/symbols order, sorting context items
- Estimate: effort and impact for each quick win
- Prioritize: by impact/effort ratio

**4.3 Design Future Implementation Strategy**
- Outline: append-only context assembly architecture
- Design: cache_control marker placement strategy
- Plan: phased rollout (Lead Agent → Worker Agent → SDK Client)
- Estimate: effort and timeline for full implementation

**4.4 Create Implementation Checklist for Future Sprint**
- List: all changes required for append-only assembly
- Include: system prompt addition, cache_control markers, ordering stabilization
- Add: testing strategy, measurement approach, rollback plan
- Estimate: story points and sprint allocation

**4.5 Document Decision Points**
- Identify: key decisions needed before implementation
- Examples: cache boundary placement, tier change invalidation strategy
- Document: trade-offs for each decision
- Create: decision matrix for future team

## Success Metrics

| Metric | Target | Measurement Method |
|--------|--------|-------------------|
| Audit Completion | 100% | All phases documented |
| Cache-Breaking Patterns Identified | 100% | All patterns documented with frequency |
| Baseline Metrics Established | 100% | Metrics collected from 100+ tasks |
| Implementation Roadmap Created | 100% | Detailed plan for future sprint |
| Quick Wins Identified | 3+ | Changes with high impact/effort ratio |
| Cost Reduction Potential Quantified | Yes | Estimated savings at different cache hit rates |

## Deliverables

1. **CONTEXT_ASSEMBLY_AUDIT.md** - Detailed audit of current context assembly order
2. **CACHE_BREAKING_PATTERNS_ANALYSIS.md** - Identified patterns with frequency and impact
3. **BASELINE_METRICS_REPORT.md** - Current token usage and cost baseline
4. **CONTEXT_ASSEMBLY_OPTIMIZATION_ROADMAP.md** - Implementation strategy for future sprint
5. **Context Assembly Logging** - Debug logging in anthropic.py provider
6. **Baseline Metrics Database** - Extended schema with audit data
7. **Implementation Checklist** - Detailed tasks for future implementation

## Files to Modify (Audit Phase Only)

| File | Changes Required |
|------|-----------------|
| `file:codeframe/providers/anthropic.py` | Add debug logging for context assembly |
| `file:codeframe/lib/metrics_tracker.py` | Add baseline metrics collection |
| `file:codeframe/persistence/database.py` | Extend token_usage schema with audit data |
| Documentation | Create audit reports and roadmap documents |

## Cost Impact Estimate (Future Potential)

**Current State (No Caching):**
- Average task: 50 LLM calls
- Average static context: 10K tokens per call
- Total input tokens: 50 × 10K = 500K tokens
- Cost at Claude Sonnet rates ($3/MTok): $1.50 per task

**With Caching (70% hit rate - Future Target):**
- First call: 10K tokens (full cost)
- Remaining 49 calls: 10K tokens cached (90% discount)
- Cached tokens: 49 × 10K = 490K tokens
- Cost: $0.30 (first call) + $0.147 (cached calls) = $0.447 per task
- **Potential savings: 70% cost reduction ($1.05 per task)**

**At Scale (1000 tasks/month - Future Potential):**
- Current cost: $1,500/month
- With caching: $447/month
- **Potential monthly savings: $1,053**

## Timeline

- **Phase 1 (Audit)**: 2-3 days - Document current context assembly
- **Phase 2 (Analysis)**: 2-3 days - Identify cache-breaking patterns
- **Phase 3 (Measurement)**: 3-4 days - Collect baseline metrics
- **Phase 4 (Documentation)**: 2-3 days - Create roadmap and reports
- **Total**: 9-13 days for complete audit and roadmap creation
- **Future Implementation**: Estimated 5-7 days (separate sprint)

---

Execution Information

Branch: main
Commit: 1260477