-
Notifications
You must be signed in to change notification settings - Fork 5.1k
docs: add Entroly MCP cost optimization cookbook #12559
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
juyterman1000
wants to merge
1
commit into
continuedev:main
Choose a base branch
from
juyterman1000:add-entroly-mcp-cookbook
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| --- | ||
| title: "Reduce LLM API Costs with Entroly MCP and Continue" | ||
| description: "Use Continue with the Entroly MCP server to compress context by 70-95%, align provider caches, and cut LLM API bills without losing answer quality." | ||
| sidebarTitle: "Cost Reduction with Entroly" | ||
| --- | ||
|
|
||
| <Card title="What You'll Build" icon="compress"> | ||
| A cost-optimized AI coding workflow that uses Continue with the Entroly MCP server to: | ||
| - Compress context by 70-95% before sending to the LLM | ||
| - Align prefixes for provider cache discounts (Anthropic 90%, OpenAI 50%) | ||
| - Verify model outputs against supplied evidence with WITNESS | ||
| - Monitor real-time savings via a local dashboard | ||
| </Card> | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Before starting, ensure you have: | ||
|
|
||
| - Continue installed in your IDE (VS Code, JetBrains, etc.) | ||
| - Python 3.10+ installed | ||
| - An LLM API key (Anthropic, OpenAI, etc.) | ||
|
|
||
| ## Setup | ||
|
|
||
| <Steps> | ||
| <Step title="Install Entroly"> | ||
| ```bash | ||
| pip install entroly | ||
| ``` | ||
| </Step> | ||
|
|
||
| <Step title="Add Entroly MCP Server to Continue"> | ||
| Add the following to your Continue config: | ||
|
|
||
| ```yaml title="config.yaml" | ||
| mcpServers: | ||
| - name: Entroly | ||
| command: uvx | ||
| args: | ||
| - "--from" | ||
| - "entroly" | ||
| - "entroly" | ||
| - "serve" | ||
| ``` | ||
| </Step> | ||
|
|
||
| <Step title="Verify the Connection"> | ||
| Launch Continue and type `@` — you should see Entroly in the context providers list. | ||
|
|
||
| Try asking: | ||
| ``` | ||
| Use Entroly to analyze this repository and show me the token savings estimate. | ||
| ``` | ||
| </Step> | ||
| </Steps> | ||
|
|
||
| ## Alternative: Proxy Mode | ||
|
|
||
| For transparent cost reduction without changing your Continue config, run Entroly as a local proxy: | ||
|
|
||
| ```bash | ||
| entroly proxy | ||
| ``` | ||
|
|
||
| Then update your provider's base URL in Continue to point at the proxy: | ||
|
|
||
| ```yaml title="config.yaml" | ||
| models: | ||
| - name: claude-sonnet | ||
| provider: anthropic | ||
| apiBase: http://localhost:9377 | ||
| ``` | ||
|
|
||
| All requests are compressed transparently — no other config changes needed. | ||
|
|
||
| ## How It Works | ||
|
|
||
| ``` | ||
| Continue IDE ──► Entroly (local) ──► LLM Provider | ||
| │ | ||
| ├─ Rank repo files (BM25 + entropy + dep-graph) | ||
| ├─ Select under budget (knapsack, reversible) | ||
| ├─ Cache-align prefix (keep provider cache hot) | ||
| └─ Verify the reply (WITNESS hallucination guard) | ||
| ``` | ||
|
|
||
| ## Monitor Savings | ||
|
|
||
| Open the Entroly dashboard to see real-time cost metrics: | ||
|
|
||
| ```bash | ||
| entroly dashboard | ||
| # Opens http://localhost:9378 | ||
| ``` | ||
|
|
||
| The dashboard shows: | ||
| - Tokens saved per request | ||
| - Cumulative dollar savings | ||
| - Cache hit rates | ||
| - Per-lever cost breakdown | ||
|
|
||
| ## Key Benefits for Continue Users | ||
|
|
||
| | Feature | Benefit | | ||
| |---|---| | ||
| | **Context compression** | 70-95% fewer input tokens on large repos | | ||
| | **Cache alignment** | Captures provider prefix-cache discounts | | ||
| | **Reversible compression** | Original context recoverable via CCR handles | | ||
| | **WITNESS guard** | $0 hallucination check on every response | | ||
| | **Local-first** | No code sent for analysis — runs on-device | | ||
|
|
||
| ## Resources | ||
|
|
||
| <CardGroup cols={2}> | ||
| <Card title="Entroly Repository" icon="github" href="https://github.com/juyterman1000/entroly"> | ||
| Full documentation, benchmarks, and source code | ||
| </Card> | ||
| <Card title="MCP in Continue" icon="book" href="/customize/deep-dives/mcp"> | ||
| How MCP works with Continue agents | ||
| </Card> | ||
| </CardGroup> | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: Missing
uv/uvxprerequisite despite usinguvxin MCP server commandPrompt for AI agents