CAPO
CAPO, short for Cost-Aware Prompt Optimization, treats the cost of optimization as a first-class concern. It evolves prompts with an LLM, but adds two ideas from automated machine learning: racing, which stops testing weak candidates early to save evaluations, and a length penalty, which keeps winning prompts short. It also tunes the instruction and the few-shot examples together, producing prompts that are both accurate and economical.
*These are learning tools, not prompts. They teach you to write your own, think of them as training wheels that guide you while you learn, then fall away once you can ride on your own.
Optimize for Accuracy and Cost Together
Prompt optimizers can be expensive to run, because each candidate prompt has to be evaluated on many examples, and they can also produce winners that are long and costly to use in production. Most methods focus only on accuracy and ignore both of these costs. CAPO puts cost at the center of the search.
It borrows two well-known ideas from automated machine learning. The first is racing: instead of evaluating every candidate on the full set, CAPO tests candidates on a few examples first and drops the clearly weak ones early, spending its evaluation budget only on the promising prompts. The second is a multi-objective view that penalizes length, so among prompts of similar accuracy it prefers the shorter, cheaper one.
On top of this, CAPO evolves prompts with LLM-driven mutation and crossover, and it optimizes the instruction and the few-shot examples jointly rather than one at a time. Presented at a dedicated automated-machine-learning venue, it reaches strong accuracy while using a smaller evaluation budget and yielding shorter prompts than optimizers that chase accuracy alone.
Racing cuts the cost of the search itself by discarding weak candidates before they consume a full evaluation. The length penalty cuts the cost of the resulting prompt by favoring shorter winners. Together they make both the optimization and the deployed prompt cheaper, without giving up accuracy.
The CAPO Process
Evolve candidates, race them, and select for accuracy and brevity
Evolve a Population of Prompts
Starting from an initial prompt and the task description, CAPO uses LLM-driven mutation and crossover to generate a population of candidate prompts, varying both the instruction and the few-shot examples together.
Candidates differ in wording, in which examples they include, and in how many, all generated as variations to test.
Race to Prune Weak Candidates
Rather than fully evaluating every candidate, CAPO tests them on a small batch first and eliminates the clearly weak ones. Only survivors are evaluated further, so the evaluation budget is concentrated where it matters.
A candidate that fails most of the first ten examples is dropped before it is ever run on the remaining hundreds.
Score With a Length Penalty
Surviving candidates are judged on accuracy and on length together. Among prompts that perform similarly, the shorter one scores higher, steering the search toward prompts that will be cheaper to run in production.
Two prompts hit the same accuracy; the one using three examples instead of eight is preferred for its lower token cost.
Iterate to the Best Trade-Off
The best candidates seed the next generation, and the cycle repeats until the budget is spent. The output is a prompt that balances high accuracy with low length, having spent fewer evaluations to find it.
The final prompt is both accurate and compact, and the search reached it with fewer full evaluations than a non-racing method.
See the Difference
Accuracy-only search versus a cost-aware one
Accuracy-Only Optimizer
Every candidate is fully evaluated and judged only on accuracy. Long prompts that happen to score well are kept, and weak candidates still consume full evaluations.
The search is expensive, and the winning prompt can be long, so it is also expensive to run afterward.
CAPO
Racing drops weak candidates early, and a length penalty favors short winners, all while evolving instructions and examples together.
Strong accuracy reached with a smaller evaluation budget, and the resulting prompt is shorter and cheaper to deploy.
Practice Responsible AI
Always verify AI-generated content before use. AI systems can produce confident but incorrect responses. When using AI professionally, transparent disclosure is both best practice and increasingly a legal requirement.
Most US states are actively legislating AI transparency and accountability. Critical thinking remains your strongest tool against misinformation.
CAPO in Action
See how racing and the length penalty shape the search
Twelve candidate prompts are generated for a classification task, each with different wording and example sets.
First pass: all twelve are tested on a small batch. Five perform clearly worse and are dropped immediately.
Budget saved: those five never run on the full set, so the evaluation budget is spent on the seven survivors instead.
Effect: the same quality of winner is found with far fewer total evaluations.
Two surviving prompts reach the same accuracy: one uses eight few-shot examples, the other uses three.
Scoring: the length penalty gives the three-example prompt a higher combined score despite the accuracy tie.
Result: the shorter prompt is selected, so every production call afterward costs fewer tokens with no loss in accuracy.
The task is extraction from messy text, where the instruction wording and the choice of examples both matter.
Because it evolves both at once, CAPO can discover that a clearer instruction lets it drop two redundant examples, shortening the prompt while holding accuracy.
Result: a compact prompt whose instruction and examples were tuned to fit each other, not optimized in isolation.
When to Use CAPO
Best when both the search cost and the prompt cost matter
Perfect For
When each evaluation is expensive, racing lets you find a strong prompt without running every candidate on the full dataset.
Where the deployed prompt runs at high volume, the length penalty pays off on every call by keeping the winner short.
Tasks that rely on in-context examples benefit from jointly tuning the instruction and the example set rather than fixing one and searching the other.
When you explicitly want the best accuracy-per-token rather than raw accuracy, the multi-objective scoring targets exactly that.
Skip It When
If neither the search nor the prompt length matters, a simpler accuracy-only optimizer removes the extra machinery.
Racing helps most when there are many candidates to prune; with only a few, its early-stopping gains are limited.
If the task genuinely needs a long, detailed prompt, a length penalty can push the search away from the prompt it actually needs.
Use Cases
Where CAPO delivers the most value
High-Volume Endpoints
Optimize prompts for services that run millions of times, where a shorter winning prompt directly lowers the ongoing token bill.
Classification and Extraction
Tune few-shot prompts for structured tasks, letting joint search find the smallest example set that still hits target accuracy.
Budget-Bound Benchmarking
Reach competitive benchmark scores when compute for the optimization run itself is limited, thanks to racing's early pruning.
Cost-Sensitive Deployments
Produce prompts tuned for the best accuracy per token, which matters most for teams operating under tight cost ceilings.
Evolutionary Prompt Search
Run a population-based optimizer that mutates and recombines prompts, with cost controls that keep the search itself affordable.
AutoML for Prompts
Apply established automated-machine-learning techniques, racing and multi-objective selection, to the problem of prompt design.
Where CAPO Fits
The cost-aware, AutoML-flavored optimizer
CAPO joins OPRO, MIPRO, and GEPA as an automatic optimizer, distinguished by treating cost as an objective. Reach for it when the price of the optimization run and the length of the final prompt both matter to you.
Related Techniques
Explore complementary prompt-optimization techniques
Optimize With Cost in Mind
Explore how prompt length and accuracy trade off in the Prompt Builder, or see how CAPO relates to the wider family of prompt optimizers.