Summary
In September we posted a v0 proposal for checking whether a solver endpoint is ready for production before it goes live: replay archived production auctions against it with the time budget the driver gives, and score what comes back against what won (t/3572). Another solver team has since run it on Plasma and contributed a fix, and bleu, which holds the BYOS (Bring Your Own Solver) grant, has pointed a new sub-solver team to it.
This grant funds two milestones. Milestone 1 turns the v0 proposal into a v1 spec, makes it a one-command check a new team can run, adds a single-order replay mode for engines that bid one order at a time, and ships cow-backtester 1.0 as the reference implementation. Milestone 2 adds a report a team can keep, share and verify. The core request is 6,000 USD for 15 working days over six weeks. An optional Solana item, capped at 3,200 USD, starts only if CoW publishes Solana auction data.
Author(s)
Applicant and party of record: Fiftysix LLC, which operates kaisersolver and is the KYC-verified entity through which it joined the CoW DAO bonding pool.
The engineering is done by Kaiser (@kaisersolver on this forum), who builds and owns the kaisersolver engine and both tools below, and is the point of contact. GitHub: KaiserSolver for the tools, KaiserSnipes for upstream contributions.
Experiences and qualifications
kaisersolver has competed in production since May 2026. It is live on Arbitrum One and Base, and on BNB Chain since 18 September. Solving on CoW is kaisersolver’s only business, and both tools below started as its internal tooling.
- cow-backtester (PyPI) replays archived auctions against any solver’s
/solveendpoint, reconstructs the winning set from on-chain settlements, and scores both sides on one documented basis.--readinessturns a run into a READY / REVIEW / NOT READY verdict. It ships observed time budgets for Arbitrum, Base and BNB, and on the seven other supported chains the user supplies the budget. - cow-certify (PyPI) verifies CoW settlements from public data: sixteen checks on ten production chains plus Sepolia, a CLI and a browser app that agree verdict for verdict on an 80-settlement corpus, and a CIP-87 penalty-cap replay that reproduced all 171 published caps in its staging corpus. An independent contributor ran its 256 penalty-cap vectors against services’ own
calculate()and reported that all of them pass (t/3546 #17). cow-certify is not funded here.
Where public data is ambiguous, both tools report the case as unproven instead of counting it against anyone, and both READMEs list their limits.
kaisersolver’s own readiness runs are public (kaisersolver-readiness). Over 9,098 production auctions replayed on 14 and 15 September, its endpoints answered every request with no deadline misses and no transport errors. All three chains came out REVIEW, with these warnings: bid coverage under 50% on all three, latency headroom on Base and BNB, capture on Arbitrum and BNB, implausible-price flags on all three, and incomplete scan and field coverage on BNB. A live solver landing on REVIEW everywhere tells us the v0 capture and coverage thresholds are stricter than what live solvers achieve, and calibrating them per chain is on the list we are shipping regardless (below).
The October runs (12 October) are REVIEW again on all three chains, with fewer warnings: bid coverage now passes on Base and BNB, capture now passes on Arbitrum, and BNB’s scan coverage gap is gone with a keyed RPC.
Other work in the protocol’s record:
- CIP-87 (t/3520 #6): revert, late-settlement and order-size data for Base, Arbitrum and mainnet, measured from public data and posted during the discussion.
- solver-rewards #507: a measurement of late settlements and of the race the issue describes, across 36,596 settlements on three chains.
- docs #652: a fix to the competition rules page, merged 11 August.
Grant Description
Category: developer tooling for solvers, under CIP-82’s first priority, developer onboarding and enablement.
The problem
A solver gets no scored signal about its endpoint until it competes. The consistency metric (CIP-85) is computed from production wins and settlements, so it says nothing about a solver that is not yet live. The onboarding page says the shadow competition runs only on Plasma now and asks new bonding-pool solvers to start on BNB Chain, so a pool solver’s first chain has no shadow environment.
Between 20 September and 11 October, seven teams or individuals asked in the CoW solver Telegram how to join the shadow competition or test a new solver, and two more asked how to get started. One asked how to test before committing capital, and one asked for an offline auction export. Four said their solver already runs. Each of these is answered one at a time in that chat. A readiness run lets a team check its own endpoint before asking, and a report can give CoW’s solver team one document to look at when a team asks to go live.
BYOS sub-solvers have the same gap. bleu said access to the production endpoint for sub-solvers is still being discussed (t/3476 #17), and one integrating team found the staging feed generally empty (#16). bleu suggested cow-backtester to that team to check how competitive it is against production solvers, noting that it is made for solvers and could be adapted (#17). The team replied that it would benchmark recent BNB production auctions “using/adapting cow-backtester” (#18).
How this differs from existing tools
- Running the services locally. The docs show how to run the autopilot and driver locally in shadow mode against live order flow. That gives a solver live auctions. It does not score the answers against what won, time them against the driver’s budget, or give a verdict, and it needs
servicesbuilt and run locally. Readiness replays archived auctions without running any CoW service. - The Playground, and the funded Offline Development Mode and Performance Testing Suite grants, run and load-test the services. They do not compare a solver’s output with what won on real order flow.
Limits, stated up front
The endpoint answers with its current view of liquidity, not the state at the auction’s block, and its prices are claimed, not simulated. Capture (the share of the winners’ surplus the endpoint would have matched) is therefore an indication, and reports label it that way. Answer rate, transport errors, deadline misses, latency and solution validity do not depend on this. The corpus is auctions someone settled, and reports say so. A report’s SHA-256 digests show that it has not been edited since it was produced; it remains a self-reported result.
What exists, what is free, and what this grant funds
Already public and not part of this request: the v0 proposal (t/3572), --readiness, and kaisersolver’s readiness record, with a report per chain every month starting with the October runs, as committed in t/3572.
Shipping regardless, as promised in t/3572 §11: per-chain threshold calibration, wire-deadline budgets for the chains we run on, capture on the protocol’s referenceScore basis next to the decoded one, and a quote-lane profile.
Funded by this grant:
- Milestone 1: a v1 spec another team can implement; one command from install to a verdict, with a quickstart run in CI; single-order replay with an adapter for bleu’s BYOS proposal format; budget profiles for every chain CoW lists; the fixes from ZeroNine’s Plasma run; cow-backtester 1.0.
- Milestone 2: a report a team can keep or share, with a
verifycommand, and an opt-in export of a team’s budget row.
Who uses it
- ZeroNine, a solver in shadow on Plasma, ran readiness over 520 auctions, contributed a fix (cow-backtester PR #1) and posted the run (t/3572 #3). We have been in contact with ZeroNine since September and have discussed our respective applications. On 26 September we agreed to be named as a second user in theirs (t/3612), on the condition that no grant funds reach us or cow-certify. Their Plasma run, which answered the open call for budget data in t/3572, and PR #1 both came before that agreement. Neither grant depends on the other.
- bleu pointed a BYOS sub-solver team to cow-backtester, and the team said it would use or adapt it (t/3476 #17, #18). Single-order mode in Milestone 1 is one way to make that adaptation.
- The seven teams and individuals in the CoW solver Telegram described above.
We have not yet heard from CoW’s solver team on the three questions in t/3572 §10. We would welcome a short note from @harisang or @felixhenneke on whether per-chain budgets and single-order replay would help onboarding, so that we can narrow the scope to what is useful.
Neutrality and disclosure
The tools run on the user’s own machine, depend on no service we operate, send nothing to us, and need no key or account from us. Any solver can use them.
The conflict is that kaisersolver is a live solver, and we are defining measurements that new solvers, who would compete with it, might be judged by. To limit it, the spec sets no requirement, the core team’s definitions replace ours wherever they differ, the spec will live in the cowdao-grants organization from Milestone 1, and we publish kaisersolver’s own results under the same rules, REVIEW included. We use the backtester to improve kaisersolver, and kaisersolver plans to solve on Solana, so the optional Solana item would help us as well as other teams.
Out of scope
Best execution and EBBO (whether a better price existed elsewhere); anything that needs privileged data or changes to production services; any finding against another solver that public data cannot prove. Integration docs belong to the core team, so no docs.cow.fi page is a deliverable.
Technologies and methodology
Python 3.10+, eth_abi as the only runtime dependency, GitHub Actions CI, MIT license. v0 cites cowprotocol/services at commit e911013. v1 cites every metric definition at a pinned commit, records in each report which commit it used, and adopts the core team’s definitions where they differ. Only immutable facts are cached, and every unscanned block range or excluded auction is counted and printed with its reason.
Type of Grant
Milestone-based, with one optional item.
Milestones
| Milestones | Title | Due date | Funding request |
|---|---|---|---|
| Milestone 1 | Readiness Spec v1, one-command check, single-order replay and cow-backtester 1.0 | Week 5 | 5,000 USD |
| Milestone 2 | Shareable, verifiable readiness report | Week 6 | 1,000 USD |
| Optional O1 | Solana readiness profile | 4 weeks after its precondition is confirmed | up to 3,200 USD |
Weeks count from the day the Snapshot vote passes. Neither core milestone depends on a merge or an approval in a repository we do not control.
Specifics: Milestone 1, Readiness Spec v1, one-command check, single-order replay and cow-backtester 1.0
- Spec v1. The six metrics, the verdict rule, the guards, the 500-auction evidence floor and a JSON report schema, with each metric mapped to the
servicesdriver and autopilot code it mirrors at a pinned commit. If the core team can state per-chain solve deadlines, the spec cites them instead of measured values. - One-command check. One command from install to a verdict on one chain, needing only an RPC and an endpoint, with a written quickstart run end to end in CI against the bundled mock solver.
- Single-order replay. Each order in an archived production auction is sent as its own request, for engines that bid one order at a time, such as BYOS sub-solvers, or that cover a declared set of tokens or pairs. The response’s executed amounts are scored on the same basis as full-auction replay, against what the winner delivered for that order at uniform clearing prices. Orders the winner settled in a batch or as a coincidence of wants are flagged. An adapter accepts responses in bleu’s published BYOS proposal format (BYOS docs), pinned to a
byos-service-tscommit. Nothing in the milestone depends on BYOS being deployed or on access to it. - Budget profiles for every chain in CoW’s competition rules (ten today), each marked as observed, contributed or upper bound from public data, with the method documented.
- Fixes from ZeroNine’s Plasma run: capture over the auctions a solver bid on, shown as prominently as overall capture; an optional declared scope of tokens or pairs; rank suppressed when the gap to the winner is inside the scoring basis tolerance; and a statement in every report of what the replay did and did not exercise.
- Scoring reference. Where the spec checks scores, it pins to
services, and cites the Solver Score Conformance Test Set (t/3612) if that grant goes ahead. Milestone 1 does not depend on it. - cow-backtester 1.0, tagged and on PyPI, implementing v1 in both modes.
We will ask the core team to review the metric definitions in this thread. Their definitions win where they differ. That review is not billed.
Definition of done: spec tagged v1.0; the quickstart passes in CI from a clean runner, with its running time recorded; budget table with a status for every chain; single-order mode passing offline tests on fixtures from at least two chains, plus one fixture in the BYOS proposal format; each Plasma-run fix covered by an offline test; cow-backtester 1.0 on PyPI; release notes with the commands to run the tests and reproduce each deliverable.
Specifics: Milestone 2, Shareable, verifiable readiness report
- A report a team can keep or share: the checksummed record format from kaisersolver-readiness, packaged in cow-backtester, with a
verifycommand that checks a report against its digests. The raw bids stay with the team. - An opt-in export of a team’s budget row for the per-chain table, so teams on chains we do not run on can calibrate those rows.
Definition of done: a report produced by the tool passes verify, and the same report with one byte changed fails it; the row export matches the published table’s format; a tagged release on PyPI with review instructions.
Optional O1: Solana readiness profile
Precondition: Solana auction bodies in a public archive, as the EVM chains’ are in the solver-instances bucket today, and a published solve deadline. On 11 October 2026 the bucket had no Solana prefix.
- Replay of archived Solana auctions with the published budget.
- Answer rate, transport errors, deadline misses and latency as defined in v1.
- Validity and capture, scored where public Solana settlement data allows and otherwise reported as “not evaluated”, never passed by default.
Definition of done: a Solana row in the budget table, one published report, and release notes with review instructions.
O1 starts when the precondition is met and we confirm it in this thread, and is due four weeks later. The amount is a cap: we will confirm or reduce it once the data format is known. If the precondition is not met in time to finish within six months of approval, O1 lapses unpaid.
Length
Six weeks for the core, from the day the Snapshot vote passes. O1, if accepted, finishes within six months of approval or lapses unpaid. We will post progress in this thread every two weeks.
Funding Request
6,000 USD for the core, paid in DAI on Ethereum mainnet as the committee asked in t/3569 #9, per milestone after review: 5,000 for Milestone 1 and 1,000 for Milestone 2. Up to 3,200 USD for the optional Solana item, accepted or declined separately. No COW requested.
The request is priced at 400 USD per working day for 15 days of engineering. Fiftysix LLC carries delivery liability for the milestones and handles reporting and administration. The engineering is done by @kaisersolver, alongside his ongoing work on kaisersolver.
| Item | Work, in days | Total days |
|---|---|---|
| Milestone 1 | spec v1 and schema 3, one-command check and quickstart in CI 2, budget profiles 1.5, Plasma-run fixes 1.5, single-order mode and BYOS adapter 4, release 0.5 | 12.5 |
| Milestone 2 | record and verify 1.5, row export 0.5, release 0.5 |
2.5 |
| Optional O1 | Solana replay 3, validity and capture where possible 3, metrics and report 2 | up to 8 |
Milestone 1 is useful on its own if the committee stops there.
Payment Address (Ethereum mainnet)
0x689d90B257722edB0759e1D07ED956494DFc805e
Other Information
- Repositories: cow-backtester, cow-certify, kaisersolver-readiness.
- Sample artifacts: Base readiness report, 2026-10-12; cow-certify browser app; one certificate (one of our own Arbitrum settlements; every check passes).
- Maintenance: not funded. After the grant we will give up to 6 hours a month for 12 months, unpaid, to keep the tools working as the auction archive, the competition API and the chain list change, and to answer issues on GitHub. We use both tools every day, so we would maintain them anyway.
- Success measures, reported in this thread 90 days after Milestone 2: readiness reports from teams other than ours, counted as public reports or posts in this thread (target 3); at least one single-order run from a team that bids single orders; budget rows contributed for chains we do not run on.
- Assumptions and risks: the solver-instances bucket and the competition API stay public as they are today, and if either changes we will say so here and agree on an adjustment with the reviewer; public RPC limits and the roughly 30-day archive retention, handled by caching immutable facts, disclosing unscanned spans and supporting bring-your-own RPC; the BYOS format may change, so we pin a commit and record it in every report.
Terms and Conditions
By submitting this grant application, I acknowledge and agree to be bound by the CoW DAO Participation Agreement and the CoW DAO Grant Agreement Terms.
Posted by Kaiser on behalf of Fiftysix LLC.
Note to Committee: Please notify the Grantee of their reviewer and their steward in the thread and latest upon successful approval of the Grant on Snapshot.