Move Solver Rewards Mechanism from Theory to Practice

Technical
1

The new solver rewards mechanism has been ongoing for a couple of months. It’s only until recently that we got the time to dig into the way in which our solutions are being assessed and how our participation in the solver competition translates into P&L based on the new (more complex) rewards mechanism.

Part of the new rewards mechanism consists of penalizing solvers when their solutions fail at execution time. In this case, the solver is penalized based on the score of the third best solution (see this post for more detail) reference solution. While in theory this seems appropriate, the reward mechanism fails to consider cases in which the third best solution would also revert at execution time.

Thanks to the new auction result notification service deployed by CoW Protocol we can look into some examples of seemingly unfair penalties for our solver. The service notifies us if our settlement suceeded or failed at the execution block. We consider the following three auctions:

Auction 7000251

  • Solver wins auction.
  • Solution proposed simulates successfully at the pending block in which the solution was submitted to the Driver (17788186).
  • Solution proposed simulates successfully one block after submission to the Driver (17788187).
  • Driver attempts to settle solution at block 17788188 (two blocks after original submission), but slippage limits are triggered.
  • Solutions of next two best solvers also fail at block 17788188. Their slippage tolerance limits are also triggered.

Auction 7000269

  • Solver wins auction.
  • Solution proposed simulates successfully at the pending block in which the solution was submitted (17788200).
  • Solution proposed simulates successfully at blocks 17788201, 17788202, 17788203, 17788204.
  • Solution is flagged as failed even though it could have been executed.

Auction 6987617

  • Solver wins auction.
  • Solution proposed simulates successfully at the pending block in which the solution was submitted to the Driver (17777715).
  • Solution proposed simulates successfully one block after submission to the Driver (17777716).
  • Driver attempts to settle solution at block 17777717 (two blocks after original submission), but slippage limits are triggered. In order for the settlement to be successful, the solver would need to accept a negative slippage larger than 0.01 ETH, which is the maximum penalty attributable to the solver. Hence, the resolver is acting reasonably by taking the rewards penalty as opposed to the negative slippage loss.
  • The solutions from the 4 next best solvers (except for 1inch solver which allowed for negative slippage larger than 0.01ETH) also fail.

Conclusion

In summary, we identify the following practical imperfections from the current rewards mechanism:

  1. It assumes the score used to compute the penalty in case of failure is feasible at the execution block. However, as it can be seen by the examples provided, this is not always the case.

  2. It does not take into account the feasibility of a trade to determine if the solver should be attributed a penalty or not. For instance, unfeasible trades could be flagged based on the number of failing solutions across all solvers that participated in an auction.

  3. It does not take into account protocol induced failures. For instance, the solution for auction 7000269 could have been executed throughout several blocks. The solutions for auctions 6987617 and 7000251 were feasible for two successive blocks, but the lag between submission and execution triggered a reasonably high the slippage tolerance.

This being said, we propose to shift from the theoretical definition of failure (which is oblivious to all these practical nuances) to a practical definition of failure in which the following questions are addressed:

  • What is a solver induced failure? If solver can prove solution succeeds at the pending block she submitted to the Driver (and one block thereafter), then should it be considered a solver induced failure? If solution fails due to extreme slippage (i.e. slippage beyond max reward of solver or something like that), should it be considered a solver induced failure?

  • What is a failure that should be attributed to the protocol? If solution simulates successfully x blocks after submission to the Driver, should the protocol pay the penalty instead of the solver?

1 Like
2

Thanks for bringing up this issue. We agree that, in some instances, our current mechanism penalizes solvers when the protocol is slow. We are working hard to make the protocol faster, that is, to reduce as much as possible the time between the auction being released, the solutions being received, and the winning solution being settled on chain. A big part of this effort is “solver-driver colocation,” which showed dramatic speed improvements in our recent tests. Also, in a fully colocated world (where solvers run their own “driver”), each solver has full control over the submission and can resubmit a different solution if the route changed.

Our ultimate goal is to make the auction as fast as one block: release the auction immediately after a block is added and settle the winning solution in the subsequent block. This is an ambitious objective. To keep us entirely focused on it, we prefer not to introduce half-baked hotfixes. Also, in the “one action per block” world, the difference between computing the penalty using “reference solution” vs. “reference solution that simulated” would be minimal (remember that penalties are capped anyway).

3

Thanks for your response @AndreaC .

I still don’t see how even within the 1 auction per block scenario a solver should be penalized when solutions from other solvers also fail.

Can you please explain why in your point of view it is fair to use the reference score of a solution that would also fail as benchmark to the solution of the winning/reverting solver?

Taking this into consideration would only require one additional step to simulate the txs at the top-of-execution-block.

4

Hi @Filius ,

I’m not arguing that the current system is better or worse than what you propose; it may be that your proposal is better. My question is whether the practical difference between the two approaches is so large that we should devote resources to this problem. These resources would be at the expense of other initiatives like, for example, achieving one auction per block.

Our view is that, even now, the difference between the two approaches is likely to be negligible (because of the cap), and even if they are not they should become negligible once we make our auction faster. For the moment, we believe it is more cost-effective to focus on improving the speed of the action than opening additional workstreams. BTW, your proposal would fix the problem right now, but wouldn’t necessarily work in the future when the protocol is more decentralized (and nobody sits in the middle simulating txs).