Some comments on capping and on a testing period for the new scheme.
We intend to use a cap that leads to similar rewards compared to Oct-Jan. This results in a cap of around 0.01ETH. This cap is relatively small, leading to capping of payments in about a third of all auctions. One disadvantages of such a cap is that payments adapt less to market conditions. Another disadvantage is that it breaks truthfulness of the auction mechanism. A short analysis on the impact on incentives can be found here. (Note that you can look at the figures in the notebook as is. If you want to run the code and play around with the sliders, you need to have julia installed. The same holds for the other notebooks in the proposal.) Not using a cap would result in additional payments of up to 8% of surplus.
Solvers have to learn to submit scores, e.g., by having a model for costs and revert risk. Therefore we intend to have a transitioning period for the new reward mechanism. One suggestion is as follows:
(a) Two weeks of optional submission of either a score or a scoreDiscount. If a scoreDiscount is submitted, then driver computes a score as score = simulatedQuality - simulatedCost - scoreDiscount. Solutions are still ranked and rewarded by the old scheme. Solvers can see how they would have ranked using the new criterion and what their reward would have been.
(b) Four weeks of using the new scheme. Submitting either a score or a scoreDiscount is mandatory. The ranking of solutions and rewards are computed using the new scheme. This period is used to observe the competition and the effect of capping. If rewards are significantly smaller than expected the cap is increased. If rewards are significantly larger than expected the cap is reduced.
(c) After that testing period, the submission of a score becomes mandatory.
I still need to respond to your comments on my first post. Will finally have time tomorrow. In the meanwhile, I wanted to ask how should solvers proceed to compute simulatedCost? If I am not mistaken, currently the driver simulates our solutions and obtains a gas estimate from the simulation, which is then used to obtain the objective value and rank the solutions.
Would you be expecting that solvers simulate their solutions before submission?
I just updated the proposal one more time. The main changes are:
Information on the total budget for rewards was added and how they impact the choice of the capping parameter and consistency based payments. The planned budget is larger than the amount we have spend on rewards over the last few month by almost a third. We plan to spend this additional amount on consistency based rewards.
A timeline was added with information on a transitioning period to the new reward scheme. We expect to move the proposal to the voting stage early next week. In that case and if the CiP passes, the accounting using the new scheme would start mid of March (14.03.2023).
To avoid confusion, the naming of rewards changed slightly. The proposal now distinguishes between payment and reward. The naming is now in line with the current reward scheme. (In previous versions of the proposal, the name reward was used for the full payment including gas reimbursements.)
Longer answer:
For now we plan to simulate all solutions, as we currently do. However, for the proposed reward mechanism it is at least necessary for solvers to have a good idea about gas costs before they submit a score and solution. One way to obtain those costs is by running a simulation themselves. Note that solvers would not try to replicate the exact cost simulation we currently use but try to estimate the actual cost they would have by executing the transaction.
We plan to have a transitioning period, where solvers can opt to submit a scoreDiscount instead of a score, as you have suggested. In this case, the driver will use the result from its simulation to compute a score by subtracting the score discount from the simulated objective value.
With driver/solver colocation, all simulations should be done on the side of solvers. The protocol would then just look at submitted scores and choose a winner. The winning solver would have to make sure that their settlement is mined.
how the COW/ETH exchange rate will be computed though.
Generally, the easiest approach would be to use the price provided by a fixed source on, say, the day before payout. To fall in line with price sources we use in Dune ā this would be coming from coinpaprikaās API accessed via the code here.
Might not hurt to work toward a more sophisticated approach though.
āHow does this mechanism work in the case when a solver executes more than 1 settlement in the same transaction?ā
This should be entirely against the rules and any solver doing this should be permanently eliminated from the game. In fact, I believe it is against the rules from a āuniform clearing pricesā perspective ā unless the solver uses the same prices in each settlement.
However, the contract does not prevent such a scenario from happening. Apart from the obvious ZERO rewards are attributed for such batches we will also want to prepare our software in a way that can keep on chugging and act as expected when events like this occur. A simplified treatment of this might be to treat the entire transaction as a single ābatchā and perform slippage/surplus calculations according to this rule (although there are two price vectors). Another approach could be to separate these into the multiple batches they areā¦ but then there is a problem of having only a single reference score for multiple batches.
The vision of CoW Protocol is to have one price per token per block, so even having multiple settlements (different transactions) in the same block should be avoided.
Given that this is theoretically possible (if a tx keeps dangling in the mem-pool, while a new auction is being kicked off), I agree that two settlements within the same transaction should be implicitly forbidden (and probably is already by social consensus).
I donāt think this is particularly related to the new solver rewards model (it already applies to the current status quo) so I would not explicitly add this to the CiP.
It is worth noting that in the solver+driver colocation model, external solver teams are expected to run a driver (this is a simplified version of the existing solver binary part of the core protocol services) and the solver engine (what the existing HTTP external solvers are today). We would provide what we are calling a āreference driver implementationā which does the transaction simulation, settlement calldata building, etc. as before.
That being said, as @felixhenneke mentioned, solvers that are more accurately aware of gas characteristics of their settlement would likely have a competitive edge over solvers that arenāt. However, the way the driver + solver co-location is designed, there should be no initial implementation burden on external solvers (i.e. existing HTTP solver engines will continue to work as-is), but enable external solver teams to āmergeā the driver and solver engine components over time to become more advanced and competitive participants in the competition.
I think this is technically forbidden ATM as all external solver accounts are EOAs and unable to execute two settlements in the same transaction. I agree this isnāt part of this CIP, but I donāt think we plan on changing this in any way that would enable solvers to submit multiple settlements in the same transaction (the only account technically capable of doing that is the solver rewards Safe multisig, but this is not a participating solver and is used exclusively for fee withdrawal).
Two settlements in the same block can (and do) happen for a couple reasons:
Pending transactions in the mempool that end up getting included in the same block
Production and Barn environments having transactions execute in the same block
I would like to sugest to make pernanent the idea of allowing the solver to not submit any score, and if so simply compute it in the driver using the current model for estimating revert risk and the simulated objective and costs. At least until the solver driver co-location is in production.
The reason is very practical: I have zero interest in spending time adding both the risk estimation and simulation machinery in my solver, specially knowing that the plan is to have a module that already does that co-located with the solver.
Having a default score this makes sense to me in the long run. If it turns out that it needs to be adjusted, then I would rather adjust the co-located driver than reimplement it from scratch.
The current plan is to have driver/solver co-location done around the same time as the start of the new scheme. In that case no additional work would be necessary.
If co-location does not happen in time, the submission of score discounts or more complicated default scores sound like a good option to me. The dates in the transitioning period would then change accordingly.
We do not currently plan to permanently provide sophisticated default scores. Personally, I expect that solvers implementing their own risk and costs analysis will have a significant advantage in the competition. The dry run and the testing period will give us more insights there.