Thanks Haris!
I think these are two separate issues: there’s the CIP-74 consequences, in this post, and there’s the penalties issues I raised on the other post, which are pre-CIP-74. Possibly the former makes the latter more apparent, but I don’t think it changed anything. I’ll stick with your suggestion and reply here, but I’d like to focus only on the penalties.
the reference score should, ideally, already account for the revert probability
I agree, as it is, things work on expectation (an variance-wise too).
Thought experiment: Imagine that you could prove, that for the N solvers that submitted a solution to an auction, all of them would have reverted, not because there was something wrong with their solutions, but because someone payed for a CEX arbitrage worth 10x more than the 1 ETH order on cowswap. Do you think the protocol should still charge a penalty in that case?
My point is that a solver can only do so much with what it is given (the order volume). There are externalities common to all solvers, and when all solvers learn to “price the penalty” game, they will be passed, as fees, to the user. Why would a user pay for the CEX-arber event, that even happened in some other auction?
(Sure, one can argue the CEX-arber can come and work as a cowswap solver, but until cowswap dominates the world, let’s find an interim solution 
)
For reference: the original post.