Welcome to Ohio

How Half the Ethereum Network Wound up in Ohio

By Phillip McCabe. Originally published 08/28/22

The State of MEV, Staking & Centralization on the Ethereum Network Post-Merge, Or:
How Half the Ethereum Network wound up in Ohio

Before we start, Crypto Caverns is not against regulation. Properly done, reasonable regulations can help form a stable marketplace. We like good regulation, and we like good blockchain technology. Vulnerable blockchain technology however can result in unpredictable and unstable situations when attacked. A global financial network must be at least as resilient to disruption as the status quo, and we’re concerned about the present state of the network.

What is MEV?

MEV is short for Maximal Extracted Value, formerly known as Miner Extracted Value. It’s not a new topic for those familiar with DeFi or Ethereum, but post-merge has taken on a new, more serious form that many aren’t aware of.

DeFi is a complex web of interoperable smart contracts. It’s an amazing feat of human ingenuity that has turned blockchain technology from simple currency into any number of imaginative decentralized financial (or other) products and is here to stay. How it works is for another document but suffice to say that naturally occurring market inefficiencies arise and similar to legacy markets, there is incentive to capture these inefficiencies. 

MEV can come in as many forms as DeFi itself, but more generally these transactions are arbitrage, calling known liquidation functions on over leveraged accounts, or sandwiching inefficient trades. It’s generally considered a nuisance by most DeFi users and when it first got big it caused an issue with gas on the Ethereum network. 

How MEV Worked

Some MEV opportunities are well known, but most profitable ones will require an absurd amount of research by an individual or firm. Essential to most successful MEV is having some level of control over the order of transactions within a block, or across blocks. Having no other choice but to bid for blockspace to miners, gas would quickly and arguably artificially rise as bots competed for MEV. 

Early “Searchers” looking for MEV opportunities would find themselves shut out if they were not a big firm due to the gas costs. Worse, Searchers used to be incentivized to submit their transactions to the biggest pools as they would have the best chance of being included in the required block. This would in turn let the pool advertise larger rewards and attract more miners to the pool. This became a positive feedback loop attracting more searchers to the pool and more miners and so on. There was a fear that this would centralize in perpetuity. 

What resulted was an organization known as Flashbots attempting to solve this issue by bringing it into the light and offering free tools to better democratize this process. Searchers will now submit their findings to an open relay, run by Flashbots, which would allow any pool to connect. Now searchers could get a chance at every block, and every pool had a chance at MEV. 

Naturally, mining pools would run the mev-boost software and Flashbots relay as an added revenue stream and boon to their users. At one point, an estimated 90% of all MEV was going over the Flashbots Relay, located on a server in Ohio.

What’s Changed and Why This is Now an Issue

With MEV under Proof of Work, even a system where 90% of the transactions are moving across a single relay went largely unnoticed. What was noticed was miners complaining that mining pools were frequently robbing them of their promised earnings. Why did nobody care about the relay? To answer this we need to take a step back.

It’s frequently pointed out that mining pools are a point of centralization and that Bitcoiners should look at the huge percentage of hashrate at single large pools. This position has some issues.

First, while running a mining pool at scale is no small job- compared to what it takes to create the mining operations they are servicing, setting up a mining pool is trivial. Second, moving from one pool to another is also trivial, and can be done at a moment’s notice or even automated to be done under whatever conditions the mining operation would prefer. 

This means that the party with final responsibility for securing the network (the party with the equipment) has very little to no costs to continue proper, uncensored operation of the network should one of the parties he’s trusted upstream misbehave. It can be done as fast as the behavior is noticed, and the equipment restarted. Advanced setups might not even need the restart.

In contrast, under staking systems the party with final responsibility is the entity in possession of the staked coins. We can pretend to operate decentralized pools, but ultimately nodes like miners are physical hardware that must be operated and secured by a human. In the harshest simulation, we should consider pooled staked coins completely in possession of the pool and unable to be freed by their rightful owners. 

However let’s be charitable enough to assume they only misbehave with regards to their transaction processing/censorship and not outright stealing from users. All staking networks with a slashing mechanism must involve a token lockup period, else the slashing could be avoided. To protect against long distance attacks and align stakers incentives to secure the network, this period is usually chosen to be as long as reasonably possible, typically in the 21 – 30 day area.

This exact friction placed on the party with final responsibility (here, stakers) is what changes this from a rather trivial issue to a serious one. The responsible party must now weigh their options: lose income for the specified unlocking period, or continue with a situation they might not prefer but is the only optimal choice for them. The “honest” decision now has a much higher price tag than before.

Builder-Proposer Separation Does Nothing Under Staking Systems

It was proposed that attempting to return to a model similar to Proof of Work, where the pools make blocks but the parties with final responsibility do the signing on headers only, would potentially be at least a partial remedy. We remove the ability for the entity writing blocks to the network (miner or validator) to see the transactions they were validating, instead relying on outside parties known as “block builders” who will collect the MEV transactions from the searchers then pass them along the relay. They would hide their contents from the validator, removing the ability for the (now very concentrated under staking) pool of validators to censor transactions. 

However this didn’t inhibit censorship on the blockchain, but rather moved the ability/responsibility for this activity to a much more concentrated, less understood and more difficult to enter pool of participants: the MEV Relays.

The Hands of the Validator are Tied

Transaction flow at time of writing
Current Transaction Flow

Validator staking pools are in a zero-sum competition for the staked coins on the network. Though presently staked ETH is locked until further notice, we can assume they will someday be somewhat liquid with a sufficient unlock delay to prevent slashing abuse. Even then, the staking pool with the highest yield will trend ever larger, as users staking at any other pool will find themselves losing market share.

These are handcuffs however, as the pool is now compelled to deliver the highest yield possible to its stakers. But isn’t this a good thing? Now we keep the pools as honest as the Proof of Work pools, maybe even more. But closing one window has opened another. Being compelled to offer the highest yield, but no longer having any real control over the contents of the blocks, validator pools must seek the highest paying relay- all the time. 

What does this mean? Staking pools can only care about how much they are being paid by the Relay or risk destruction in the long run. Given the very small number of both Relays and Staking Pools- deal making between relay and pool become trivial and incentivized. Well funded Relays could pay pools directly to carry their sub-par or selectively censorious Relay. Things like trades between companies for off-chain marketing, or even payment for order flow are easy. 

Validators will not know or be able to tell if the block they are signing off on is censored, as it’s not revealed by the builder until after it’s signed for, to prevent the validator from stealing the MEV– And it all has to be this way, regardless of consensus mechanism. It’s only a problem when you remove the balance between large economic nodes with liquidity and production of the blockchain. In a proof of stake system where your party of final responsibility can’t and won’t ever be free to leave at a moment’s notice from the parties they place their delegated trust in, this equates to the capture of your network by those nodes. 

The band-aid to attempt to fix what is ultimately a systemic issue with all purely staking networks has only made things worse. Having parties with dissimilar interests working together on a network makes the network stronger. To illustrate the relationships, ask Bitcoiners: imagine Bitcoin where the largest owners of mining farms are Coinbase, FTX, Kraken, and Binance. Do you really care what pool they use, or does it not matter? Is this a network you’re interested in or consider secure?

Porportion of staked ETH by entity

Things Were Better Under PoW

We mentioned that previously, the Flashbots relay controlled up to 90% of the MEV transactions under Proof of Work mining. In both cases, there is still the option of transactions in the open mempool. But the underlying incentive dynamic has changed, the system of checks and balances removed. 

If your transaction isn’t “MEV’d” or otherwise not on the relay, it’s because it’s worth less to the network. People with sandwiched transactions are effectively paying extra gas to the network without explicitly meaning to- meaning people without MEV on their transaction will pay higher gas, and that is the system working as intended. “Soft, Unintentional Censorship” in this way is fair and not really an issue. What we have now is a series of band-aids and obfuscation layers to a system that is only truly a problem under proof of stake.

Builder-Proposer separation was real, and effective under Proof of Work. The heavy economic validating nodes of the network had essentially nothing to do with creating the blocks, only checking them, in a system where rejection means they can’t use the network. There was also no hard legal culpability on their part, nor was any part of the block production system effectively able to be terminated. Mining Pools, Miners, Exchanges and Users exist in a Mexican standoff with a bonus of being able to pass the buck on legal responsibility endlessly. The most vulnerable part would have been the practically disposable mining pools, who were doing the actual assembly of the transactions.

But now that the relay has disproportionate power over the validators due to the nature of staking pools, we’ve unintentionally exacerbated a problem that was only really a UX issue under Proof of Work, to a systemic existential threat under Proof of Stake. Additionally, it isn’t a “someday, this might happen” warning- that warning has been given since Ethereum decided to start building a consensus mechanism which had been rejected by Satoshi & Team before Bitcoin was even released– This is the current state of Ethereum, today. The Relay is the new King of Ethereum: “DAO’d”, or otherwise.

So what do we do? What’s the point? Well I don’t think anyone reasonably expects Ethereum to spend another 6 years moving back to Proof of Work. Like many of the replies to similar concerns raised in forums, this is as good as it gets. I don’t have solutions here, other than maybe to point out that at no point was the switch to proof of stake a technical imperative, nor was there any evidence supporting Ethereum’s energy usage made any difference for the environment. New, ZK systems may yield solutions that render all of this moot. Whatever happens from here, I can only hope its lessons are not forgotten.

Ohio actually isn’t that bad anyway.

Now you are stuck in Ohio.
Image Source: Natalie Dee

References:

https://www.mevwatch.info/

https://blog.bitmex.com/ofac-sanctions-ethereum-pos-some-technical-nuances/

https://blog.bitmex.com/flashbots/

https://ethresear.ch/t/two-slot-proposer-builder-separation/10980/5

https://cointelegraph.com/news/ethereum-at-the-center-of-centralization-debate-as-sec-lays-claim

https://ethresear.ch/t/mev-boost-merge-ready-flashbots-architecture/11177

Disclaimer: This content is intended for informational purposes only. It is not financial, legal, or tax advice, and is not guaranteed to be correct, complete or up-to-date. Always consult with a licensed professional.