Ethereum All Core Developers Execution Call #178 Writeup
On January 4, 2024, Ethereum developers gathered over Zoom for All Core Developers Execution (ACDE) Call #178. The ACDE calls, usually chaired by Ethereum Foundation Protocol Lead Tim Beiko, are a bi-weekly meeting series where developers discuss and coordinate changes to the execution layer (EL) of Ethereum. This week, the call was chaired by a pseudonymous Geth EL developer by the screen name “Lightclient”. Developers reconfirmed dates for the next three public testnet activations of the Cancun/Deneb (Dencun) upgrade. They also discussed priorities for code changes, also called Ethereum Improvement Proposals (EIPs), in the next hard fork upgrade after Dencun, dubbed Prague/Electra.
There were no specific updates made to the Dencun upgrade over the holidays. Since the last ACD call on December 21, client teams have been working on preparing new releases for the Goerli testnet. Because of prior delays to upgrade testing because of Prysm, Geth developer Marius van der Wijden asked for an update from the Prysm client team on their progress with cutting a new release. Prysm developer Terence Tsao confirmed that the Prysm team will be ready with a new release for the Goerli hard fork next week. However, the release for Goerli will be a “pre-release”, meaning it will not be the version of Prysm that will be recommended for use on Ethereum mainnet. After the Goerli hard fork, the Prysm team plans on cutting another release with certain changes and updates that will be recommended for users to run on mainnet and tested on the Sepolia and/or Holesky testnets.
While Tsao said the Prysm team is comfortable with a Goerli hard fork activation date on January 17 as discussed on ACDE #177, he recommended holding off on setting dates for the Sepolia and Holesky hard fork activations until after the Goerli hard fork. Since ACDE #177, Ethereum Foundation Protocol Support Lead Tim Beiko has proposed public testnet fork times for all three Ethereum public testnets, Goerli, Sepolia, and Holesky. The proposed fork activation times are as follows:
Goerli - 17 Jan 2024 - Epoch 231680 – Timestamp 1705473120
Sepolia - 30 Jan 2024 - Epoch 132608 – Timestamp 1706655072
Holesky - 07 Feb 2024 - Epoch 29696 – Timestamp 1707305664
Lightclient asked whether other client teams outside of Prysm are comfortable with the proposed timing by Beiko for a Goerli hard fork activation. All client teams on the call including Geth, Lodestar, Lighthouse, Teku, and Besu, confirmed the timing looks good to them and that they would be ready with a release for Goerli node operators at the latest by next week. The Lighthouse client team noted that their release like Prysm may be a pre-release, given that they are still testing certain networking features on their client.
Dencun Timeline Disagreements
Then, Lightclient opened the floor for discussion about the proposed activation times for the Sepolia and Holesky testnets. A pseudonymous developer for Prysm by the screen name “Potuz” recommended holding off from setting dates for the last two testnet upgrades before mainnet. “[We should] try not to commit to dates now because things may not really go fine with Goerli and going back from that is a problem. Adding a new release with the right epoch with no changes is easy. Removing a release and fixing bugs is a problem. It takes much longer than a couple of weeks,” said Potuz.
Lightclient highlighted that client teams would not have to put out a release until a week after the Goerli hard fork and therefore, may not have to necessarily remove a release unless an issue with the upgrade is discovered on Goerli after January 24 or thereabouts. Geth developer Marius van der Wijden said that he doesn’t see any harm in setting dates for the Sepolia and Holesky testnets given that developers can always change them if something on Goerli breaks.
Ethereum Foundation DevOps Engineer Barnabas Busa wrote in the Zoom chat that in his view it only makes sense to cut a new release for the Sepolia and Holesky upgrades after confirming the releases for Goerli are working properly. Agreeing with this sentiment, a Lighthouse developer by the screen name “Sean” said that developers can set a “tentative” date for the Sepolia hard fork but should see how Goerli goes before moving forward with the January 30 date.
Potuz recommended an additional week for testing between the Goerli and Sepolia hard fork activations, basically two weeks for analysis, instead of three. He said the additional week of testing would give time for the client releases to “soak” for a few days before client teams are required to cut new releases again for the next testnet upgrade. “Two weeks is just too close. That’s what I’m pointing out,” said Potuz, adding that the three-week turnaround may not be necessary between the Sepolia and Holesky hard fork activations if Goerli client releases are adequately analyzed and tested.
Potuz’s views were controversial. Ethereum Foundation Ansgar Dietrichs called the time between the first public testnet activation for an upgrade and the mainnet activation for an upgrade typically a “dead time” for developers that does not need extending. However, Dietrichs also noted that the desire to extend the time between testnet upgrades is one that developers should more seriously discuss in the context of hard forks in general, not only the Dencun upgrade. “If there’s a desire to have a more drawn-out process that’s something we should probably then discuss when we have time and not right before a hard fork,” said Dietrichs.
Lightclient agreed with Dietrichs’ sentiment saying that developers likely would have been more lenient about extending testnet timelines for Dencun if the discussion had been had earlier in say October. “I think also part of it is we wanted to ship this [upgrade] in the fall of last year so now we’re really trying to make it happen and I think we’re taking a little bit more aggressive timelines,” said Lightclient.
Sticking With Aggressive Timelines
Based on sentiments shared by developers on the call, Ethereum Foundation DevOps Engineer Parithosh Jayanthi suggested delaying the Sepolia hard fork upgrade by a week or so and setting a date for the Sepolia hard fork on the ACD call on January 25th after the Goerli upgrade. Marius van der Wijden was opposed to relying on ACD calls exclusively to re-discuss dates for testnet upgrade activations. “What I would really like to prevent is for us to have to go to another All Core Devs to confirm dates,” he said, adding, “I would hate to go on to another All Core Devs calls just to say, ‘Okay, Sepolia is a go now.’ And now we have to wait two weeks until we can actually do Sepolia.”
Trying to appease all sides, Geth developer Guillaume Ballet recommended creating two sets of tentative dates for the Sepolia hard fork, one that developers stick with if the results of the Goerli hard fork are positive and another that developers fall back to if the results of the Goerli hard fork are negative. However, Lightclient and Dietrichs were both opposed to this idea given that the nature of bugs and issues on Goerli would have to be evaluated first before developers can realistically set a new timeline for the Sepolia hard fork.
As an aside, a pseudonymous developer for the Ethereum Foundation testing team by the screen name “Danceratopz” asked whether developers would want to wait to evaluate blob expiry on the Goerli testnet before moving on to upgrading Sepolia. As background, blob expiry refers to the removal of blob data from Ethereum state after a period of roughly two weeks. For more information on blobs and proto-danksharding, read this Galaxy Research report.
Sean from Lighthouse and Justin Florentine from the Besu team were both in favor of evaluating blob expiry on one of the three testnets before activating Dencun on mainnet. Florentine emphasized that waiting for blob expiry on a testnet would also benefit Layer-2 rollup protocol teams and application developers in terms of preparation for the Dencun upgrade. Sean from Lighthouse said that while blob expiry was not essential to observe on Goerli, it could be a reason for extending the testing period between Sepolia and Holesky so that developers and Layer-2 teams can go through the full blob lifecycle on Sepolia. There was no clear agreement from other developers on the call about Sean’s suggestion.
Instead, Lightclient asked developers on the call if they were comfortable sticking with the proposed timeline by Beiko, which is upgrading Sepolia on January 30 and Holesky a week after on February 7. Because there was no more vocal disagreement from developers, Lightclient said developers would stick with the original timeline. Potuz wrote in the Zoom chat that he would prefer upgrading both the Sepolia and Holesky testnets at once on February 7, rather than upgrading the former one week earlier. In a Discord message after the call, Lightclient reconfirmed that the testnet timeline for Dencun will remain unchanged for now.
Next, developers discussed what EIPs should be prioritized for the next upgrade after Dencun, Prague/Electra. Marius van der Wijden said that developers should focus on delivering the Verkle tree upgrade in Prague/Electra over other EIPs. He added two caveats to this sentiment, the first being the readiness of Verkle. As discussed on ACDE #177, developers are planning a dedicated ACDE call to dive deeper into the implementation details of Verkle and its readiness for a hard fork upgrade.
The second caveat mentioned by van der Wijden is the ability to decouple upgrades on the EL from the consensus layer (CL). Van der Wijden mentioned that there are “high priority, super urgent” EIPs on the CL that may need to be implemented faster than the Verkle upgrade on the EL. “I think it’s important for the consensus layer folks to discuss whether it would make sense for them to have a hard fork with those [urgent] changes and if that can be done without involvement of the EL or whether it needs involvement of the EL and we would need to do a joint hard fork anyway and then I would be okay with having a smaller hard fork. So, high priority is Verkle definitely and we should push for it with these two caveats,” said van der Wijden.
Ethereum Foundation Researcher Ansgar Dietrichs wrote in the Zoom chat that he was “strongly opposed” to focusing on Verkle for the Prague/Electra upgrade as that would likely mean the upgrade is delayed to 2025 given the complexity of the code changes required for Verkle. Lukasz Rozmej, a developer for the Nethermind client, was also in agreement with Dietrichs. “My experience tells me that state redesigns are extremely hard and they take an extremely long time,” said Rozmej, adding, “While I think Verkle is great and it’s doing great progress I think if we just focus on Verkle, it would take at least a year for the next hard fork or even more. So, my proposition would be to potentially focus on some smaller hard fork while each team would commit to Verkle and assign appropriate resources, work force, brain power, however you want to call it, for this topic.”
Focusing on Verkle
Developers were divided on whether Prague/Electra should focus on Verkle or prioritize smaller code changes that could be shipped faster than Verkle. Ballet emphasized that in his view there is “no such thing as a small fork” and the more developers wait before implementing Verkle, the harder it will be to implement the update to Ethereum’s state. Tomasz K. Stańczak, also a developer for the Nethermind client, recommended taking an ambitious approach by committing to more EIPs than could likely be included in Prague/Electra. “[Let’s] take advantage of the team’s capacity and the year where we have to show that we handle our biggest challenges. If Verkle ends up showing to the teams that there’s more and more difficulties piling up by March then maybe people question once again and say, ‘Okay, Verkle drops.’ But then we stay with a pretty good set of other EIPs that we will include,” said Stańczak, specifying some other big ticket EIPs aside from Verkle that could be included in Prague/Electra as EIPs related to staking, re-staking, and account abstraction.
In response to Stańczak, Lightclient said that it may be difficult for developers to continue to debate what EIPs should be included in Prague/Electra after committing to a set of EIPs, one of which, referring to Verkle, is “an 18-to-24-month project.” Andrew Ashikhmin, a developer for the Erigon client, was in favor of shipping smaller EIPs in the Prague/Electra fork and working on Verkle in parallel for a future fork thereafter. Ballet was in favor of Stańczak’s proposal to focus on Verkle for Prague/Electra and drop it from the upgrade if major issues are found with its implementation that require more time to resolve.
Focusing on CL-Only Upgrades
On the topic of decoupling EL and CL upgrades, Potuz mentioned that there is only one EIP proposed for Prague/Electra that would require a change to the CL only. “[The only change] that is purely CL is removing attestation index committee … and all the other changes, even those that look like they will be only CL like Max EB, they depend on other changes that are from the EL. So, I think pure CL forks are not going to happen. At least, I don’t see them happening this year and not next year. We don’t have enough proposals that are purely CL,” said Potuz.
Even so, Ansgar Dietrichs said that there are certain EIPs that are mostly CL-focused upgrades and require minor changes to the EL that could be executed easily by EL client teams. These EIPs would still require a coordinated hard fork between the EL and CL. Dietrichs then added that in his view from the CL side, data availability sampling (DAS) is the most important code change to ship after EIP 4844. There was some disagreement between Dietrichs and Lightclient about whether DAS would require a hard fork to implement.
Focusing on EOF and Other EIPs
A pseudonymous developer by the screen name “Rodiazet” who works for the Ipsilon team at the Ethereum Foundation, which is a team dedicated to research on the Ethereum Virtual Machine (EVM), was in favor of implementing EOF in Prague/Electra. As background, EOF, which stands for EVM Object Format, is a bundle of improvements to the EVM that was initially considered for inclusion in the Cancun/Deneb upgrade. Refer to prior call notes to learn more about EOF.
There were a few other EIPs that developers on the call raised for consideration outside of Verkle such as EIP 5920 (PAY opcode) and EIP 2537 (Precompile for BLS12-381 curve operations). The full list of EIP candidates for Prague/Electra can be found in the upgrade meta thread on the Ethereum Magicians website here. While most developers were in favor of prioritizing Verkle in some capacity after Cancun/Deneb, it was unclear to what extent Verkle should be prioritized for Prague/Electra over smaller EIPs that could be implemented more quickly and easily in 2024. Lightclient emphasized that developers did not need to reach a final decision about the content of Prague/Electra on this week’s call. He recommended continued discussion on the topic during forthcoming ACD calls.
Developers then quickly touched on the EIPs featured on the Prague/Electra Meta Thread that had not yet been discussed on the call, including but not limited to EIPs such as:
EIP-7002: Execution layer triggerable exits
EIP-7549: Move committee index outside Attestation
EIP-3074: AUTH and AUTHCALL opcodes
EIP-6110: Supply validator deposits on chain
EIP-6913: SETCODE instruction
EIP-7377: Migration Transaction
EIP-4444: Bound Historical Data in Execution Clients
EIP-6404: SSZ Transactions Root 6
EIP-6465: SSZ Withdrawals Root 4
EIP-6466: SSZ Receipts Root 4
EIP-7212: Precompile for secp256r1 Curve Support
For a detailed overview of the sentiments shared about the above EIPs, refer to the full call recording posted on YouTube.
Formalizing EIP 7587
Finally, Ethereum Foundation Researcher Carl Beekhuizen resurfaced discussions about EIP 7587, which would reserve a set of precompile addresses for use by Layer-2 protocols. For background on EIP 7587, refer to prior call notes here. Beekhuizen asked developers how to best go about formalizing the EIP as an informational EIP that creates a norm for Ethereum governance processes moving forward. Nethermind developer Ahmad Bitar recommended including the EIP in the EIP 1 document which outlines guidelines for the EIP process. Lightclient recommended discussing this topic further on the Ethereum Magicians website and revisiting this topic as needed during the next ACD call.