Ethereum All Core Developers Consensus Call #120 Writeup
On October 19, 2023, Ethereum developers gathered over Zoom for All Core Developers Consensus (ACDC) call #120. Chaired by Ethereum Foundation Researcher Danny Ryan, the ACDC calls are a bi-weekly meeting series where developers discuss and coordinate changes to the consensus layer (CL) of Ethereum. This week, developers discussed:
version 1.4.0-beta.3 of CL specifications,
launch conditions for Devnet-10,
and blob latency analysis by Gajinder Singh, a software developer who maintains the Lodestar and EthereumJS clients.
The Summoning
Danny Ryan announced the release of new CL code specifications for the Cancun/Deneb (Dencun) upgrade, dubbed “The Summoning.” The release, formally tagged as version 1.4.0-beta.3 in the CL GitHub repository, features two main changes:
Mainnet KZG configurations: The formatting work required to finalize the output from Ethereum’s trusted setup ceremony has been completed and included in the latest CL spec release. For background on what KZG is and the role it plays in EIP 4844, read this Galaxy Research report.
New gossip rule: Teku developer Enrico Del Fante has created a new gossip rule to ensure that CL nodes do not propagate a higher number of blobs than the maximum number of blobs per block, which is currently defined in the specifications as six blobs per block. This will ensure that validators cannot spam the network with invalid messages exceeding six blobs per block.
Devnet-10
The changes in version 1.4.0-beta.3 of CL specifications will be tested on the next devnet, Devnet-10. However, there are a few blockers to the launch of Devnet-10. Barnabas Busa, a DevOps Engineer for the Ethereum Foundation, said that he is still waiting on client teams for new software releases. Once these are ready, Busa hopes to launch the next devnet sometime tomorrow on Friday, October 20th.
A pseudonymous developer for the Prysm client who goes by the screen name “Potuz” noted that the mainnet KZG configurations included in the latest CL specifications cannot be merged to Geth and therefore Prysm until the changes are made to “go-kzg,” a separate code repository implementing the KZG commitment scheme into the Go programming language. Developers expressed frustration on the call about the dependencies between go-kzg, Geth, and Prysm repositories. This is not the first time Potuz and other developers have raised concerns about the challenges of using KZG libraries for EIP 4844.
Parithosh Jayanthi, also a DevOps engineer at the Ethereum Foundation, shared that developers could launch Devnet-10 without mainnet KZG configurations and new client releases but this would mean developers are not testing any new code on Devnet-10, but rather re-testing the code launched on Devnet-9.
This week, developers discovered an issue in CL client implementations on Devnet-9. Mario Vega who is on the testing team at the Ethereum Foundation explained validators are not correctly pruning invalid blobs. If a validator maliciously broadcasts a correct blob to certain peers and an invalid blob to other peers, the peers that had received the invalid blob will be unable to follow the head of the chain, Vega said. The precise conditions that result in such behavior have already been created into a hive test, meaning these conditions can be easily reproduced and tested against new client releases so that developers can know immediately whether their fixes for this issue are working or not.
Related to this issue, Enrico Del Fante mentioned that the Teku client will not import a block under certain conditions if the block contains a blob or number of blobs with an index that doesn’t match any existing blob commitments in the block. Ethereum Foundation Researcher Dankrad Feist noted that the validator proposing blocks intentionally with invalid blobs would get to no benefit from doing so other than potentially increasing the computational load of Ethereum’s peer-to-peer network and causing certain validators receiving the block to fall out of sync with the network. Feist emphasized that there is no economic gain from such behavior and even if a validator were to do this, the additional computational load on Ethereum’s peer-to-peer layer would be capped by the maximum number of blobs per block.
Even so, to dissuade this type of behavior from validators, developers discussed potentially adding a new slashing condition. The new slashing condition would try to monitor for the inclusion of invalid blobs into a block and dissuade validators from intentionally causing strain on the peer-to-peer layer. However, due to the high cost in terms of research and analysis needed to change Ethereum’s validator economics, Ryan suggested that this proposal be discussed further in context of the next upgrade after Dencun, Prague/Electra.
Ryan added that developers could also specify in CL specifications for Dencun the correct behavior under this scenario, which is that validators should still import blocks regardless of whether blob indexes match block commitments. Ryan suggested that since there is no economic rationale for a validator to propagate blocks with invalid blobs in the way described by Del Fante and the maximum strain on network load such irrational behavior could cause is capped by the max blobs per block limit, the minor change to CL specifications should be enough to address the issue in the short-term while developers consider more permanent solutions like a new slashing condition for future upgrades.
Jayanthi reconfirmed with Ryan that developers will not launch Devnet-10 until mainnet KZG configurations are setup in clients and the issue identified by Mario Vega is resolved in at least the Prysm client. Ryan reaffirmed that the conversation about a new slashing condition is only in context for the Prague/Electra upgrade, not Dencun, and the change to CL specifications about the importing of blocks with invalid blobs should not be a blocker for the launch of Devnet-10. Representatives from the Prysm and Lighthouse client teams said that they will have an updated release for their software by tomorrow, October 20.
Block Latency Analysis
On Saturday, October 14, Gajinder Singh, maintainer of Ethereum clients Lodestar and EthereumJS, shared new analysis on block import latency based on number of blobs gossiped. Singh noted that from his experiments on Devnet-9, the latency of blocks increases significantly, the higher the number of blobs received by the validator.
The following is a table summarizing Singh’s findings. The first column lists the percentage of full block arrivals. The values in the table are the seconds it takes for the block to be imported.
Caption: Block import latency in seconds by number of blobs included in a block.
Source: Gajinder Singh, Twitter
Singh tweeted that while the latency introduced by one blob per block is negligible, any number higher than two introduces significant latency. Dankrad Feist said that the numbers from Singh’s analysis were very different from his experiments propagating large blocks on mainnet Ethereum. “It seems like this data can’t be correct,” asserted Feist, adding that blobs are processed in parallel with blocks so adding the latency from blob processing to block times are an inaccurate way to create projections about block propagation times on mainnet. Singh’s projections about block propagation with blobs can be found here. Feist called Singh’s analysis “overly pessimistic.”
Despite the disagreement, both Feist and Ryan admitted that Singh’s analysis did warrant a replication by another client team. Ryan encouraged another CL team to try and reproduce Singh’s experiment on Devnet-9 to see if they get similar values and results. If there is new data to present about block latencies with the introduction of blobs, Ryan encouraged developers to re-discuss the topic on next week’s ACDE call.
Legal Disclosure:
This document, and the information contained herein, has been provided to you by Galaxy Digital Holdings LP and its affiliates (“Galaxy Digital”) solely for informational purposes. This document may not be reproduced or redistributed in whole or in part, in any format, without the express written approval of Galaxy Digital. Neither the information, nor any opinion contained in this document, constitutes an offer to buy or sell, or a solicitation of an offer to buy or sell, any advisory services, securities, futures, options or other financial instruments or to participate in any advisory services or trading strategy. Nothing contained in this document constitutes investment, legal or tax advice or is an endorsementof any of the digital assets or companies mentioned herein. You should make your own investigations and evaluations of the information herein. Any decisions based on information contained in this document are the sole responsibility of the reader. Certain statements in this document reflect Galaxy Digital’s views, estimates, opinions or predictions (which may be based on proprietary models and assumptions, including, in particular, Galaxy Digital’s views on the current and future market for certain digital assets), and there is no guarantee that these views, estimates, opinions or predictions are currently accurate or that they will be ultimately realized. To the extent these assumptions or models are not correct or circumstances change, the actual performance may vary substantially from, and be less than, the estimates included herein. None of Galaxy Digital nor any of its affiliates, shareholders, partners, members, directors, officers, management, employees or representatives makes any representation or warranty, express or implied, as to the accuracy or completeness of any of the information or any other information (whether communicated in written or oral form) transmitted or made available to you. Each of the aforementioned parties expressly disclaims any and all liability relating to or resulting from the use of this information. Certain information contained herein (including financial information) has been obtained from published and non-published sources. Such information has not been independently verified by Galaxy Digital and, Galaxy Digital, does not assume responsibility for the accuracy of such information. Affiliates of Galaxy Digital may have owned or may own investments in some of the digital assets and protocols discussed in this document. Except where otherwise indicated, the information in this document is based on matters as they exist as of the date of preparation and not as of any future date, and will not be updated or otherwise revised to reflect information that subsequently becomes available, or circumstances existing or changes occurring after the date hereof. This document provides links to other Websites that we think might be of interest to you. Please note that when you click on one of these links, you may be moving to a provider’s website that is not associated with Galaxy Digital. These linked sites and their providers are not controlled by us, and we are not responsible for the contents or the proper operation of any linked site. The inclusion of any link does not imply our endorsement or our adoption of the statements therein. We encourage you to read the terms of use and privacy statements of these linked sites as their policies may differ from ours. The foregoing does not constitute a “research report” as defined by FINRA Rule 2241 or a “debt research report” as defined by FINRA Rule 2242 and was not prepared by Galaxy Digital Partners LLC. For all inquiries, please email [email protected]. ©Copyright Galaxy Digital Holdings LP 2023. All rights reserved.