With the arrival of the beacon chain in 2020, Ethereum today supports two types of clients: those focusing on the execution-layer (often referred to as eth1) and those powering the proof-of-stake consensus layer (aka eth2). As those running validators are well aware (since they run both client types), each serves a specific purpose, and powers a different part of one Ethereum.

Recognizing the importance that both client sets represent to the long term health and network diversity of Ethereum, our work to support the full landscape of clients is fundamental to our work in the ecosystem.

Layer-specific client support updates

the execution-layer

In 2021, the Ethereum Foundation plans to allocate at least $4 million USD to execution-layer client teams.

These grants, the first of which have been distributed, support the following client teams: Besu, Geth (Go Ethereum), Nethermind, Open Ethereum, and Turbo Geth.

It was clear even in Ethereum’s early days that greater client diversity helped to strengthen Ethereum at its core. That’s why we’re reaffirming our support for their work with this first step toward sustainability today.

the consensus-layer

In 2021, the EF is allocating at minimum $7 million USD to consensus-layer clients. This is an increase from our funding in 2020, where about$4.5 million USD was spent on client teams, as detailed in previous posts (ESP 2020 Allocation Updates, Q1, Q2, Q3, Q4). Additionally, support may be announced in periodic research and ecosystem support program updates over the course of the year for audits, formal verification, testing, bug bounties, tooling and other community support.

The consensus-layer today has four clients, Lighthouse, Teku, Prysm and Nimbus that are online, and which have been stable since the launch of the beacon chain in December. At the time of this post, these clients have enabled the beacon chain to be secured by over 3.5mm Ether, which are supplied by over 110,000 active validators. This feat, and the accomplishments of these teams to date can not be overstated.

Additionally, a fifth client, Lodestar, has provided quality javascript tooling (which is already used by various explorers and other beacon chain tracking tools) and other API standardization, and aims to bring beacon chain light-clients to production this year!

Our support in 2021 will be primarily geared toward these five clients, with other opportunities possible as development progresses.

Moving forward together

As Danny Ryan explained in his recent Finalized #23 post, “as a beacon chain staker, you need an Ethereum PoW endpoint to successfully perform all of your various duties as a validator”. This means that today and after the merge, both client types are required to power the network. And because the execution-layer’s familiar technology remains regardless of changes to consensus, there is no action required by dapps, developers or users to continue using Ethereum as they always have. In short, both client types remain necessary, and the long-term strength, health and decentralization of Ethereum succeeds as they succeed.

Over the last months, we’ve seen new interest and innovation on Ethereum, and more is surely on the horizon. As network use increases and Ethereum’s potential appears even more vast, the sustainable support of open source software on both layers is essential, and we look forward to providing further updates on our work in this area.

Ethereum’s diverse client ecosystem is at the foundation of all that we’re building together. This includes both execution-layer and consensus-layer clients, both of which are essential parts of Ethereum’s post-merge future.

Supporting execution-layer (formerly “Eth1”) clients remains one of the Ethereum Foundation’s highest priorities. These client teams have supported Ethereum’s growth over the past several years, and they will continue to provide critical infrastructure for the network post-merge, as Ethereum transitions to a Proof of Stake consensus system. Since January 2020, the EF has spent more than $10M on execution-layer client R&D. Our steadfast support will continue as these teams adapt and scale along with the broader community.

At the same time, we believe it is healthy for Ethereum clients to receive funding and support from a broader range of community stakeholders, reflecting the growth and diversification of the ecosystem.

Collaborative efforts, and specifically those dedicated to supporting teams and builders that maintain a diverse set of Ethereum clients, are a lasting and impactful way to benefit Ethereum in the long-term.

Building together

Today, we’re excited to announce that Compound Grants, Kraken, Lido, Synthetix, The Graph and Uniswap Grants are donating $250K each to support Ethereum execution-layer client teams. This project is the result of several months of work, and will supplement the significant funding provided by the Ethereum Foundation this year. This fundraise will have no impact on the EFs continued financial support for these teams.

The support of capable and leading entities from across industries will strengthen and further the innovations taking place on Ethereum.

This project represents an effort to secure Ethereum’s long-term growth, health and decentralization. Each of these elements can be exemplified by client diversity, strength of the teams themselves, and our confidence that Ethereum will continue to succeed as they succeed.

Why begin with this approach?

Part of our goal with this first round is to lessen risks that come from reliance on any single team or entity as we all work to maintain quality open source products in a diverse, decentralized, and sustainable way.

We continue to work towards more decentralized funding mechanisms for Ethereum’s public goods, and this is one more step that enables larger groups of participants to directly support client development and maintenance.

Recipients

With this goal of client diversity and the network’s success at heart, we are proud to announce the recipients of this round: Besu, Erigon, Geth, Nethermind, and Nimbus.

As a public good, Ethereum benefits from shared responsibility. This is true in terms of client diversity and the depth and width of the ecosystem’s ability to support itself.

Ethereum is a project with the potential to change and improve the very core of human, social and market interactions on a global scale. With groundbreaking applications and innovation on the network and ever-increasing adoption, the Ethereum community is pushing the boundaries of what’s technically and technologically feasible with each new day. We’re excited to take this next step to continue that growth, and ask that you join us in our effort to strengthen Ethereum by guaranteeing that Ethereum remains a diverse and sustainable project, and one that is maintained by many for a long time to come.

Note: this post was updated on April 4, 2022 to include a full copy of the Client Incentive Program details.

---

A diverse set of clients is key to the Ethereum network’s health and decentralization. Diversity ensures that innovation continues at the base layer of the protocol, that the network is resilient in the face of potential attacks or bugs, and that a broad set of participants are engaged in debating potential changes to core protocol.

While clients provide an essential service to the network (without them, there is no network!), it has historically been difficult for them to capture value. Recently, more avenues have become available for these teams to build sustainable businesses, but most of those focus on mainnet-adjacent opportunities rather than the main Ethereum network. Additionally, these opportunities generally do not scale proportionally to the amount of value created.

To ensure that client teams have a strong incentive to maintain the core Ethereum network over the long term, the Ethereum Foundation has launched a Client Incentive Program. This program offers client teams ETH-denominated rewards which unlock over time, as long as they continue to build software which meets the performance and security requirements of mainnet.

Specifically, teams in the program will receive a total of 144 validators (4608 ETH) each to operate on mainnet. The size of these grants recognizes both the excellent work performed over the past few years and the many development challenges expected well into the future. One team, whose client is more recently mainnet compatible than their peers, has been included in the program with a 50% stake. The teams eligible for the program are, alphabetically:

• Erigon
• Go-ethereum (geth)
• Hyperledger Besu
• Lighthouse
• Lodestar (50% stake)
• Nethermind
• Nimbus
• Prysm
• Teku

The validator deposits are made up-front to be operated by teams immediately, while the withdrawal credentials (the ownership of the funds) will be vested over several years, with the first tranche unlocked at the delivery of Beacon Chain withdrawals. In order to receive this and subsequent tranches of validator withdrawal credentials, teams must continue to maintain their clients, meet performance benchmarks on mainnet, and generally contribute toward delivering the Ethereum community’s roadmap, as it evolves over time. After The Merge, client teams will also receive transactionm fees collected by their validators. This, along with staking rewards, will begin to provide a steady source of revenue to teams.

As the grants vest, teams are free to do what they please with the validators they control – e.g. continue to stake and earn rewards, withdraw and liquidate, or some combination of the two. Also note, the Client Incentive Program is in addition to any grants that the EF provides to these teams.

A full copy of the program’s details has been included as an appendix below.

Geth’s participation in this program is unique, since they are a team housed within the Ethereum Foundation. However, the Geth team – like the other clients listed above – will have complete discretion over how to use these validators, earned fees, and their ETH deposits as the grants vest.

The structure of the program aligns teams with the long term health of the network and ensures they are incentivized to build secure and performant software. It was designed to be backwards-looking and reward teams who have already delivered production-quality software. We hope that it provides a foundation for a healthy incentivization of core contributors to Ethereum. As always, the Ecosystem Support Program is available, and eager, to fund earlier innovative Ethereum implementation efforts including new client teams.

We are excited to finally share this initiative publicly, and we look forward to seeing more ways for the community to come together and support public goods!

---

Client Incentive Program Details

Given the aggregate total of ETH that is planned to be distributed to client teams (about 42,000 ETH when considering validator rewards, or, as of April 4, 2022, over $145MM in value), we recognize the community’s interest in learning more about how distributions will take place, and how milestones will be met.

The full details of the incentive program, as shared with client teams, are below.

Program Goals & Eligibility

The program aims to provide long-term support and incentives for teams towards maintaining reliable clients and a healthy network overall.

For client teams to be eligible, they should already be contributing to the general development of Ethereum and intend to support the upcoming transition to proof of stake. Throughout the program, teams will need to maintain certain levels of performance to be eligible for the rewards. More on this below.

Configuration

Structure

The following are the high level steps performed by “EF” and the “Client” through the life of this plan.

1. Make deposits
2. Transfer control of active signing keys
3. Client operate nodes/validators
4. Release withdrawal credentials in waves

1. Make deposits

After a client has agreed to join the program, the EF creates NUM_VALIDATORS 32-ETH deposits.

Total ETH at stake in the client incentivization plan is equal to NUM_VALIDATORS * 32.

In consultation with client teams, a formal start date for this program will be determined where teams will gain control of validators, approximately between October 1, 2021 and whenever The Merge occurs.

2. Transfer control of active signing keys

After step 1, there will be NUM_VALIDATORS privkeys mapping to the pubkeys in the validator deposits controlled by a single mnemonic. These keys must be securely transferred to the client team.

This mnemonic is transfered to the Client via one of the following:

1. Using asymmetric encryption (e.g. PGP) via a known/validated public key of the recipient Client
2. Read verbally 25% at a time over 4 encrypted calls of various platforms
3. Through an alternatively negotiated, secure means

The Client then generates NUM_VALIDATORS keystores using the mnemonic and verifies that each privkey maps sequentially to the batch of validator pubkey deposits made in their name.

The EF retains the mnemonic in cold storage in the event that active keys must be used to exit validators from the program.

3. Client operates nodes/validators

Deposits are made; keys are transferred. Now, the Client is in charge of the management of the associated validators until withdrawal credential privkeys are released. Specifically, the Client must use their own software as an execution-engine or consensus layer and is responsible for choosing and maintaining support for a counterpart client throughout the incentivization period.

Performance of the Client’s validators can be assessed simply by viewing chain metrics, but additional node performance metrics might be requested.

4. Release sets of withdrawal credentials upon meeting milestones

Waves of validators are to be released to the Client upon meeting pre-defined milestones via the transfer of the underlying privkeys for the validator withdrawal credentials.

When a wave of validators is released, this ends the obligation of the Client to the EF for those validators. The Client is free to choose to continue to validate, to exit, to withdrawal, etc.

These keys will be pgp encrypted and transferred in batches.

Milestones

Due to the dynamic nature of the ever evolving Ethereum roadmap, simplicity is favored in the choice of milestones.

A wave of credentials are released when withdrawals from the beacon chain are enabled, with a minimum period of one year between the launch of the Client Incentive Program (CIP) and the complete release of the first wave of credentials.

If withdrawals from the beacon chain are enabled within or before the first year of the CIP, the first wave of credentials will be released monthly, in equal tranches, from the first month after withdrawals are enabled, to the one year mark of the program. For example, if withdrawals are enabled 6 months after the start of the program, then 1/6th of the first tranche will be released on months 6, 7, 8, 9, 10, 11 and 12. Otherwise, the first wave of credentials will be released when withdrawals are enabled. Subsequent waves are released over time if the Client continues to meet expectations. Specifically, the milestones are as follows:

1. Release INITIAL_RELEASE validators at the time at which withdrawals from the beacon chain are enabled (WITHDRAWALS_ENABLED_TIME).
2. for i, num_validators in enumerate(TIMED_RELEASES), release num_validators validators at time WITHDRAWALS_ENABLED_TIME + (i + 1) * 6_months if client operation continues to exhibit successful metrics.

Success metrics

Client/validator performance must consistently meet a set of success metrics to continue participation in this program.

The first NUM_PERFORMANCE validators of the deposited validators are tracked by the EF to assess metrics. The last NUM_CANARIES validators of the deposited validators are free to be used by the Client for testing, experimental releases, etc. Canary validators are not expected to constantly meet the success metrics but are still subject to slashing rules.

Metrics

The following are the success metrics that the Client must meet:

• Client validators on average do not drop below MIN_ACCEPTABLE_BALANCE balance
• Client validators have at least MIN_ATTESTATION_PERCENTAGE percentage of expected attestations included on chain over any METRICS_WINDOW epoch period
• Client validators have at least MIN_BLOCK_PERCENTAGE percentage of expected blocks included on chain over any METRICS_WINDOW epoch period

Moreover, client teams are expected to actively participation in research and development of crucial network upgrades. The EF is solely responsible for determining whether this metric has been met.

Above all else, the EF expect client teams to actively work toward ensuring a robust and healthy network. The EF recognizes that in some scenarios these metrics are not entirely in the control of the Client (e.g. large portion of the network offline for an extended period of time due to issues with another client). In most such cases, the METRICS_WINDOW has been selected to be long enough to account for issues and recovery, but in such exceptional scenarios, the EF will also take into account exogenous factors outside of the Client’s control.

Note: In the context of this plan, validator top-ups are against the rules and should generally be avoided. If in some scenario a top-up would benefit the health of the network, the EF and client can discuss and reformulate the metrics/milestones accordingly.

Probation

If the Client drops below the success metrics, the Client’s incentivization status moves into probation. During a probationary period the Client has MAX_PROBATION_WINDOW epochs to get metrics back to successful standards, and during a probationary period the Client cannot have any validator credentials released. The amount of time spent in probation pushes back the release of any validator credentials by at least that amount of time.

If Client metrics remain in probationary status for more than MAX_PROBATION_WINDOW epochs, the EF can at their discretion partially or fully remove the Client from the incentivization program and partially or fully exit the Client’s validators.

Slashing

In the event that one or more of a Client’s validators is slashed, such a validator is removed from the incentive program.

If the event is relatively isolated and quickly remedied, the EF can at its sole discretion choose to place a maximum of 16 of the slashed ETH per slashed validator back into the program to be released at the final milestone.

If the slashable event is exceedingly large, negligent, or repeated, the EF can at their discretion partially or fully remove the Client from the incentivization program and partially or fully exit the Client’s validators.

Note: Performance and canary validators are both subject to the slashing rules.

Cross-Layer Dependencies

While the Client is fully responsible to ensure that their operation is run in a performant and non-slashable way, we recognize that there is a limit to what execution or consensus layer teams can do to mitigate issues on the other layer. Specifically, this means we expect the Client to adopt best practices with regards to running their nodes, but will not penalize them in the case of a widespread issue caused by the other layer. Best practices when running validators include:

• Ensuring that the Client can interoperate with most/all major clients, at least on canary validators;
• Ensuring that the Client’s failures are decorrelated from the rest of the network, both by relying on diverse clients and hosting setups;
• Ideally ensuring that the Client’s counterpart nodes are split across >1 client in case of a client-specific issue;
• Ensuring that the Client has the ability to switch from one counterpart client to another in the case of a client-specific issue.

Terms

This plan is an opt-in additional incentivization plan for clients. Participation in this plan and the amount of locked funds available in the plan will have no bearing on future client grant decisions. Clients can include a small stipend for node infrastructure in grant requests regardless of participation.

Prerequisites of participation in this plan are successful participation in multi-client testnets and generally demonstrating production readiness at all times.

In general and especially in the event of exceptional and unforeseen scenarios concerning the client, the client team, the Ethereum roadmap, and/or the Ethereum mainnet, the EF is solely responsible for deciding how to award withdrawal credentials and/or restructure the terms of this incentive plan at any time.

Such exceptional scenarios include, but are not limited to, the following:

• Separate client teams merging into one
• Client team splitting into two
• Client team ceasing the maintenence of a component (e.g. validator client) or the entirety of their software
• Ethereum roadmap radically changing such that the milestones no longer reflect achievable goals
• Ethereum mainnet has extended issues with stability, finality, or otherwise proper function
• Ethereum mainnet undergoes a contentious hardfork