Welcome to the third installment of eth2 quick update.

tldr;

• Harden fork choice defences in response to audits
• Introducing challenges.ethereum.org
• Herumi grant for a super-fast BLS implementation

Fork choice defenses

This week, we hardened the defenses of the Phase 0 fork choice rule. Much of our time is spent analyzing eth2 to ensure that the system can remain stable even under adverse conditions (e.g. a large attacker, a major network outage or partition, etc). With any large engineering project, the aim is to find as many potential issues and harden against these scenarios prior to release. This is why we’ve engaged with many external teams and researchers to perform audits, economic analysis, and formal verification of eth2, but at the same time as we move toward release, we must remain vigilant and flexible to respond to unforseen attacks in production (e.g. the Shanghai DoS attacks of 2016).

Helping in this effort, a consensus researcher, Ryuya Nakamura, has analyzed eth2’s consensus algorithm (Casper FFG) and fork choice rule (LMD GHOST) to better determine how these two pieces of the protocol can in some scenarios be exploited. He has a number of excellent write-ups on ethresearch.ch discussing in detail some attack vectors and recommended solutions. Check out Decoy-flip-flop attack on LMD GHOST, Analysis of bouncing attack on FFG, and Prevention of bounding attack on FFG for some informative and fun reads.

Fortunately, the recommended solutions to the decoy-flip-flop and bouncing attacks are both simple modifications to the fork choice specification of Phase 0. The decoy-flip-flop attack is solved by only considering validator attestations from the current and previous epoch when adding attestations to the view of the fork choice. You can see the PR making this change to the spec here. The bounce attack is solved by only updating the latest justified checkpoint within the fork choice during the first k slots of an epoch. After k slots, you defer the inclusion of a new justified checkpoint into the fork choice until the next epoch boundary. You can check out this modification here.

A number of other components of the Phase 0 specification are out for audit currently. Although we don’t expect anything major, we do expect to a small number of additional post-audit suggested changes to make it to PRs in the coming months.

challenges.ethereum.org

We just released challenges.ethereum.org. This site is a single source for all of the current bounties hosted by the Ethereum Foundation and/or related to Ethereum research and development. The bounties range from finding vulnerabilities in existing Ethereum infrastructure to finding collisions in new hash functions. Something for everyone

We will continue building out EF’s bounty program in the coming months so check back for more opportunities.

Herumi BLS grant

We’ve given out a grant to Shigeo Mitsunari, the maintainer of the super-fast Herumi pairing library and BLS signature library. This grant is to bring the libraries up to spec with the new BLS standard (new hash-to-G2, constant-time signing, etc) as well as to help accomplish some more practical items for eth2 usability (Rust integration, fuzzing work, etc).

In addition to the feature component of the grant, there is also a performance component. The Herumi library is already 2-3x faster than the next best BLS option, but because signature verification is one of the main bottlenecks in eth2, any additional gains here would be excellent, enabling eth2 to run under even higher load and in more adverse scenarios. Shigeo will spend some additional time and effort further optimizing the library to ensure it runs as fast as possible .

Welcome to the fourth installment of eth2 quick update. There are a lot of moving pieces to talk about this week. Other than the heroic eth2 client development going on, these are the highlights:

tldr;

Differential fuzzing grant

Sigma Prime has been awarded a grant to lead the differential fuzzing effort for eth2 clients. This effort is critical to the success of launching a multi-client network by aiding in catching consensus issues prior to mainnet.

The act of “fuzzing” is the act of throwing many random inputs at a piece of software to see how it reacts. When fuzzing a single piece of software, the goal is often to find inputs that lead to unexpected crashes. When we find such inputs, we then figure out what went wrong and harden the software to this type of input.

Differential fuzzing is a bit different. Instead of explicitly looking for crashes, we look for instances in which different implementations of a protocol have a different output for the same input. In a blockchain context, we use differential fuzzing to find cases in which a series of blocks leads to a different resulting state on two different clients. Ideally in production there are no such cases.

Light client task force

Chainsafe/Lodestar, the recipients of an Ethereum Foundation grant for research and development on eth2 light clients, has formed the Light Client Task Force. This group has tasked themselves with ensuring that light clients are first class citizens in eth2. To this end, they are hosting a monthly call aimed at driving light client research, standards, specs, and education.

The need for a rich ecosystem of light clients and light client servers is only amplified in a sharded protocol like eth2. Even if a client is syncing some subset of the protocol (e.g. just a couple of shards), a user will very often need to get information about accounts, contracts, and the general state of things on another shard. A client could inefficiently sync the entire additional shard, but more often than not, lightly requesting information about specific accounts on the shard with succinct proofs will be the way to go.

Tune in to the next Light Client Task Force call to stay up-to-date on all things light in eth2.

eth1 -> eth2

In the early days of eth2, the transfer of ether from the existing ethereum chain (eth1) into the new beacon chain (eth2) will be uni-directional. That is, the ether moved into staking on eth2 will not be transferable (to start) back to eth1. The choice of a single directional transfer into validation is in an effort to minimize the risk profile that eth2 induces upon eth1, and to allow for a quicker development cycle on eth2 without having to fork eth1 in the process. There is some movement around creating a bi-directional bridge, but I’ll save discussion of the bridge mechanics and the trade-offs for a later post. Today, I’d like to get more into how this uni-directional transfer works and how it can be safely implemented without changing eth1.

On the existing ethereum PoW chain, we will deploy the eth2 validator contract. This contract has a single function called deposit which takes in a number of parameters to initialize a new validator (e.g. public key, withdrawal credentials, an ETH deposit, etc). There is no withdrawal function on this contract. Barring a fork to add in a bi-directional bridge, this deposited ETH now only exists in eth2 on the beacon chain.

It is then the validators’ responsibility on the beacon chain to come to consensus on the state of this contract such that new deposits can be processed. This is done by eth2 block proposers embedding recent eth1 data into a beacon block field called eth1_data. When enough block proposers during a voting period agree on recent eth1_data, this data is enshrined in the beacon chain state allowing for new deposits to be processed.

An important note about this mechanism is that the eth1_data is deep in the eth1 PoW chain — ~1000 blocks of “follow distance”. This follow distance induces a high latency in processing new validator deposits, but provides a high degree of safety in the coupling of these two systems. The eth1 chain would have to re-org deeper than 1000 blocks to break the link, and in such a case would require some manual intervention to overcome.

We are researching and prototyping the utilization of the beacon chain to finalize eth1 (i.e. the finality gadget). This would require eth1 to defer its fork choice ultimately to the beacon chain, gaining security from the PoS validators, and allowing for a much quicker eth1 to eth2 deposits. The finality gadget also opens up other fun things such as the bi-directional bridge and exposing the eth2 data-layer to eth1. More on all of this in a later post .

Welcome to this week’s eth2 quick update!

tldr;

Shasper joins Prysmatic’s testnet

Parity’s eth2 client, Shasper, successfully joined Prysmatic’s Sapphire Testnet marking the first public multi-client eth2 testnet. This is the exciting start of many multi-client testnets to come in the next month.

You can now pull down the Shasper codebase and with a few commands, and connect to the Sapphire testnet. If you want to give it a shot, follow the instructions here.

Eth2 block explorers launch

Not one but two (!) eth2 block explorers recently launched. Both of these block explorers currently monitor Prysmatic Labs’ Sapphire Testnet, providing slot-by-slot info as validators build the beacon chain.

Bitfly launched their beaconcha.in block explorer a couple of weeks ago and continue to add exciting features by the day. Etherscan just launched their block explorer yesterday, and it, too, looks packed with cool features and data. Both will be great options to monitor Prysmatic’s and other testnets as they come online.

We’re excited to see more and more user/developer tooling being built around eth2 clients and testnets

First eth2 networking call

Up until this point, we’ve relied upon github issues/pull-requests, adhoc chats, the main eth2 call, and in person meetings to organize the networking components of the eth2 spec. This has largely worked well enough, but researching and architecting a network to support a sharded blockchain protocol is an issue that increasingly deserves some more specific attention and resources.

To this end, we had our first eth2 networking-specific call this week. Although this call is not live-streamed like the general eth2 call, it is public and open to all contributors. Thank you to the p2p networking experts across the various eth2 teams that pushed for this call. I was originally resistant due to the additional coordination overhead, but the first call proved very fruitful and I look forward the next.

As always Ben Edgington (call notes) and Mamy Ratsimbazafy (call notes) took excellent notes. We still have plenty to dig into, and our next call will be in approximately 2 weeks time.

Thank you Will Villanueva from the Quilt team for organizing the first eth2 phase2 community call. Similar to the new networking and light client calls, phase 2 has enough going on to warrant a regular call to keep the research and development more organized.

This first call acted as a general update and survey of the various on-going threads across the many teams and individuals involved. You can read up on the notes here. Subsequent calls are intended to be deeper work sessions on specific technical issues.

A major goal for Phase 2 is to get through the initial wave of stateless protocol research and to use the findings to narrow the scope into a more concrete plan to execute in 2020. These calls are an excellent step forward toward this goal.

Forkchoice state fix

Researchers at San Jose State University led by Yan X Zhang have been diligently working to formalize the joint properties of eth2’s consensus mechanics (Casper FFG) and fork choice rule (LMD GHOST). Before publishing their work, they found a corner case in the interworkings of FFG+GHOST in which a branch of the block tree might include the latest justified/finalized blocks but not actually show those blocks as justified/finalized in the on-chain state. To leave such “non-viable” branches in the fork choice can lead to situations in which a validator’s vote might not be coherent with local finality info and might, in certain scenarios, lead to liveness failures. You can read more about this particular scenario and the proposed solution here.

To address this scenario, we have a fix under review in the specs repo. We expect this fix to be released within the week.

Specification and implementation of the new BLS standards

The long-awaited BLS standards were recently presented for public comment at the IETF Meeting 106. You can check out the presentation and slides for yourself. The presentation went as planned and the standard is expected to be adopted by a number of blockchain projects and (eventually) into IETF. To more semi-officially enshrine this standard before the long IETF process finsihes, I expect the EF and many other projects to more officially announce intended usage soon.

There are two draft pull-requests ([1] [2]) under review in the specs repo, as well as an implementation of the new standard under review in py_ecc. Once review is complete, we will generate the new BLS test vectors for general consumption by eth2 clients. The intention is to switch testnets over to the modified BLS scheme come January.

The BLS standards also remove one of the final blockers for launching the eth2 deposit contract. Runtime Verification is currently finishing up their report on the formal verification and analysis of the deposit contract bytecode. This report is expected to be published by the end of the month for public review, after which we can finally launch this thing .

Thanks to Joseph Schweitzer and Danny Ryan for review.

Welcome back! Having discussed eth2’s design philosophy last time, today’s focus is on eth2’s incentives through the lens of that philosophy. More specifically, we look at the incentives effecting eth2 and how they are realised in the form of rewards, penalties, and slashings.

We then walk through how and why validators are incentivised to remain online, why you won’t be slashed for going offline, and more. Let’s dig in.

If not for being offline, when do slashings occur?

Slashing has two purposes: (1) to make it prohibitively expensive to attack eth2, and (2) to stop validators from being lazy by checking that they actually perform their duties. Slashing a validator is to destroy (a portion of) the validator’s stake if they act in a provably destructive manner. The two major ways a validator can behave slashably maliciously within eth2 phase 0 are double voting and surround voting (read the original paper for more on how Casper FFG works in detail):

Double voting is when a validator votes for two different blocks during the same epoch, which means they are signalling support for two different versions of reality. The simplest example of why this is forbidden is a validator sending transaction $a$ in block $A$ and $b$ in block $B$ where $a$ and $b$ spend the same ETH. This is the Proof of Stake version of the classic double-spend attack.

Slashing of surround votes also prevents two versions of the chain from becoming finalised by punishing validators who create votes which present multiple different versions of reality which they claim to be true at the same time. More specifically, attestations (votes for blocks) are surround votes when a validator attests to one version of reality and later attests to another version, but in a way that doesn’t make clear that they no longer believe in the first.

Double and surround voting are the only way validators can be slashed within phase 0, but additional rules are added in later phases to ensure that validators actually store and make available the shard data that they sign (which prevents validators from being lazy or from withholding information).

A validator that correctly follows the protocol never emits a slashable vote in normal operations. If not an intentionally malicious action, forming a slashable message only occurs as a result of some bug or accident. To minimise the pain of such errors, the amount of stake destroyed is proportional to the number of other validators slashed around the same time. If a small number of validators commit some slashable offence, it is unlikely that they are trying to attack eth2 because a successful attack requires many validators. Slashings that occur in small numbers are therefore assumed to be honest mistakes and are punished lightly (a minimum of 1 ETH). On the other hand if many validators commit an offence during a similar time, then a large amount of their stake is burnt (up to their full balance) as it is assumed to be an attack on the network.

Validators that are slashed are prevented from participating in the protocol further and are forcibly exited. In the case of an honest mistake, this prevents offending validators from doing further harm to themselves by being slashed again; whereas in the malign instance, this removes malicious validators from the protocol.

So what happens to validators who are offline?

Validators that are offline when they are supposed to be participating in the protocol are penalised, but in the normal case these validators only stand to lose what they would have made as rewards had they participated correctly in the protocol. This means that validators that are online > 50% of the time will still see their stake increase over time.

As a result of this mechanism, validator clients that need to go offline for maintenance etc, are usually best off if they just go offine for a short time instead of exiting and re-joining the protocol (both of which have associated delays).

This means that validators need not go to extreme lengths with backup clients or redundant internet connections as the repercussions of being offline are not so severe. In fact, any such system in which two entities can sign messages can be detrimental as primary and backup clients could end up both being online at the same time and emitting slashable votes (via the double voting mechanism explained earlier) as was the case with the first Cosmos slashing.

This regime of offline penalties holds provided that blocks are being finalised (2/3 of validators (weighted by stake) are online and their votes are being counted). This is the expected state of eth2 during normal operation. If less than 2/3 of nodes are online then something has gone catastrophically wrong in the realm of eth2. The family of consensus protocols that Eth’s Casper is a part of can no longer reach agreement under these conditions.

What does eth2 do if > 1/3 of validators are offline?

This is where the inactivity leak mentioned at the start of the article comes in. The inactivity leak reduces the balances of the offline nodes over time so that the ratio of online validators to total validators (weighted by stake) can once again exceed 2/3 so eth2 can continue to make decisions as a protocol.

Inactivity leaks are one of the ways eth2 has been designed to survive a WW3-style event. If such an event were to knock out more than 1/3 of all validators, then the offline validators would find that their balances decreased to the point that their participation was no longer needed for eth2 to continue as a chain.

Anti-correlation and decentralisation

Both the slashing mechanism and the inactivity leak encourage validators to make decisions that cause their nodes to fail in manners different to those of others. That is — to ensure the smallest possible slashings and to prevent inactivity leaks, a validator should attempt to have their clients fail in ways that are different to others’.

This places pressure on all validators to decentralise every aspect of being a validator as, for example, validators that rely on the same source of truth like Infura or use AWS to host their clients will be worse off if something goes wrong.

With all the many ways to be punished, why would a someone want to be a validator?

As stated in the first article, “validators will be lazy, take bribes, and they will try to attack the system unless they are otherwise incentivised not to.” The punishments discussed so far discourage bad behaviour, but rewards are needed to encourage validators to perform actions that benefit eth2.

There are 3 major classes of rewards:

Whistleblower rewards

A validator that raises the alarm on another validator by providing proof that gets them slashed is rewarded for their efforts in cleaning up the eth2 streets.

Proposer rewards

Validators are randomly assigned the duty of producing a block; the chosen validator is called the proposer. A proposer is rewarded for their efforts in the following ways:

• Including a proof from a wistleblower that gets a validator slashed
• Including new attestations from other validators

These rewards encourage validators to provide helpful information to the chain when they are chosen to produce a block.

Attester rewards

Attestations are votes that signal that a validator agrees with a decision in eth2. These types of messages form the basis of consensus and are rewarded in 5 different ways:

• Getting your attestation on-chain
• Agreeing with other validators about the history of the chain
• Agreeing with others about the head of the chain
• Getting your attestation on chain quickly
• Pointing to the correct block in the assigned shard

Scaling validator earnings

There are two common approaches for paying validators in PoS systems: fixed rewards and fixed inflation. In the fixed reward model, validators are paid a fixed amount for doing their jobs, and the inflation rate then depends on how many validators sign up. This has the problem of how to correctly set the reward rate. If the reward rate is set too low then too few validators will participate, while a reward rate that is too high encourages extensive validation beyond the requisite security and wastes money.

The complimentary model is one with a fixed inflation rate where some total reward is divided amongst the active validators. This model has the benefit of allowing market forces to find the right amount to pay validators as they all make individual decisions about whether or not to participate based on current earnings. There are downsides to this model. Validator earnings can be erratic making profitability decisions difficult for individual validators. This model also makes the protocol vulnerable to discouragement attacks in which validators attempt to prevent each other from participating to increase their own profit (even at their own temporary loss).

eth2 aims to have the best of both worlds by choosing a reward model in which validator rewards are proportional to the square root of the total amount of ETH staked. This hybrid model attempts to suppress variations in inflation and validator return rates while still allowing market forces to determine the correct amount to pay each validator for the security provided.

Hope for the best, but expect the worst

Each of the facets of eth2’s incentive scheme is a result of designing a protocol under the philosophy laid out in the last article. Examples of this include the anti-correlation mechanisms encouraging decentralisation and inactivity leaks helping eth2 to survive World War 3, but the main idea underpinning how the incentives work is the assumption that “validators will be lazy, take bribes, and that they will try to attack the system unless they are otherwise incentivised not to”. If someone attacks eth2 in one of the ways discussed here, they better be prepared to throw away a lot of ETH because one way or another they are going to lose it all.

Welcome to the first eth2 quick update of 2020! This is going to be an exciting year.

tldr;

• Release of v0.10.0 spec as stable target for multi-client testnets and security reviews
• @paulhauner and @sigp_io team hard at work building Lighthouse
• Relaunch of Prysm testnet, now with aggregators and mainnet configuration
• A new proposal for an expedited merging of eth1+eth2 (aka Phase 1.5)

Release of v0.10.0 for security reviews and multi-client testnets

v0.10.0 — 404 Not Found was released last week. Read the release notes for the technical details (integration of IETF BLS, simpler eth1 caching, etc), but what does it actually mean for eth2?

v0.10.0 is the culmination of changes introduced to the Phase 0 spec after the unfreezing following deVcon as well as the integration of the new IETF BLS standards. For some time, these new spec changes disrupted the smooth development cycle of the eth2 client teams and postponed the release of testnets that were imminent. With the release of v0.10.0, we are now entering back into a smooth rhythm.

New and updated resources

With the release of v0.10.0, we once again have a very stable target both for client testnets as well as third party audits and security reviews.

To this end, we recently updated a number of onboarding resources (and created a few new ones). If you are looking to better understand the eth2 Phase 0 specs, you should definitely give these a look!

Audits underway

Least Authority kicked off their security review this Monday and are currently digging deep. We have high expectations and are enthusiastic to be working with a team of such caliber!

Another exciting piece of news is in the form of an audit and formalization of the Phase 0 cryptoeconomics led by a new EF team — Robust Incentives Group (RIG).

RIG has constructed a cadCAD environment to simulate the eth2 economic models and explore a diverse range of attacks: including timing attacks, the potential effects of different sized cartels, and more!

Check out the core of their work in the form of an easy to understand python notebook.

What’s next?

While I’ll just give you a glimpse here, I promise to dive into more detail in subsequent posts.

In parallel to ongoing audits, clients will code up to v0.10.0, integrate new BLS standards, and get some steady testnets running. Once initial stability in v0.10.0 is achieved, multi-client activity will begin.

With respect to this activity, I first expect a small number of multi-client tests against largely single-client testnets. By this I mean I expect to see some clients joining what were previously single-client testnets — as minority nodes — to test initial interopability.

Once we have success here, we’ll orchestrate a shared genesis with 2+ clients from the start. I expect some level of chaos and a lot of learning in this phase. Ambiguities in the spec might become apparent with minor clarifying changes integrated. We might even find out something was broken that we previously thought was fine — large testnets are their own form of audit.

Once audit results begin to roll in throughout February, we expect to make a wave of changes to the Phase 0 spec. The extent of these changes and the implications they might have on client development and timelines is still to be determined. Regardless, a post-audit v0.11.0 release will drop near the start of March.

If the changes are small, clients will integrate the changes, patch testnets, and continue forward. If deeper, the integrations might take longer, require additional testing, and full restarts to existing nets. After sufficient time at a stable v0.11.x, a final v1.0.0 will be cut for mainnet release: the exact length of time will depend on the depth of the changes.

This is a high level look at the coming months. I’ll keep you updated and fill in more details as we go.

Lighthouse on an optimization tear

Although Sigma Prime has been relatively quiet since they took down their first public testnet, the team has been hard at work! Lighthouse lead, Paul Hauner, just dropped a exciting tweetstorm filled with juicy details of recent optimizations and gains on their eth2 client.

Apparently they’ve been running a not-so-publicized public testnet for the past 4 weeks making impressive gains on everything from BLS to block processing to DB reads/writes. Pull it down and try it yourself!

Oh and if you have some frontend chops and want to get involved, Sigma Prime just put out an RfP for a Lighthouse User Interface. This kind of work is important to ensure that validating is readily accessible to all types of users.

Prysmatic’s testnet relaunches with mainnet config and aggregators

Prysmatic Labs recently restarted their public testnet now with the mainnet configuration and the attestation aggregation strategy! As discussed in prior posts, the mainnet configuration has larger caches, longer epochs, and is generally a bit more hefty than the initially used minimal config. Operating a stable testnet with this configuration and 10s of thousands of validators is an important milestone that Prysmatic is currently working through.

If you want to get your hands dirty, become a validator or join the conversation on discord.

Oh, and they’re hiring! Working remotely with Prysm is a great opportunity for an experienced dev. Help build the future!

Phase 1.5

Over the holidays, Vitalik posted a new proposal on how we might expedite the eth1+eth2 merger and begin reaping the benefits of the new eth2 infrastructure much sooner. This proposal suggests migrating eth1 into a shard of eth2 after the Phase 1 infrastruture (shard data chains) is added but before a full Phase 2 (featuring extensible EEs and all the cross-shard bells and whistles) — thus Phase 1.5!

A phase 1.5 has many potential benefits to the ethereum protocol and to its developers and users.

1. Eth1 living inside of eth2 allows for native access to the scalable sharded data layer. Some of the most exciting constructions being built on ethereum today are layer 2 protocols that scale with the amount of data available to layer 1. These will pair beautifully with eth2 even with just a single chain with native computation (i.e. eth1 as shard). Broadly, these constructions are called “rollups”. They come in many flavors and I expect this general design space to continue to expand and be fruitful.
2. Eth1 migrating to a shard of eth2 eliminates PoW from the protocol, greatly reducing issuance and halting energy intensive mining on ethereum once and for all.
3. Finally, integrating eth1 into eth2 on an earlier schedule reduces the amount of moving parts — unifying the system, the community, and the development of the core protocol. Although the eth2 infrastructure is being developed first in parallel to the existing ethereum chain, an earlier integration of eth1 into eth2 (beyond just the technical wins) helps ensure the community of protocol developers, application developers, random contributors, and end-users remains united within a single, cohesive Ethereum.

Based upon initial discussions and reactions, developers and community members are excited by this proposal. Phase 1.5 as it currently stands largely relies upon the success of two independent components — Phase 1 of eth2 and Stateless Ethereum on eth1. The relative timelines of each of these components will inform as to how and when this proposal might come to fruition. Over the coming months, we will do due diligence on better spec’ing and understanding the technical challenges at hand so that we are prepared to move on this if/when the time is right .

Keep it coming

tldr;

Runtime Verification audit and verification of deposit contract

Runtime Verification recently completed their audit and formal verification of the eth2 deposit contract bytecode. This is a significant milestone bringing us closer to the eth2 Phase 0 mainnet. Now that this work is complete, I ask for review and comment by the community. If there are gaps or errors in the formal specification, please post an issue on the eth2 specs repo.

The formal semantics specified in the K Framework define the precise behaviors the EVM bytecode should exibit and proves that these behaviors hold. These include input validations, updates to the iterative merkle tree, logs, and more. Take a look here for a (semi)high-level discussion of what is specified, and dig in deeper here for the full formal K specification.

I want to thank Daejun Park (Runtime Verification) for leading the effort, and Martin Lundfall and Carl Beekhuizen for much feedback and review along the way.

Again, if this stuff is your cup of tea, now is the time to provide input and feedback on the formal verification — please take a look.

The word of the month is “optimization”

The past month has been all about optimizations.

Although a 10x optimization here and a 100x optimization there doesn’t feel so tangible to the Ethereum community today, this phase of development is just as important as any other in getting us to the finish line.

Beacon chain optimizations are critical

(why can’t we just max out our machines with the beacon chain)

The beacon chain — the core of eth2 — is a requisite component for the rest of the sharded system. To sync any shard — whether it be a single shard or many, a client must sync the beacon chain. Thus, to be able to run the beacon chain and a handful of shards on a consumer machine, it is paramount that the beacon chain is relatively low in resource consumption even when high validator participation (~300k+ validators).

To this end, much of the effort of eth2 client teams in the past month has been dedicated to optimizations — reducing resource requirements of phase 0, the beacon chain.

I’m pleased to report we’re seeing fantastic progress. What follows is not comprehensive, but is instead just a glimpse to give you an idea of the work.

Lighthouse runs 100k validators like a breeze

Lighthouse brought down their ~16k validator testnet a couple of weeks ago after an attestation gossip relay loop caused the nodes to essentially DoS themselves. Sigma Prime quickly patched this bug and looked to bigger and better things — i.e. a 100k validator testnet! The past two weeks have been dedicated to optimizations to make this real-world scale testnet a reality.

A goal of each progressive Lighthouse testnet is to ensure that thousands of validators can easily run on a small VPS provisioned with 2 CPUS and 8GB of RAM. Initial tests with 100k validators saw clients use a consistent 8GB of RAM, but after a few days of optimizations Paul was able to reduce this to a steady 2.5GB with some ideas to get it even lower soon. Lighthouse also made 70% gains in the hashing of state which along with BLS signature verification is proving to be the main computational bottleneck in eth2 clients.

The new Lighthouse testnet launch is imminent. Pop into their discord to follow progress

Prysmatic testnet still chugging and sync massively improved

A couple of weeks ago the current Prysm testnet celebrated their 100,000th slot with over 28k validators validating. Today, the testnet passed slot 180k and has over 35k active validators. Keeping a public testnet going while at the same time cranking out updates, optimizations, stability patches, etc is quite a feat.

There is a ton of tangible progress ongoing in Prysm. I’ve spoken with a number of validators over the past few months and from their perspective, the client continues to markedly improve. One especially exciting item is improved sync speeds. The Prysmatic team optimized their client sync from ~0.3 blocks/second to more than 20 blocks/second. This greatly improves validator UX, allowing them to connect and start contributing to the network much faster.

Another exciting addition to the Prysm testnet is alethio’s new eth2 node monitor — eth2stats.io. This is an opt-in service that allows nodes to aggregate stats in single place. This will allow us to better understand the state of testnets and ultimately eth2 mainnet.

Don’t trust me! Pull it down and try it out for yourself.

Everyone loves proto_array

The core eth2 spec frequently (knowingly) specifies expected behavior non-optimally. The spec code is instead optimized for readability of intention rather than for performance.

A spec describes correct behavior of a system, while an algorithm is a procedure for executing a specified behavior. Many different algorithms can faithfully implement the same specification. Thus the eth2 spec allows for a wide variety of different implementations of each component as client teams take into account any number of different tradeoffs (e.g. computational complexity, memory usage, implementation complexity, etc).

One such example is the fork choice — the spec used to find the head of the chain. The eth2 spec specifies the behavior using a naive algorithm to clearly show the moving parts and edge cases — e.g. how to update weights when a new attestation comes in, what to do when a new block is finalized, etc. A direct implementation of the spec algorithm would never meet the production needs of eth2. Instead, client teams must think more deeply about the computational tradeoffs in the context of their client operation and implement a more sophisticated algorithm to meet those needs.

Lucky for client teams, about 12 months ago Protolambda implemented a bunch of different fork choice algorithms, documenting the benefits and tradeoffs of each. Recently, Paul from Sigma Prime observed a major bottleneck in Lighthouse’s fork choice algorithm and went shopping for something new. He uncovered proto_array in proto’s old list.

It took some work to port proto_array to fit the most recent spec, but once integrated, proto_array proved “to run in orders of magnitude less time and perform significantly less database reads.” After the initial integration into Lighthouse, it was quickly picked up by Prysmatic as well and is available in their most recent release. With this algorithm’s clear advantages over alternatives, proto_array is quickly becoming a crowd favorite, and I fully expect to see some other teams pick it up soon!

Ongoing Phase 2 research — Quilt, eWASM, and now TXRX

Phase 2 of eth2 is the addition of state and execution into the sharded eth2 universe. Although some core principles are relatively defined (e.g. communication between shards via crosslinks and merkle proofs), the Phase 2 design landscape is still relatively wide open. Quilt (ConsenSys research team) and eWASM (EF research team) have spent much of their efforts in the past year researching and better defining this wide open design space in parallel to the ongoing work to specify and build Phases 0 and 1.

To that end, there has been a flurry of recent activity of public calls, discussions, and ethresear.ch posts. There are some great resources to help get the lay of the land. The following is just a small sample:

In addition to Quilt and eWASM, the newly formed TXRX (ConsenSys research team) are dedicating a portion of their efforts toward Phase 2 research as well, initially focusing on better understanding cross-shard transaction complexity as well as researching and prototyping possible paths for the integration of eth1 into eth2.

All of the Phase 2 R&D is a relatively green field. There is a huge opportunity here to dig deep and make an impact. Throughout this year, expect more concrete specifications as well as developer playgrounds to sink your teeth into.

Whiteblock releases libp2p gossipsub test results

This week, Whiteblock released libp2p gossipsub testing results as the culmination of a grant co-funded by ConsenSys and the Ethereum Foundation. This work aims to validate the gossipsub algorithm for the uses of eth2 and to provide insight into the boundaries of performance to aid followup tests and algorithmic enhancements.

The tl;dr is that the results of this wave of testing look solid, but further tests should be performed to better observe how message propogation scales with network size. Check out the full report detailing their methodology, topology, experiments, and results!

Stacked Spring!

This Spring is stacked with exciting conferences, hackathons, eth2 bounties, and more! There will be a group of eth2 researchers and engineers at each of these events. Please come chat! We’d love to talk to you about engineering progress, validating on testnets, what to expect this year, and anything else that might be on your mind.

Now is a great time to get involved! Many clients are in the testnet phase so there are all sorts of tools to build, experiments to run, and fun to be had.

Here is a glimpse of the many events slated to have solid eth2 representation: