Roundup #5 | Ethereum Foundation Blog

Development has steadily continued over the last month and a half as we approach the launch of Metropolis. Over a series of core dev calls over the last few months, we have specified and finalized the EIPs for Metropolis, and made the appropriate changes to the Yellow Paper. Metropolis has now been split up into two consecutive forks, named “Byzantium” and “Constantinople”. EIPs for Byzantium (Metropolis part one) are essentially finalized, and the last remaining work to be done has to do with writing tests and making sure that all clients are passing all tests.

The Ethereum network continues to see all-time highs in transaction usage, and the network has maintained stability through this period with consistently low uncle rates.

Research

Work on the Casper implementation has continued, and the Casper hybrid fork choice rule is now fully passing its initial test suite. Next steps include further testing, and integration into pyethapp.
Work on a sharding implementation in python has started in full force at the sharding repo, as we have added several new developers over the past few months to help with this. See the docs folder for a partial specification of the roadmap.
Work has started on a “testing language” which can be used to quickly write and run tests for proof of work, Casper and sharding fork choice rules. This should substantially improve coverage and accelerate testing for both Casper and sharding.
A co-authored Plasma whitepaper has been released. Multiple groups in the community have already expressed interest in creating implementations.
The research documents have been reorganized somewhat. See papers at https://github.com/ethereum/research/tree/master/papers and the problems list at https://github.com/ethereum/research/wiki/Problems.

Geth

Implemented the following:

Currently working on:

Mist

Mist has surpassed two million cumulative downloads for all versions. Version 0.8.10 (of both Mist and Wallet) have had over 830k downloads.
Mist 0.9 has been released, with some major features including support for Swarm upload and ENS supported urls.
The Ethereum Wallet has also been updated to support ENS on all address fields, including send and contract execution address fields. Tokens are easier to subscribe to by simply typing the three letter symbol.
The Mist repo now has over 800 open issues, including 700 new issues in the last few months. We are working on restructuring how to handle issues, as many of them are not individual issues but are requests for tech support.
Mist research has advanced, and now enables dapp development within a restricted subset of javascript that would allow running mist in a browser. We are working on releasing more details and a proof of concept in the coming month.

ENS

The weekend of August 11-13 in London was the first community-driven ENS workshop, where representatives of the community at large including representatives of multiple wallets, individuals working on ethereum projects and other ENS stakeholders discussed the current state and future of ENS. A summary post is available here, but here are some highlights:

The community is interested in seeing the next version of the ENS registrar focus on providing incentives to release unused domains, while minimising costs for users of ENS.
The community set priorities of creating a dispute resolution system as a voluntary second layer solution, and providing safeguards from impersonators and other security features to be built into multiple voluntary second-layer solutions.
There is a consensus on how domain owners can give safeguards to subdomain owners.
The community felt that some work should be attempted to improve the efficiencies of the domain reselling process.

Web3.js

In the last two weeks, web3.js beta was released and immediately and widely used by the community. A few packages already started integrating the new version into their libraries. Since the first beta release, issues were found and fixed and improvements have been added so that the beta is currently at beta.18.

Once web3.js 1.0 is ready for prime time, a separate blog post will introduce its new features.

To try it out yourself, either clone the 1.0 branch or install it via npm install web3@1.0.0-beta.18

Please add issues and suggested improvements to the GitHub issues.

Light Client

New LES/2 protocol version is being prepared.
Ultra-fast log history search has been implemented (needs LES/2 to work in light client mode).

C++

Metropolis changes are being finalized, and new precompiled contracts are being benchmarked.
The Hunter C++ package manager has been integrated into the project to increase portability.
A prototype of state snapshot support for C++ client is being prepared.
The ethminer was removed from the cpp-ethereum project. The new ethminer repo was created for the mining community. We are helping with the maintenance of the code, and building and distributing the binaries. The miner itself has been improved, and can switch work packages much more quickly.

Solidity

We added support for “experimental” features that can be switched on via a “pragma” statement in the source code. One of these features is the new ABI encoder, which can encode arbitrarily nested arrays! Another experimental feature that will soon be available is the automated verification of assertions and overflow conditions using an SMT solver. We also fixed two bugs in the code generator. Apart from that, we implemented many smaller changes that improve error messages, added new warnings and improved security. Lastly, work has begun on implementing view and pure function specifiers.

Remix

We are currently pushing a lot of refactoring and UI/UX changes.
In the near future, the master branch will be deployed to https://yann300.github.io/remix instead of remix.ethereum.org, allowing us to fix bugs and polish everything before pushing back to remix.ethereum.org.
Important note: Please do not use it for your normal development work as it isn’t stable yet. And feel free to comment it – https://gitter.im/ethereum/remix.

EVMJIT

Metropolis support was added to EVMJIT; all tests were passed. Also EVM-C has been updated to support new features for Metropolis, like support for REVERT and return buffers. The EVM-C interface is evolving slowly, but only VM trace support is missing to mark it as fully functional.

Swarm

While the swarm testnet has migrated to new hardware, the team is busy churning out the new features for our POC3 roadmap planned for after Metropolis. The team has been working on a devp2p network simulation and testing framework for the past nine months, and the module is now ready to merge to go-ethereum. PSS (bzz whispered), our new node-to-node messaging protocol, now fully incorporates whisper’s encryption module and supports remote peer sessions with Diffie-Helmann key exchange for udp-like protocols. The biggest component of our forthcoming release is the complete rewrite of the swarm network layer. As part of this endeavor, the connectivity (kademlia overlay topology) and the discovery (topology bootstrapping) components are ready to merge.

The final missing piece for a proper dropbox backend (to complement FUSE and ENS) is

chunk-level encryption for privacy. We came up with an elegant solution: the Keccak-Feistel blockcypher, which will be included in POC3.

The team has started working on the third orange paper describing generalised swap, swear and swindle games (courtroom contract suite) to drive decentralised services by providing scalable payment, incentivisation and service guarantees. Videos of talks from the Berlin Swarm Summit in June are available on the summit website on swarm http://swarm-gateways.net/bzz:/summit2017.ethersphere.eth/.

Whisper

Version 5 released.
Proposal for version 6: discussion in progress (EIP#627).

Ewasm

Ewasm itself is close to finished in its current form. Work on Hera, an implementation of Ewasm that is compatible with the VM API using in Ethereum C++, is being resumed since EVM-C interfaces matured significantly.

EthereumJS

Issues with handling large numbers and certain edge cases have been identified and fixed. Implementation of Byzantium changes is ongoing. The most challenging area is the elliptic curve pairing precompiles. To support these precompiles, we will use Zcash’s Rust implementation compiled to asm.js. There is also work being done to support the new tracing format and optimise the VM for speed.

EthereumJ

A team of two developers has started working on bringing EthereumJ up to compatibility with Byzantium in preparation for the first Metropolis hard fork.

Pyethereum

Some bug fixes have been made as well as updated to related libraries (eg. rlp.hex_decode and rlp.hex_encode in pyrlp). Metropolis EIPs are essentially implemented, and Byzantium state and block test compliance are continuing to improve.

Py-EVM

Py-EVM is a new implementation of the EVM written in python. The library is heavily influenced by the existing Pyethereum library, with the key difference being a more modular architecture.

Recent development includes:

Expansion of the test suite beyond the JSON-fixture based tests.
Implementation of EIP150 for rules.
Implementation of the DAO Fork rules.
Support for pluggable ECC signing backends.
Support for pluggable database backends.

Web3.py

Recent improvements to the library include:

New Middleware API.
Bug fixes related to string encodings.
Better error messaging for contract calls against non-existent contracts.
Validation of contract ABI.
Validation of ERC55 checksummed addresses.

Security

We have received numerous bug bounty submissions, and have awarded Juno Im with 5000 points for a Mist-vulnerability. A Solidity-bug regarding ecRecover has been patched (see https://github.com/ethereum/solidity/releases/tag/v0.4.14).

We are continuing to improving testing, using Hive and as well as fuzzing the various raw VMs, with internal state inspection after each opcode. See Testing (below).

New opcodes and precompiles in Metropolis are being benchmarked for finalization of gas costs.

Testing

We made a number of infrastructure improvements:

General Blockchain tests execute blockchain test cases on all fork rules.
General State Tests are convertible into blockchain tests in order to run on hive.
Blockchain tests can now be generated 20% faster.

Work in progress includes:

The consensus test suite ethereum/tests is almost up-to-date with the Metropolis EIPs. Only the two most recent changes remain to be applied (#684 preventing overwriting contract/contract collision and #649 Metropolis difficulty bomb).

Test format validator is finding many mistakes in json tests files, gradually applying looser rules to stricter rules.

We are also exploring more advanced techniques and closer collaboration with the community:

Fuzz tests research has progressed.

Random test generation in progress is already discovering issues between clients.
The test contributor program has resulted in community contribution to the tests repository.

Viper

Viper has seen substantial progress over the last month and a half, and contributions from outside contributors are increasing. New security features such as payable and internal modifiers have been added, along with more tests, and the language now also has support for accepting and returning fixed-size lists as inputs and outputs.

See the https://github.com/ethereum/viper/blob/master/README.md and commit history for a changelog.

Bamboo

Bamboo is a programming language for Ethereum contracts. At the end of July, the first release of Bamboo became available in the OCaml package manager opam install bamboo. Bamboo is now capable of implementing a simple payment channel and an ERC20 token. Moreover, a few contributors have started working on both enhancements and as well as documentation.

Eth-Isabelle

Eth-Isabelle specifies the Ethereum Virtual Machine for theorem provers such as Isabelle/HOL and Coq. The project received substantive pull-requests. Separation logic tactics from @seed shortens many proofs by half. The proof that a non-owner cannot harm a wallet is more structured than ever. An independent JULIA interpreter by @mrsmkl serves as an executable specification of JULIA (an intermediate language in the Solidity compiler). It would be fruitful to test this interpreter against the implementation. This interpreter can also be used as a basis for a proven-correct compiler from JULIA to EVM.