Implement the EVMC interface in Nimbus

[zah]

We can consider this complete once we are able to link Nimbus into Aleth (formerly cpp-ethereum) or Geth and libaleth or evmjit into Nimbus.

Potential use cases:

1. Compare the performance of Nimbus against the other VMs.

2. Test the blockchain sync algorithm with a VM that's known to work correctly.

3. Test our VM code with a sync algorithm that is known to be correct.

As an unusual bonus feature, we can add support in Nimbus for running multiple EVMC implementations in parallel and verifying that they arrived at the same result. Something similar is recommended as a "best practice" for running public Ethereum nodes. To guard against potential security exploits, you would run two separate implementations in parallel and verify that they reach the same state after each block.

[chfast]

Do you need any help with this?

[tersec]

@chfast that would be helpful, yes. I'm not very familiar with the structure of either Aleth (which, as far as I know, claims to be EMVC-compatible) or Geth, especially in terms of how one might link with them.

[chfast]

In both cases the most convenient way is to dynamically load a shared library with a VM.

[chfast]

Do you have the EVM ready?

[tersec]

Approximately? It runs as part of Nimbus, and I adapted its message interface in https://github.com/status-im/nimbus/commit/f91bb16bdd5c1e3c70e200599c19a1e5f89569d3 to to mostly align with EVMC.

[chfast]

Does it pass all state tests?

[tersec]

Likely not. It doesn't implement CALL, in particular. I've been browsing the GeneralStateTests' code and many of them should work. I wouldn't expect full functionality immediately.

[chfast]

I think we should wait until you have the full compatible VM because it makes testing a lot easier.
If the rest of the client has better progress, you can try using aleth-interpreter (shipped with every Aleth bundle).

We probably should make a call to discuss this. There is a lot to explain and I have a lot of questions.
I'm also in Nimbus public chat room https://gitter.im/status-im/nimbus.

[tersec]

Makes sense.

Would be happy to discuss this in a call, or any other way.

[mratsim]

See also: ethereum/evmc#157 for upstreaming the Nim bindings

[axic]

I think we should wait until you have the full compatible VM because it makes testing a lot easier.

This has happened in 2019 with evmone.

Recently we've also merged Rust and Java bindings into the main tree of EVMC. So in total it supports 5 bindings: C, C++, Go, Rust, Java.

[jlokier]

For what it's worth, we're now nearly there with EVMC compatibility, in fact we're going to use EVMC as the primary internal interface when current refactoring is done. It's a well-designed interface that makes a good internal boundary as well.

We plan to use some Nimbus-specific but small extensions when available to get extra functionality, in particular to avoid recursion (for smaller stack use) and to support async-await-style blocking (i.e. no threads) when requesting data from storage. At first those will be mandatory extensions, because the priority is Nimbus functionality, and it lets us remove what will become legacy paths in/out of the EVM. While those extensions are mandatory, it won't be quite EVMC compatible, just very close.

After that, the plan is to make the extensions optional. That will be the step that makes it fully compatible and lets us interop with other EVMs and EVMC hosts. For concurrency we'll have to use threads like everyone else when doing that. To keep stack use small when there isn't a no-recursion extension, we'll probably use the same trick as Aleth, spawning a second thread when an EVM call stack gets too deep for the current one.

[axic]

@jlokier @chfast would you be interested having some brief discussion call to figure out some plans?

[jlokier]

Sure, a call would be fine. Also if you're on Discord feel free to drop in on Nimbus #nimbus-eth1 :-) It's probably the better place to arrange a call anyway; I tend to miss some GitHub comments.

You might prefer to wait until the EVMC transaction refactor is done in Nimbus, so there's a bit more experience and a bit less speculation.

[axic]

Sure, a call would be fine. Also if you're on Discord feel free to drop in on Nimbus #nimbus-eth1 :-) It's probably the better place to arrange a call anyway; I tend to miss some GitHub comments.

Huh, way too many discords 😅

You might prefer to wait until the EVMC transaction refactor is done in Nimbus, so there's a bit more experience and a bit less speculation.

That sounds good. One aspect we'd be interested in hearing if any limitation is encountered, because there are same changes done on the evmone side which suggests to us perhaps there would be room for improvement.

In case we don't set up anything earlier and/or join discord, could you leave a ping here once you feel it would be a good time to talk?

[jlokier]

It's not a bad time to talk and I like talking, so don't feel you need to wait. But there's something to be said for having a completed implementation to ground the conversation.

Pings are hard to remember but at some point I'm going to enjoy closing this issue, at which point I'll see your comment and be reminded :-)

[jangko]

We already have both EVM interface and host interface implemented for EVMC.
only need more test against other implementation. see #475
closing.