Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change aggregatePublicKeys and aggregateSignatures benchmarks #72

Open
paulmillr opened this issue Dec 9, 2020 · 5 comments
Open

Change aggregatePublicKeys and aggregateSignatures benchmarks #72

paulmillr opened this issue Dec 9, 2020 · 5 comments

Comments

@paulmillr
Copy link
Contributor

How is aggregating 30 keys useful?

eth2 would be around 32k validators very soon, and probably 320k after some time. And each slot is voted by around 1k voters

I think it would make sense to benchmark 32, 1024, 32768 and 131072 public keys / signatures.

@dapplion
Copy link
Contributor

dapplion commented Dec 9, 2020

The aggregation is divided into two steps, first per committee and then a final aggregation. Maybe we could benchmark 32, 128 and 256. More than that the benchmarks would start to take a long time. Also, the point of the benchmarks is to compare the implementations between themselves, it's not imperative to benchmark Eth2.0 behavior

@paulmillr
Copy link
Contributor Author

final aggregation

Doesn't final aggregation use 32k validators (right now)?

If an implementation leaks memory, or is buggy on big amounts, then it would be shown in those.

The lib is going to be used in lodestar, isn't it? Then, it's necessary to test those cases - and benchmarking them is a good test.

the benchmarks would start to take a long time

Not quite. Aggregating 2**17 signatures takes 4 seconds with noble. If you do it 2-4 times, seems acceptable to me.

@dapplion
Copy link
Contributor

dapplion commented Dec 9, 2020

Aggregating 2**17 signatures

I'm not concerned about aggregating the signatures but creating them hahaha

With noble signing 2**17 would take 3.64 hours so doing the benchmark 2-4 times you are exceeding the maximum job time of Github actions

@wemeetagain
Copy link
Member

Currently, signature aggregation only happens at the committee level.
Attestations are aggregated within a committee (signatures aggregated by a selected "aggregator" committee member at this step) then those aggregated attestations are included in blocks.
The target committee size is 128 and the max size is 2048.

There's also batch signature verification, an optional implementation speedup (implemented here as verifyMultipleSignatures), where all signatures, pubkeys, and messages from one or several blocks could be verified at once.

@paulmillr
Copy link
Contributor Author

For creating the test vectors, those should be previously created, that's the most certain way.

So, if committee size may be 2048, I think size=2048 should be tested. And 128 should be default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants