Check slashability of attestations in batches to avoid sequential bottleneck

[michaelsproul]

Issue Addressed

Closes:

• Batch attestation slashability checking #1914

Proposed Changes

Sign attestations prior to checking them against the slashing protection DB. This allows us to avoid the sequential DB checks which are observed in traces here:

• Instrument attestation signing. #8508 (comment)

Additional Info

This PR builds on:

• Instrument attestation signing. #8508

This is a rework of Eitan's PR:

• Batch attestation slashibility checking #6219

I started by trying to resolve merge conflicts, but there were so many breakages I ended up redoing it. I also left out some of the other changes (like the AttestationDataService) as we are probably going to introduce a new version of that in the course of implementing the head monitor + consensus service, see:

• Enable multi node attestation consensus for the VC #8445

[michaelsproul]

Self-review complete. This is ready for review and hopefully merge 🤞

[jimmygchen]

Nice, I've done another round of review and this looks solid! Thanks

[mergify]

Merge Queue Status

🚫 The pull request has left the queue (rule: default) at 7099fda

This pull request spent 24 minutes 47 seconds in the queue, including 23 minutes 6 seconds running CI.
The checks were run on draft #8701.

Required conditions to merge

• check-success=local-testnet-success
• check-success=test-suite-success

Reason

The merge conditions cannot be satisfied due to failing checks

Hint

You may have to fix your CI before adding the pull request to the queue again.
If you update this pull request, to fix the CI, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio requeue comment.

[michaelsproul]

@mergify requeue

[mergify]

requeue

✅ The queue state of this pull request has been cleaned. It can be re-embarked automatically

[mergify]

Merge Queue Status

✅ The pull request has been merged at 7099fda

This pull request spent 1 hour 58 minutes 46 seconds in the queue, including 1 hour 57 minutes 26 seconds running CI.
The checks were run on draft #8702.

Required conditions to merge

• check-success=local-testnet-success
• check-success=test-suite-success

[jimmygchen]

Seeing improved attestation task times but worse aggregate task times - i think you noticed and mentioned this last time, just noting it down in case we want to investigate furhter next week @michaelsproul

[michaelsproul]

hmm I don't remember seeing this previously, might be worth a look