Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(permissionless batches): batch production toolkit and operator recovery #1555

Draft
wants to merge 26 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
dca69ce
implement first steps for minimal recovery to permissionlessly produc…
jonastheis Oct 23, 2024
2e09118
Merge remote-tracking branch 'origin/develop' into jt/permissionless-…
jonastheis Oct 24, 2024
496314f
add config for recovery mode
jonastheis Oct 24, 2024
c12d380
structure code and implement restoreMinimalPreviousState and fetchL2B…
jonastheis Oct 24, 2024
c329959
produce chunks from specified L2 blocks and batch from chunks
jonastheis Oct 24, 2024
71f240b
start implementation of restoring full previous state for relayer
jonastheis Oct 25, 2024
acc7083
implement processFinalizedBatch
jonastheis Oct 28, 2024
59ea991
handle batches
jonastheis Oct 29, 2024
2df07a9
introduce ForceL1MessageCount to config to be able to set a custom L1…
jonastheis Nov 6, 2024
0f5ebf3
add Dockerfile for relayer in permissionless batches mode
jonastheis Nov 6, 2024
f96af8e
add docker compose file to spin up all necessary services together
jonastheis Nov 6, 2024
16a471d
move docker file to build/dockerfiles
jonastheis Nov 19, 2024
d25094b
add coordinator-cron and proving-service-bundle
jonastheis Nov 19, 2024
603feed
add bundle creation
jonastheis Nov 19, 2024
5ddd6d6
add permissionless-batches/conf/ to dockerignore files
jonastheis Nov 20, 2024
5ff6fd0
refactor cmd/permissionless_batches/app
jonastheis Nov 20, 2024
0108873
implement RecoveryNeeded functionality to allow re-running without ov…
jonastheis Nov 20, 2024
a6f914a
refactor cmd/rollup_relayer/app and split into FullRecovery struct
jonastheis Nov 20, 2024
0123502
clean up
jonastheis Nov 21, 2024
596d9fe
introduce profiles to docker-compose.yml
jonastheis Nov 21, 2024
d85bdf5
initial instructions in README.md
jonastheis Nov 21, 2024
a9eac08
address review comments
jonastheis Nov 22, 2024
606162e
add dummy configuration and documentation for permissionless batch pr…
jonastheis Nov 25, 2024
30c0201
ignore /conf folder from git
jonastheis Nov 25, 2024
ec9d862
add documentation for operator recovery
jonastheis Nov 26, 2024
6ef4775
address review comments
jonastheis Nov 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions go.work
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@ use (
./database
./rollup
./tests/integration-test
//../go-ethereum
)
249 changes: 244 additions & 5 deletions go.work.sum

Large diffs are not rendered by default.

79 changes: 79 additions & 0 deletions permissionless-batches/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
name: permissionless-batches

services:
relayer:
build:
context: ../
dockerfile: permissionless-batches/recovery_permissionless_batches.Dockerfile
container_name: permissionless-batches-relayer
volumes:
- ./conf/relayer:/app/conf
command: "--config /app/conf/config.json"
depends_on:
db:
condition: service_healthy

db:
image: postgres:17.0
environment:
POSTGRES_HOST_AUTH_METHOD: trust
POSTGRES_USER: postgres
POSTGRES_DB: scroll
healthcheck:
test: [ "CMD-SHELL", "pg_isready -U postgres" ]
interval: 1s
timeout: 1s
retries: 10
volumes:
- db_data:/var/lib/postgresql/data
ports:
- "5432:5432"

coordinator:
build:
context: ../
dockerfile: build/dockerfiles/coordinator-api.Dockerfile
volumes:
- ./conf/coordinator/:/app/conf
command: "--config /app/conf/config.json --http.port 8390 --verbosity 5"
depends_on:
db:
condition: service_healthy
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8390/coordinator/v1/challenge"]
interval: 1s
timeout: 1s
retries: 10
start_period: 5m

proving-service-chunk:
image: scrolltech/sdk-cloud-prover:sindri-v0.0.4
platform: linux/amd64
command: "--config /app/config.json"
environment:
PROVER_NAME_PREFIX: "sindri_chunk"
CIRCUIT_TYPE: 1 # 1 for chunk proving
RUST_BACKTRACE: 1
volumes:
- ./conf/proving-service/chunk/:/app/keys
- ./conf/proving-service/config.json:/app/config.json
depends_on:
coordinator:
condition: service_healthy

proving-service-batch:
image: scrolltech/sdk-cloud-prover:sindri-v0.0.4
platform: linux/amd64
command: "--config /app/config.json"
environment:
PROVER_NAME_PREFIX: "sindri_batch"
CIRCUIT_TYPE: 2 # 2 for chunk proving
volumes:
- ./conf/proving-service/batch/:/app/keys
- ./conf/proving-service/config.json:/app/config.json
depends_on:
coordinator:
condition: service_healthy

volumes:
db_data:
30 changes: 30 additions & 0 deletions permissionless-batches/recovery_permissionless_batches.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Download Go dependencies
FROM scrolltech/go-rust-builder:go-1.21-rust-nightly-2023-12-03 as base

WORKDIR /src
COPY go.work* ./
COPY ./rollup/go.* ./rollup/
COPY ./common/go.* ./common/
COPY ./coordinator/go.* ./coordinator/
COPY ./database/go.* ./database/
COPY ./tests/integration-test/go.* ./tests/integration-test/
COPY ./bridge-history-api/go.* ./bridge-history-api/
RUN go mod download -x

# Build rollup_relayer
FROM base as builder

RUN --mount=target=. \
--mount=type=cache,target=/root/.cache/go-build \
cd /src/rollup/cmd/permissionless_batches/ && CGO_LDFLAGS="-ldl" go build -v -p 4 -o /bin/rollup_relayer

# Pull rollup_relayer into a second stage deploy ubuntu container
FROM ubuntu:20.04

RUN apt update && apt install vim netcat-openbsd net-tools curl ca-certificates -y

ENV CGO_LDFLAGS="-ldl"

COPY --from=builder /bin/rollup_relayer /bin/
WORKDIR /app
ENTRYPOINT ["rollup_relayer"]
Loading