Skip to content

Commit

Permalink
chore: add readme for indexer (#12)
Browse files Browse the repository at this point in the history
* chore: add readme for indexer
  • Loading branch information
jrwbabylonlab authored Oct 14, 2024
1 parent fa87df6 commit 742093c
Show file tree
Hide file tree
Showing 5 changed files with 99 additions and 9 deletions.
92 changes: 91 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,91 @@
# babylon-staking-indexer
# Babylon BTC Staking Indexer

## Overview

The **Babylon BTC Staking Indexer** is a core component of the
Babylon blockchain’s staking architecture, responsible for syncing delegation
and finality provider events from both the **Babylon blockchain (BBN)** and
**Bitcoin (BTC)**. It processes, stores, and transforms on-chain events into
API-friendly data structures, enabling the Babylon Staking API to efficiently
serve staking-related data to the frontend.

### Key Responsibilities

- **Delegation Sync**: Syncs delegation and Babylon&BTC-related events, storing them in
MongoDB for easy retrieval.
- **Finality Provider Sync**: Tracks and updates the state of finality providers
(FPs), including status changes, creation, and edits.
- **Global Parameters Sync**: Syncs global parameters necessary for the staking
mechanism.
- **Event Replay(pending)**: Allows manual operations like event replays, triggered by
the Admin Portal or CLI, to recover or adjust state after chain forks or re-orgs.

## Architecture

The Babylon Indexer interacts with the following components:

- **BBN (Babylon Blockchain)**: Receives delegation and finality provider events
via Cosmos CometBFT `/block_results` (over gRPC for better performance), as well
as websocket subscription on events.
- **BTC (Bitcoin)**: Syncs withdrawal transactions and other BTC-related events.
- **MongoDB**: Serves as the storage layer where delegation, global parameters
and finality provider data is stored.
- **API Event Queue**: The indexer pushes API-related events into a queue
(RabbitMQ), consumed by the Babylon API for frontend-facing operations.
- **Admin Portal/CLI**: Provides interfaces for triggering event replays and
other manual interactions with the indexer.
- **Data Transformation Service (Optional)**: Transforms delegation data from
the indexer into other formats to backfill or migrate data for API as needed.

![Architecture Diagram](./doc/diagram.jpg)

## Synchronization Process

The workflow involves:

1. **Bootstrap Process**: The indexer starts by syncing all events from the
last processed Babylon block height to the latest height.
This is a continuous process until it catches up with the most recent block.
2. **Real-time Sync**: After catching up, the indexer subscribes to
real-time WebSocket events for ongoing synchronization.
3. **Raw Data Synchronization**: The indexer primarily handles the
synchronization of:
- **Delegation**: Storing and tracking delegation data.
- **Finality Provider**: Monitoring state changes and updates for
finality providers.
- **Global Parameters**: Syncing parameters relevant to staking, unbonding,
and slashing.

4. **RabbitMQ Messaging**: When a state change occurs in any delegation,
the indexer emits a message into RabbitMQ. This allows the Babylon API to
perform metadata and statistical calculations, such as total value locked (TVL)
computations.
5. **Bitcoin Node Sync**: The indexer also syncs with the Bitcoin node to
check if delegations are in a withdrawn state, ensuring accurate tracking of
withdrawal transactions.

## Installation & Setup

### Requirements

- **Go**: Version `1.23.2` or higher is required.
- **MongoDB**: A MongoDB instance with replica sets enabled is required

1. Clone the repository

```bash
git clone git@github.com:babylonlabs-io/babylon-staking-indexer.git
cd babylon-staking-indexer
```

2. Install dependencies

```bash
go mod tidy
```

3. Run the service

```bash
make run-local
```
10 changes: 5 additions & 5 deletions bin/local-startup.sh
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
#!/bin/bash

# Check if the MongoDB container is already running
MONGO_CONTAINER_NAME="mongodb"
MONGO_CONTAINER_NAME="indexer-mongodb"
if [ $(docker ps -q -f name=^/${MONGO_CONTAINER_NAME}$) ]; then
echo "MongoDB container already running. Skipping MongoDB startup."
echo "Indexer mongoDB container already running. Skipping MongoDB startup."
else
echo "Starting MongoDB"
# Start MongoDB
docker compose up -d mongodb
echo "Starting indexer mongoDB"
# Start indexer mongoDB
docker compose up -d indexer-mongodb
fi

# Check if the RabbitMQ container is already running
Expand Down
2 changes: 1 addition & 1 deletion config/config-docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ poller:
db:
username: root
password: example
address: "mongodb://localhost:27017"
address: "mongodb://indexer-mongodb:27017/?directConnection=true"
db-name: babylon-staking-indexer
btc:
endpoint: localhost:18332
Expand Down
Binary file added doc/diagram.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ services:
- ./config/config-docker.yml:/home/babylon-staking-indexer/config.yml:Z
mongodb:
image: mongo:latest
container_name: mongodb
hostname: mongodb
container_name: indexer-mongodb
hostname: indexer-mongodb
ports:
- "27017:27017"
environment:
Expand Down

0 comments on commit 742093c

Please sign in to comment.