Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docker compose for one step setup with testnet #1000

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
7 changes: 6 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,9 @@ FROM base as result
# Copy the binaries
COPY --from=electrs-build /root/.cargo/bin/electrs /usr/bin/electrs

WORKDIR /
WORKDIR /electrs

# Copy config file
COPY ./docker/testnet/conf.toml ./electrs.toml

CMD ["/usr/bin/electrs", "--conf=/electrs/electrs.toml"]
32 changes: 16 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ allowing the user to keep real-time track of balances and transaction history us
Since it runs on the user's own machine, there is no need for the wallet to communicate with external Electrum servers,
thus preserving the privacy of the user's addresses and balances.


## Usage

**Please prefer to use OUR usage guide!**
Expand All @@ -28,25 +27,26 @@ If you can't use our guide, please ask about what you don't understand or consid
Note that this implementation of Electrum server is optimized for **personal/small-scale (family/friends) usage**.
It's a bad idea to run it publicly as it'd expose you to DoS and maybe also other attacks.
If you want to run a public server you may be interested in the [Blockstream fork of electrs](https://github.com/Blockstream/electrs)
which is better optimized for public usage at the cost of consuming *significantly* more resources.
which is better optimized for public usage at the cost of consuming _significantly_ more resources.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are those changes required?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, no, that's just my editor's auto-format plugin changed it while save the file. I'll remove this change, soon.


* [Installation from source](doc/install.md)
* [Pre-built binaries](doc/binaries.md) (No official binaries available but a beta repository is available for installation)
* [Configuration](doc/config.md)
* [Usage](doc/usage.md)
* [Monitoring](doc/monitoring.md)
* [Upgrading](doc/upgrading.md) - **contains information about important changes from older versions**
- [Installation from source](doc/install.md)
- [Pre-built binaries](doc/binaries.md) (No official binaries available but a beta repository is available for installation)
- [Configuration](doc/config.md)
- [Usage](doc/usage.md)
- [Monitoring](doc/monitoring.md)
- [Upgrading](doc/upgrading.md) - **contains information about important changes from older versions**
- [Docker](doc/docker.md)

## Features

* Supports Electrum protocol [v1.4](https://electrumx-spesmilo.readthedocs.io/en/latest/protocol.html)
* Maintains an index over transaction inputs and outputs, allowing fast balance queries
* Fast synchronization of the Bitcoin blockchain (~6.5 hours for ~504GB @ August 2023) using HDD storage.
* Low index storage overhead (~10%), relying on a local full node for transaction retrieval
* Efficient mempool tracker (allowing better fee [estimation](https://github.com/spesmilo/electrum/blob/59c1d03f018026ac301c4e74facfc64da8ae4708/RELEASE-NOTES#L34-L46))
* Low CPU & memory usage (after initial indexing)
* [`txindex`](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch03.asciidoc#txindex) is not required for the Bitcoin node
* Uses a single [RocksDB](https://github.com/spacejam/rust-rocksdb) database, for better consistency and crash recovery
- Supports Electrum protocol [v1.4](https://electrumx-spesmilo.readthedocs.io/en/latest/protocol.html)
- Maintains an index over transaction inputs and outputs, allowing fast balance queries
- Fast synchronization of the Bitcoin blockchain (~6.5 hours for ~504GB @ August 2023) using HDD storage.
- Low index storage overhead (~10%), relying on a local full node for transaction retrieval
- Efficient mempool tracker (allowing better fee [estimation](https://github.com/spesmilo/electrum/blob/59c1d03f018026ac301c4e74facfc64da8ae4708/RELEASE-NOTES#L34-L46))
- Low CPU & memory usage (after initial indexing)
- [`txindex`](https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch03.asciidoc#txindex) is not required for the Bitcoin node
- Uses a single [RocksDB](https://github.com/spacejam/rust-rocksdb) database, for better consistency and crash recovery

## Altcoins

Expand Down
70 changes: 70 additions & 0 deletions doc/docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Docker installation

## Docker-based installation from source

**Important**: The `Dockerfile` is provided for demonstration purposes and may
NOT be suitable for production use. The maintainers of electrs are not deeply
familiar with Docker, so you should DYOR. If you are not familiar with Docker
either it's probably be safer to NOT use it.

Note: currently Docker installation links statically

Note: health check only works if Prometheus is running on port 4224 inside
container

```bash
$ docker build -t electrs-app .
$ mkdir db
$ docker run --network host \
--volume $HOME/.bitcoin:/home/user/.bitcoin:ro \
--volume $PWD/db:/home/user/db \
--env ELECTRS_DB_DIR=/home/user/db \
--rm -i -t electrs-app
```

If not using the host-network, you probably want to expose the ports for electrs
and Prometheus like so:

```bash
$ docker run --volume $HOME/.bitcoin:/home/user/.bitcoin:ro \
--volume $PWD/db:/home/user/db \
--env ELECTRS_DB_DIR=/home/user/db \
--env ELECTRS_ELECTRUM_RPC_ADDR=0.0.0.0:50001 \
--env ELECTRS_MONITORING_ADDR=0.0.0.0:4224 \
--rm -i -t electrs-app
```

To access the server from outside Docker, add `-p 50001:50001 -p 4224:4224` but
be aware of the security risks. Good practice is to group containers that needs
access to the server inside the same Docker network and not expose the ports to
the outside world.

## Docker compose, one step for installation

**NOTE**: This is intend for one step setup with testnet/signet/regnet, inspired
by [Polar](https://github.com/jamaljsr/polar). **NOT RECOMMAND WITH MAINNET**

To setup testnet, just switch to the [docker](../docker/testnet), and then run:

```bash
docker compose up -d
```

If you are not using Docker Desktop, run:

```bash
docker-compose up -d
```

The default rpc auth info for bitcoin core is: `alice:alice`, if you want to
change it, just create a new one with [rpcauth.py](https://github.com/bitcoin/bitcoin/tree/master/share/rpcauth)
and replace both line 19 in `docker-compose.yml` and line 16 in `electrs.toml`
with new generated auth info:

```
-rpcauth=alice:128f22494cc2208ea8376a3d0b45a131$9cc0187c0e49f35454a3ed1250e40ecaef420cfcd294d3ac22496adbe64f04b9
```

Using it with bitcoin mainnet highly not recommand, use it with you own risk.
It can not start properly with mainnet by default, you need to generated new rpc
auth info and modify `docker-compose.yml` and `electrs.toml`.
37 changes: 2 additions & 35 deletions doc/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@

```bash
$ sudo apt update
$ sudo apt install -y clang cmake build-essential git cargo
$ git clone https://github.com/romanz/electrs
$ sudo apt install -y clang cmake build-essential git cargo
$ git clone https://github.com/romanz/electrs
$ cd electrs
$ cargo build --locked --release
$ ./target/release/electrs --version # should print the latest version
Expand Down Expand Up @@ -231,36 +231,3 @@ It's a bit long but sufficient! You will find the resulting binary in `target/aa
#### Generating man pages

If you installed `cfg_me` to generate man page, you can run `cfg_me man` to see it right away or `cfg_me -o electrs.1 man` to save it into a file (`electrs.1`).

## Docker-based installation from source

**Important**: The `Dockerfile` is provided for demonstration purposes and may NOT be suitable for production use.
The maintainers of electrs are not deeply familiar with Docker, so you should DYOR.
If you are not familiar with Docker either it's probably be safer to NOT use it.

Note: currently Docker installation links statically

Note: health check only works if Prometheus is running on port 4224 inside container

```bash
$ docker build -t electrs-app .
$ mkdir db
$ docker run --network host \
--volume $HOME/.bitcoin:/home/user/.bitcoin:ro \
--volume $PWD/db:/home/user/db \
--env ELECTRS_DB_DIR=/home/user/db \
--rm -i -t electrs-app
```

If not using the host-network, you probably want to expose the ports for electrs and Prometheus like so:

```bash
$ docker run --volume $HOME/.bitcoin:/home/user/.bitcoin:ro \
--volume $PWD/db:/home/user/db \
--env ELECTRS_DB_DIR=/home/user/db \
--env ELECTRS_ELECTRUM_RPC_ADDR=0.0.0.0:50001 \
--env ELECTRS_MONITORING_ADDR=0.0.0.0:4224 \
--rm -i -t electrs-app
```

To access the server from outside Docker, add `-p 50001:50001 -p 4224:4224` but be aware of the security risks. Good practice is to group containers that needs access to the server inside the same Docker network and not expose the ports to the outside world.
34 changes: 34 additions & 0 deletions docker/bitcoin/conf.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Which chain you want to use.
# Allowed values: 'bitcoin', 'testnet', 'regtest' or 'signet'
network = "bitcoin"

# The listening RPC address of bitcoind,
# json-rpc-ports:
# bitcoin: 8332
# testnet: 18332
# signnet: 38332
# regtest: 18443
daemon_rpc_addr = "bitcoind:8332"

# Using cookie file to auth
#cookie_file = "~/.bitcoin/.cookie"
# Using rpc auth instead of cookie file
auth = "<rpcauth-username>:<rpcauth-password>"

# The listening P2P address of bitcoind
# p2p-ports:
# bitcoin: 8333
# testnet: 18333
# signnet: 38333
# regtest: 18444
daemon_p2p_addr = "bitcoind:8333"

# Directory where the index should be stored. It should have at least 70GB of free space.
db_dir = "/electrs/db"

# The address on which electrs should listen. Warning: 0.0.0.0 is probably a bad idea!
# Tunneling is the recommended way to access electrs remotely.
electrum_rpc_addr = "127.0.0.1:50001"

# How much information about internal workings should electrs print. Increase before reporting a bug.
log_filters = "INFO"
44 changes: 44 additions & 0 deletions docker/bitcoin/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
version: "3"
services:
bitcoind:
container_name: bitcoind
privileged: true
image: oneforalonee/bitcoind:v26.0
restart: on-failure
hostname: bitcoind
stop_grace_period: 5m
environment:
USERID: ${USERID:-1000}
GROUPID: ${GROUPID:-1000}
command: >-
bitcoind -server=1 -chain=main
-debug=1 -zmqpubrawblock=tcp://0.0.0.0:28332
-zmqpubrawtx=tcp://0.0.0.0:28333 -zmqpubhashblock=tcp://0.0.0.0:28334
-txindex=1 -dnsseed=0 -upnp=0 -rpcbind=0.0.0.0 -rpcallowip=0.0.0.0/0
-rpcport=8332 -rest -listen=1 -listenonion=0
expose:
- 8332
ports:
- 8332:8332
volumes:
- ./bitcoind:/home/bitcoin/.bitcoin

electrs:
depends_on:
- bitcoind
links:
- bitcoind
container_name: electrs
privileged: true
image: oneforalonee/electrs:latest
restart: on-failure
hostname: electrs
command: >-
/usr/bin/electrs --conf=/electrs/electrs.toml
expose:
- 50001
ports:
- 50001:50001
volumes:
- ./electrs_db:/electrs/db
- ./conf.toml:/electrs/electrs.toml
34 changes: 34 additions & 0 deletions docker/testnet/conf.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Which chain you want to use.
# Allowed values: 'bitcoin', 'testnet', 'regtest' or 'signet'
network = "testnet"

# The listening RPC address of bitcoind,
# json-rpc-ports:
# bitcoin: 8332
# testnet: 18332
# signnet: 38332
# regtest: 18443
daemon_rpc_addr = "bitcoind:18332"

# Using cookie file to auth
#cookie_file = "~/.bitcoin/.cookie"
# Using rpc auth instead of cookie file
auth = "alice:alice"

# The listening P2P address of bitcoind
# p2p-ports:
# bitcoin: 8333
# testnet: 18333
# signnet: 38333
# regtest: 18444
daemon_p2p_addr = "bitcoind:18333"

# Directory where the index should be stored. It should have at least 70GB of free space.
db_dir = "/electrs/db"

# The address on which electrs should listen. Warning: 0.0.0.0 is probably a bad idea!
# Tunneling is the recommended way to access electrs remotely.
electrum_rpc_addr = "0.0.0.0:50002"

# How much information about internal workings should electrs print. Increase before reporting a bug.
log_filters = "INFO"
45 changes: 45 additions & 0 deletions docker/testnet/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
version: "3"
services:
oneforalone marked this conversation as resolved.
Show resolved Hide resolved
bitcoind:
container_name: bitcoind
privileged: true
image: oneforalonee/bitcoind:v26.0
restart: on-failure
hostname: bitcoind
stop_grace_period: 5m
environment:
USERID: ${USERID:-1000}
GROUPID: ${GROUPID:-1000}
command: >-
bitcoind -server=1 -testnet=1
-zmqpubrawblock=tcp://0.0.0.0:28334
-zmqpubrawtx=tcp://0.0.0.0:28335 -zmqpubhashblock=tcp://0.0.0.0:28336
-txindex=1 -rpcbind=0.0.0.0 -rpcallowip=0.0.0.0/0
-rpcport=18332 -rest -listen=1 -listenonion=0
-rpcauth=alice:128f22494cc2208ea8376a3d0b45a131$9cc0187c0e49f35454a3ed1250e40ecaef420cfcd294d3ac22496adbe64f04b9
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to not use a constant rpcauth?

Copy link
Author

@oneforalone oneforalone Mar 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm using this constant rpcauth just for easy use. Others may delete it and generate their own rpcauth via rpcauth.py, put it to ./bitcoind/bitcoin.conf and change the line auth = "alice:alice" in conf.toml to their own config.
Also, if you want to using cookie file to auth, there's a way to do it:

  1. you can remove the rpcauth config
  2. append one line in docker-compose.yml in [electrs] > volumes section: - ./cookie:/electrs/.cookie
  3. adjust the conf.toml file, i.e. comment out the rpcauth line, un-comment cookie_file and set it's value to /electrs/.cookie
  4. start with docker compose up -d, this command will only start bitcoind, but not electrs
  5. copy the bitcoind's cookie file's content to ./cookie file: cat ./bitcoind/testnet3/.cookie > cookie
  6. rebuild electrs with docker compose up -d --build --no-deps electrs

But i thought it's kind complicate, most people just want one-click to use it.

expose:
- 18332
ports:
- 18332:18332
volumes:
- ./bitcoind:/home/bitcoin/.bitcoin

electrs:
depends_on:
- bitcoind
links:
- bitcoind
container_name: electrs
privileged: true
image: oneforalonee/electrs:latest
restart: on-failure
hostname: electrs
command: >-
/usr/bin/electrs --conf=/electrs/electrs.toml
expose:
- 50002
ports:
- 50002:50002
volumes:
- ./db:/electrs/db
- ./conf.toml:/electrs/electrs.toml
Loading