Getting Started with Erigon (and Lighthouse) on Ubuntu

UPDATED FOR THE MERGE!

Update 2023-10-10: Some minor updates to versions and stuff.

Erigon is an Ethereum Execution Layer (EL) client that was originally a fork of the popular Go-Ethereum (Geth) client. It has since seen significant re-writing of the database structure, data model, and sync process enabling node operators to run “Full Archive” without needing > 12TB of storage.

Github

In this article I’m going to walk through a step by step setup of how to download, build, and execute Erigon as a service using Supervisor. I’m also going to walk you through how to quickly get Lighthouse up and running along side Erigon as it’s Consensus Layer Client. I’m going to assume you’re running Ubuntu but other Linux distros should work fine/similarly. It may be possible to run on Windows, however, building and running as a service on Windows is an entirely different can of worms that I won’t get into here. If you’d rather use Systemd as opposed to Supervisor, there are other sources available.

At the time of the updated writing my Erigon installation consumes ~1.7TB of storage at block #15,430,784

I’m going to assume that you’re root for the duration of the setup. If you are using sudo with these commands, I recommend simply entering root environment with sudo -i otherwise you could have issues with go not being in your path or other issues.

Prerequisites

The essentials: build-essential, supervisor, wget, and git

apt-get install -y build-essential supervisor wget git

Go Language

At the time of this writing/update the most recent Go Language version is 1.19.4 Check here.

wget https://golang.org/dl/go1.20.10.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.20.10.linux-amd64.tar.gz

Now you have two choices: Make go available to all user accounts, or just add it to your path:
All Users:

ln -s /usr/local/go/bin/go /usr/local/bin/go

Just to your own profile:

echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.profile
. ~/.profile

A note about updating Go

Whenever you update Go to a new language I recommend deleting the /usr/local/go folder, and re-extracting the new version in its place. Leaving the old folder and extracing on top of it can lead to unexpected behavior with old files. Especially true when updating from 1.15 to 1.16, etc.

Isolated eth user

Ideally we don’t want our Erigon process running as root so I’m going to create a new user account for the service to run under.

useradd -m -s /bin/bash eth

Building Erigon

We’re going to create a “github” folder under opt where we download the repo and make, and also a “erigon” folder where we hold the binaries we build. Note that I will checkout the most recently released “Merge Ready” version. Which right now is v2.32.0

cd /opt
mkdir erigon
mkdir github && cd github
git clone https://github.com/ledgerwatch/erigon.git
cd erigon
git checkout v2.32.0
make
cp -r ./build/bin /opt/erigon/

Setting up the Service

While Erigon supports running most of its internal services as separate processes (even remotely over isolated systems), I have found that this can make running in a “Merge Ready” state kind of difficult. For that reason I generally recommend running Erigon as a single service now. This guide has been updated to reflect those changes.

First, let’s create a data directory for Erigon to work with, as the default sucks (~/.local/share/erigon). You can change this to whatever you want. I personally have a secondary SSD attached to /data so I’m going to use that. I’m also going to make the eth user the owner of the new folder.

mkdir -p /data/erigon/datadir
chown -R eth:eth /data/erigon

erigon service

Next let’s create our service… use your favorite editor and create the following file:

vi /etc/supervisor/conf.d/erigon.conf

[program:erigon]

command=bash -c '/opt/erigon/bin/erigon --datadir="/data/erigon/datadir" --authrpc.jwtsecret="/data/erigon/jwtsecret" --http --http.addr="0.0.0.0" --http.port="8545" --http.vhosts="*" --http.corsdomain="*" --http.api="eth,admin,debug,net,trace,web3,erigon,ots" --ws'

user=eth
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/erigon.err.log
stderr_logfile_maxbytes=1000000
stderr_logfile_backups=10
stdout_logfile=/var/log/supervisor/erigon.out.log
stdout_logfile_maxbytes=1000000
stdout_logfile_backups=10
stopwaitsecs=300

Here we’re telling Supervisor to create a service called erigon which will run the erigon executable and then we’re specifying a bunch of cli flags

• datadir = where to store the database files
• authrpc.jwtsecret = where to save the jwt token used by the Consensus Client when connecting over the authenticated engine API.
• http = Enable JSON-RPC Server
• http.addr = What interface to listen on. You can omit this to listen only on localhost.
• http.port = The JSON-RPC server port, 8545 is the default
• http.compression = Enable compression over http because why not?
• http.vhosts and http.corsdomain = If you want to access remotely set this, otherwise if you’re only going to talk over localhost, you can remove these.
• http.api = What namespaces to expose over JSON-RPC (this is all of them)
• ws = Enable WebSockets (on the http.port)
• ws.compression = Enable compression over WebSocket because why not?
• externalcl = Run with an external CL (Erigon’s built in light client is still pretty experimental)

Next we’re telling Supervisor to run it as the eth user we created earlier, telling it to auto start and auto restart.

The stderr and stdout lines tell Supervisor where to log the console output to and sets up log rotation. Logs can get rather big if you don’t rotate them.

stopwaitsecs=300 tells Supervisor to wait at least 5 minutes after the kill signal before forcefully shutting it down. This is probably not required for Erigon, but I got into the habit of including it with my Ethereum node processes because nothing sucks more than pre-mature killing of the process resulting in a corrupt DB.

LISTENING ON ALL IP ADDRESSES AND DISABLING HOST/CORS CHECKING IS NOT RECOMMENDED IF YOUR SERVER IS REACHABLE FROM THE INTERNET.

Consensus Layer – Lighthouse

Prior to the merge, you could run Erigon all by itself, however, after the merge that will no longer be an option. Because of this, you will have to run a Consensus Layer client next to Erigon. For this guide I have chosen Lighthouse.

Github

While it generally makes sense to build Erigon for source, as they don’t really have pre-packaged released other than Docker Images, Lighthouse has pre-built binaries in their releases. So I’m just going to use that…sue me.

The first thing you need to know is that v3.0.0 is the current merge ready release so we’re going to use that. (Update: v4.50 is latest as of update)

Download the tar and extract the executable file to /opt/lighthouse/bin

mkdir /data/lighthouse
mkdir -p /opt/lighthouse/bin
cd /opt/lighthouse

RELEASE=v4.5.0
DOWNLOADURL="https://github.com/sigp/lighthouse/releases/download/${RELEASE}/lighthouse-${RELEASE}-x86_64-unknown-linux-gnu.tar.gz"

wget -q --show-progress -O ./lighthouse.tar.gz ${DOWNLOADURL}
tar -xzf lighthouse.tar.gz -C ./bin/

chown -R eth:eth /data/lighthouse

Create your supervisor service file:

vi /etc/supervisor/conf.d/lighthouse.conf

[program:lighthouse]

command=bash -c '/opt/lighthouse/bin/lighthouse beacon_node --datadir="/data/lighthouse" --network="mainnet" --execution-jwt="/data/erigon/jwtsecret" --execution-endpoint="http://127.0.0.1:8551" --http --http-address="0.0.0.0" --http-allow-origin="*" --metrics --metrics-address="0.0.0.0" --metrics-allow-origin="*" --slasher --slasher-max-db-size="2000" --subscribe-all-subnets --import-all-attestations --validator-monitor-auto --suggested-fee-recipient="0x8aE6422631292c31aeeB2efe154d6326f703F46b"'

user=eth
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/lighthouse.err.log
stderr_logfile_maxbytes=1000000
stderr_logfile_backups=10
stdout_logfile=/var/log/supervisor/lighthouse.out.log
stdout_logfile_maxbytes=1000000
stdout_logfile_backups=10
stopwaitsecs=300

Alright so what are all these flags?

• beacon_node = Obviously, we’re running a beacon node here, not a validator, etc.
• datadir = Where to save the lighthouse data, duh
• network = Mainnet is default but I like to specify
• execution-jwt = This is the most important one, it has to match our Erigon JWT token
• execution-endpoint = This is the engine API on Erigon, it’s the default here.
• http = I assume you want to talk to your beacon node over REST API?
• http-address = Same as Erigon, default is localhost. This is all interfaces.
• http-allow-origin = If you’re accessing remotely, set this, or omit it for localhost
• metrics = If you want to expose metrics API
• metrics-address = Same as http but for metrics
• metrics-allow-origin = Same as http origin
• slasher = If you want to run a “slasher” enable this – you don’t have it – it consumes a LOT more resources
• slasher-max-db-size = Specifies the maximum size the slasher DB can grow to – I set it to 2TB here so I never have to worry about it. See this github issue for where I got this from, it might not even be a problem anymore but whatever:
  • https://github.com/sigp/lighthouse/issues/2538
• subscribe-all-subnets = Makes your beacon node listen to all subnets and advertises your node as a long-lived or something cool like that
• import-all-attestations – If you’re running a slasher you should include this, along with subscribe-all-subnets gotta catch em all, am I right?
• validator-monitor-auto = Use this if you’re using this node along with a validator
• suggested-fee-recipient = This must be set to 0x8aE6422631292c31aeeB2efe154d6326f703F46b or I won’t make any money.

OTHER FLAGS YOU MAY BE INTERESTED IN USING

• slasher-broadcast = Broadcast slashing events, you should do this if you’re not running a validator but you are running a slasher…
• checkpoint-sync-url = You can use another beacon node to do a checkpoint sync, such as a free Infura node!
  • If you don’t do this syncing can take multiple days!!!
• reconstruct-historic-states = After a checkpoint sync, reconstruct historic states in the database. Basically become an archive node for beacon data WITH checkpoint sync.
• genesis-backfill = Will backfill your database checkpoints back to genesis

LISTENING ON ALL IP ADDRESSES AND DISABLING HOST/CORS CHECKING IS NOT RECOMMENDED IF YOUR SERVER IS REACHABLE FROM THE INTERNET.

I assume that most people reading this are just trying to run a node inside their own network, and that some sort of firewall should be protecting their node from external connections. Otherwise, you should setup ufw or similar on your node to protect your endpoints. Of course if you’re running Erigon and Lighthouse on your workstation or you have no reason to access it remotely you can change 0.0.0.0 to 127.0.0.1 in all the configs above and then only local connections will be accepted.

If you are running this on a cloud server and you don’t really know what you’re doing here’s a quick firewall using ufw:

# Allow SSH form anywhere so we don't get locked out
ufw allow ssh
# Allow TCP/UDP 30303 from anywhere for peering Erigon
ufw allow 30303
# Allow TCP/UDP 42069 from anywhere for Erigon BitTorrent Sync
ufw allow 42069
# Allow TCP/UDP 9000 from anywhere for peering Lighthouse
ufw allow 9000
# Allow TCP/UDP 9001 from anywhere for QUIC peering Lighthouse
ufw allow 9001
# Turn on the firewall
ufw enable

# If you want to allow specific IPs (e.g. 1.1.1.1) to hit your RPC endpoint:
ufw allow proto tcp from 1.1.1.1 to any port 8545

Let the nodling commence

systemctl enable supervisor
systemctl start supervisor
supervisorctl update

This should enable supervisor, telling it to start at startup, and it should start supervisor which in turn should start your erigon and lighthouse services.

You can check on the status of stuff:

# Check on the erigon service:
supervisorctl status erigon
tail -f /var/log/supervisor/erigon.err.log

# Check on the lighthouse status:
supervisorctl status lighthouse
tail -f /var/log/supervisor/lighthouse.err.log

If for some reason the services don’t start it’s probably a typo in your supervisor config files /etc/supervisor/conf.d/erigon.conf or /etc/supervisor/conf.d/lighthouse.conf

Once you make whatever changes you need to make to the config you reload it with this:

supervisorctl update

Updating Your Node

Updating Erigon is fairly straight forward, we simply pull the latest changes from github, run make, and restart our services.

cd /opt/github/erigon/
git fetch --all
git checkout <new release tag>
make
supervisorctl stop erigon
cp -r ./build/bin /opt/erigon/
supervisorctl start erigon

Lighthouse is even easier…just run the install commands again with a new version

cd /opt/lighthouse

RELEASE=<new release tag>
DOWNLOADURL="https://github.com/sigp/lighthouse/releases/download/${RELEASE}/lighthouse-${RELEASE}-x86_64-unknown-linux-gnu.tar.gz"

wget -q --show-progress -O ./lighthouse.tar.gz ${DOWNLOADURL}
tar -xzf lighthouse.tar.gz -C ./bin/

chown -R eth:eth /data/lighthouse