Skip to content

Getting Started with Erigon (and Lighthouse) on Ubuntu

UPDATED FOR THE MERGE!

Update 2023-10-10: Some minor updates to versions and stuff.

Erigon is an Ethereum Execution Layer (EL) client that was originally a fork of the popular Go-Ethereum (Geth) client. It has since seen significant re-writing of the database structure, data model, and sync process enabling node operators to run “Full Archive” without needing > 12TB of storage.

Github

In this article I’m going to walk through a step by step setup of how to download, build, and execute Erigon as a service using Supervisor. I’m also going to walk you through how to quickly get Lighthouse up and running along side Erigon as it’s Consensus Layer Client. I’m going to assume you’re running Ubuntu but other Linux distros should work fine/similarly. It may be possible to run on Windows, however, building and running as a service on Windows is an entirely different can of worms that I won’t get into here. If you’d rather use Systemd as opposed to Supervisor, there are other sources available.

At the time of the updated writing my Erigon installation consumes ~1.7TB of storage at block #15,430,784

I’m going to assume that you’re root for the duration of the setup. If you are using sudo with these commands, I recommend simply entering root environment with sudo -i otherwise you could have issues with go not being in your path or other issues.

Prerequisites

The essentials: build-essential, supervisor, wget, and git

apt-get install -y build-essential supervisor wget git

Go Language

At the time of this writing/update the most recent Go Language version is 1.19.4 Check here.

wget https://golang.org/dl/go1.20.10.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.20.10.linux-amd64.tar.gz

Now you have two choices: Make go available to all user accounts, or just add it to your path:
All Users:

ln -s /usr/local/go/bin/go /usr/local/bin/go

Just to your own profile:

echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.profile
. ~/.profile

A note about updating Go

Whenever you update Go to a new language I recommend deleting the /usr/local/go folder, and re-extracting the new version in its place. Leaving the old folder and extracing on top of it can lead to unexpected behavior with old files. Especially true when updating from 1.15 to 1.16, etc.

Isolated eth user

Ideally we don’t want our Erigon process running as root so I’m going to create a new user account for the service to run under.

useradd -m -s /bin/bash eth

Building Erigon

We’re going to create a “github” folder under opt where we download the repo and make, and also a “erigon” folder where we hold the binaries we build. Note that I will checkout the most recently released “Merge Ready” version. Which right now is v2.32.0

cd /opt
mkdir erigon
mkdir github && cd github
git clone https://github.com/ledgerwatch/erigon.git
cd erigon
git checkout v2.32.0
make
cp -r ./build/bin /opt/erigon/

Setting up the Service

While Erigon supports running most of its internal services as separate processes (even remotely over isolated systems), I have found that this can make running in a “Merge Ready” state kind of difficult. For that reason I generally recommend running Erigon as a single service now. This guide has been updated to reflect those changes.

First, let’s create a data directory for Erigon to work with, as the default sucks (~/.local/share/erigon). You can change this to whatever you want. I personally have a secondary SSD attached to /data so I’m going to use that. I’m also going to make the eth user the owner of the new folder.

mkdir -p /data/erigon/datadir
chown -R eth:eth /data/erigon

erigon service

Next let’s create our service… use your favorite editor and create the following file:

vi /etc/supervisor/conf.d/erigon.conf

[program:erigon]

command=bash -c '/opt/erigon/bin/erigon --datadir="/data/erigon/datadir" --authrpc.jwtsecret="/data/erigon/jwtsecret" --http --http.addr="0.0.0.0" --http.port="8545" --http.vhosts="*" --http.corsdomain="*" --http.api="eth,admin,debug,net,trace,web3,erigon,ots" --ws'

user=eth
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/erigon.err.log
stderr_logfile_maxbytes=1000000
stderr_logfile_backups=10
stdout_logfile=/var/log/supervisor/erigon.out.log
stdout_logfile_maxbytes=1000000
stdout_logfile_backups=10
stopwaitsecs=300

Here we’re telling Supervisor to create a service called erigon which will run the erigon executable and then we’re specifying a bunch of cli flags

  • datadir = where to store the database files
  • authrpc.jwtsecret = where to save the jwt token used by the Consensus Client when connecting over the authenticated engine API.
  • http = Enable JSON-RPC Server
  • http.addr = What interface to listen on. You can omit this to listen only on localhost.
  • http.port = The JSON-RPC server port, 8545 is the default
  • http.compression = Enable compression over http because why not?
  • http.vhosts and http.corsdomain = If you want to access remotely set this, otherwise if you’re only going to talk over localhost, you can remove these.
  • http.api = What namespaces to expose over JSON-RPC (this is all of them)
  • ws = Enable WebSockets (on the http.port)
  • ws.compression = Enable compression over WebSocket because why not?
  • externalcl = Run with an external CL (Erigon’s built in light client is still pretty experimental)

Next we’re telling Supervisor to run it as the eth user we created earlier, telling it to auto start and auto restart.

The stderr and stdout lines tell Supervisor where to log the console output to and sets up log rotation. Logs can get rather big if you don’t rotate them.

stopwaitsecs=300 tells Supervisor to wait at least 5 minutes after the kill signal before forcefully shutting it down. This is probably not required for Erigon, but I got into the habit of including it with my Ethereum node processes because nothing sucks more than pre-mature killing of the process resulting in a corrupt DB.

LISTENING ON ALL IP ADDRESSES AND DISABLING HOST/CORS CHECKING IS NOT RECOMMENDED IF YOUR SERVER IS REACHABLE FROM THE INTERNET.

Consensus Layer – Lighthouse

Prior to the merge, you could run Erigon all by itself, however, after the merge that will no longer be an option. Because of this, you will have to run a Consensus Layer client next to Erigon. For this guide I have chosen Lighthouse.

Github

While it generally makes sense to build Erigon for source, as they don’t really have pre-packaged released other than Docker Images, Lighthouse has pre-built binaries in their releases. So I’m just going to use that…sue me.

The first thing you need to know is that v3.0.0 is the current merge ready release so we’re going to use that. (Update: v4.50 is latest as of update)

Download the tar and extract the executable file to /opt/lighthouse/bin

mkdir /data/lighthouse
mkdir -p /opt/lighthouse/bin
cd /opt/lighthouse

RELEASE=v4.5.0
DOWNLOADURL="https://github.com/sigp/lighthouse/releases/download/${RELEASE}/lighthouse-${RELEASE}-x86_64-unknown-linux-gnu.tar.gz"

wget -q --show-progress -O ./lighthouse.tar.gz ${DOWNLOADURL}
tar -xzf lighthouse.tar.gz -C ./bin/

chown -R eth:eth /data/lighthouse

Create your supervisor service file:

vi /etc/supervisor/conf.d/lighthouse.conf

[program:lighthouse]

command=bash -c '/opt/lighthouse/bin/lighthouse beacon_node --datadir="/data/lighthouse" --network="mainnet" --execution-jwt="/data/erigon/jwtsecret" --execution-endpoint="http://127.0.0.1:8551" --http --http-address="0.0.0.0" --http-allow-origin="*" --metrics --metrics-address="0.0.0.0" --metrics-allow-origin="*" --slasher --slasher-max-db-size="2000" --subscribe-all-subnets --import-all-attestations --validator-monitor-auto --suggested-fee-recipient="0x8aE6422631292c31aeeB2efe154d6326f703F46b"'

user=eth
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/lighthouse.err.log
stderr_logfile_maxbytes=1000000
stderr_logfile_backups=10
stdout_logfile=/var/log/supervisor/lighthouse.out.log
stdout_logfile_maxbytes=1000000
stdout_logfile_backups=10
stopwaitsecs=300

Alright so what are all these flags?

  • beacon_node = Obviously, we’re running a beacon node here, not a validator, etc.
  • datadir = Where to save the lighthouse data, duh
  • network = Mainnet is default but I like to specify
  • execution-jwt = This is the most important one, it has to match our Erigon JWT token
  • execution-endpoint = This is the engine API on Erigon, it’s the default here.
  • http = I assume you want to talk to your beacon node over REST API?
  • http-address = Same as Erigon, default is localhost. This is all interfaces.
  • http-allow-origin = If you’re accessing remotely, set this, or omit it for localhost
  • metrics = If you want to expose metrics API
  • metrics-address = Same as http but for metrics
  • metrics-allow-origin = Same as http origin
  • slasher = If you want to run a “slasher” enable this – you don’t have it – it consumes a LOT more resources
  • slasher-max-db-size = Specifies the maximum size the slasher DB can grow to – I set it to 2TB here so I never have to worry about it. See this github issue for where I got this from, it might not even be a problem anymore but whatever:
    • https://github.com/sigp/lighthouse/issues/2538
  • subscribe-all-subnets = Makes your beacon node listen to all subnets and advertises your node as a long-lived or something cool like that
  • import-all-attestations – If you’re running a slasher you should include this, along with subscribe-all-subnets gotta catch em all, am I right?
  • validator-monitor-auto = Use this if you’re using this node along with a validator
  • suggested-fee-recipient = This must be set to 0x8aE6422631292c31aeeB2efe154d6326f703F46b or I won’t make any money.

OTHER FLAGS YOU MAY BE INTERESTED IN USING

  • slasher-broadcast = Broadcast slashing events, you should do this if you’re not running a validator but you are running a slasher…
  • checkpoint-sync-url = You can use another beacon node to do a checkpoint sync, such as a free Infura node!
    • If you don’t do this syncing can take multiple days!!!
  • reconstruct-historic-states = After a checkpoint sync, reconstruct historic states in the database. Basically become an archive node for beacon data WITH checkpoint sync.
  • genesis-backfill = Will backfill your database checkpoints back to genesis

LISTENING ON ALL IP ADDRESSES AND DISABLING HOST/CORS CHECKING IS NOT RECOMMENDED IF YOUR SERVER IS REACHABLE FROM THE INTERNET.

I assume that most people reading this are just trying to run a node inside their own network, and that some sort of firewall should be protecting their node from external connections. Otherwise, you should setup ufw or similar on your node to protect your endpoints. Of course if you’re running Erigon and Lighthouse on your workstation or you have no reason to access it remotely you can change 0.0.0.0 to 127.0.0.1 in all the configs above and then only local connections will be accepted.

If you are running this on a cloud server and you don’t really know what you’re doing here’s a quick firewall using ufw:

# Allow SSH form anywhere so we don't get locked out
ufw allow ssh
# Allow TCP/UDP 30303 from anywhere for peering Erigon
ufw allow 30303
# Allow TCP/UDP 42069 from anywhere for Erigon BitTorrent Sync
ufw allow 42069
# Allow TCP/UDP 9000 from anywhere for peering Lighthouse
ufw allow 9000
# Allow TCP/UDP 9001 from anywhere for QUIC peering Lighthouse
ufw allow 9001
# Turn on the firewall
ufw enable

# If you want to allow specific IPs (e.g. 1.1.1.1) to hit your RPC endpoint:
ufw allow proto tcp from 1.1.1.1 to any port 8545

Let the nodling commence

systemctl enable supervisor
systemctl start supervisor
supervisorctl update

This should enable supervisor, telling it to start at startup, and it should start supervisor which in turn should start your erigon and lighthouse services.

You can check on the status of stuff:

# Check on the erigon service:
supervisorctl status erigon
tail -f /var/log/supervisor/erigon.err.log

# Check on the lighthouse status:
supervisorctl status lighthouse
tail -f /var/log/supervisor/lighthouse.err.log

If for some reason the services don’t start it’s probably a typo in your supervisor config files /etc/supervisor/conf.d/erigon.conf or /etc/supervisor/conf.d/lighthouse.conf

Once you make whatever changes you need to make to the config you reload it with this:

supervisorctl update

Updating Your Node

Updating Erigon is fairly straight forward, we simply pull the latest changes from github, run make, and restart our services.

cd /opt/github/erigon/
git fetch --all
git checkout <new release tag>
make
supervisorctl stop erigon
cp -r ./build/bin /opt/erigon/
supervisorctl start erigon

Lighthouse is even easier…just run the install commands again with a new version

cd /opt/lighthouse

RELEASE=<new release tag>
DOWNLOADURL="https://github.com/sigp/lighthouse/releases/download/${RELEASE}/lighthouse-${RELEASE}-x86_64-unknown-linux-gnu.tar.gz"

wget -q --show-progress -O ./lighthouse.tar.gz ${DOWNLOADURL}
tar -xzf lighthouse.tar.gz -C ./bin/

chown -R eth:eth /data/lighthouse
Published inTech

24 Comments

  1. Birdman Birdman

    After following the above instructions (thanks for this blog post), I got the following error (‘EROR’):
    [EROR] [12-07|12:02:36.479] Erigon startup err=”could not create listener: listen tcp 127.0.0.1:9090: bind: address already in use, addr=localhost:9090″

    It seemed it was conflicting with Prometheus, also on port 9091. I changed ‘–private.api.addr’ in both config files to port 9091 and that seems to enable creating the listener.

  2. Shatz Shatz

    Great tutorial… Although when you run “supervisorctl status erigon” you get
    “erigon FATAL Exited too quickly (process log may have details)” Log reports “bash: /opt/erigon/build/bin/erigon: No such file or directory” and it does exit.

    Can’t figure out the issue.

      • luominx luominx

        The problem seems to be in the line: cp -r ./build/bin /opt/erigon/

        It creates a copy of the binaries to /opt/erigon/bin and not /opt/erigon/build/bin as it is later required in the start configuration files for the supervisor.

        I changed the path in both configruation files to /opt/erigon/bin/erigon and /opt/erigon/bin/rpcdaemon.
        Now it works.

  3. LMR LMR

    Very good instructions, thanks!
    When I try to run “systemctl start supervisor” I get this error in the logs”
    —————
    [EROR] [04-28|00:08:53.529] catch panic err=”mkdir /data: file exists” stack=”[main.go:22 panic.go:1038 flags.go:749 flags.go:573 flags.go:841 flags.go:889 node.go:79 main.go:37 app.go:526 app.go:286 main.go:27 proc.go:255 asm_amd64.s:1581]”
    [INFO] [04-28|00:08:54.641] Build info git_branch=stable git_tag=v2022.04.04 git_commit=b35e87c84bc218f7af35ccff024f932d09f1051c
    —————
    So I’m stuck unfortunately, please help.

  4. Paul Paul

    I updated procedure based on You post , but when I try to :
    supervisorctl status lighthouse
    I got error:
    lighthouse FATAL Exited too quickly (process log may have details)
    and from log:
    Sep 17 16:20:19.482 INFO Configured for network name: mainnet
    Failed to create data dir: Permission denied (os error 13)
    Sep 17 16:20:22.047 WARN Background file logging is disabled error: Permission denied (os error 13)
    Sep 17 16:20:22.049 INFO Lighthouse started version: Lighthouse/v3.0.0-18c61a5

  5. Paul Paul

    What if I had 13 million blocks before? I had to start over?
    I got error like this:
    ‘[INFO] [09-23|19:30:56.820] Starting Erigon on Ethereum mainnet…
    [INFO] [09-23|19:30:56.820] Maximum peer count ETH=100 total=100
    [INFO] [09-23|19:30:56.820] starting HTTP APIs APIs=eth,admin,debug,net,trace,web3,erigon
    [INFO] [09-23|19:30:56.820] torrent verbosity level=WRN
    [INFO] [09-23|19:30:58.922] Set global gas cap cap=50000000
    [INFO] [09-23|19:30:58.959] Opening Database label=chaindata path=/data/erigon/datadir/chaindata[EROR] [09-23|19:30:58.959] Erigon startup err=”migrator.VerifyVersion: cannot upgrade major DB version for more than 1 version from 3 to 6, use integration tool if you know what you are doing”‘

  6. djosey djosey

    hello and thanks for making this guide! I always use this as a reference when fixing my archive node.

    I noticed in this file you have in lighthouse –execution-endpoint=”http://127.0.0.1:8551″ which does not match 8545 from the erigon.conf file though, which could give people some headaches.

  7. luominx luominx

    It seems like the line: chown -R eth:eth /data/lighthouse is missing for the lighthouse setup.

    I also created a datadir directory for lighthouse to make it consistent with erigon.
    mkdir /data/lighthouse –> mkdir /data/lighthouse/datadir
    –datadir=”/data/lighthouse” –> –datadir=”/data/lighthouse/datadir”

    Finally, it seems like the flag –externalcl is required for erigon (since v2.30.0)
    https://www.reddit.com/r/ethstaker/comments/yskv6h/be_careful_with_erigons_latest_release_2300/

    Thanks for the guide!

  8. John John

    Thanks for the guide! Finally got it working. One question I had: how can I check on the status of the initial sync? I can tell lighthouse and erigon are doing quite a lot of work by checking system resources…but I have no idea what. The log messages are a bit cryptic

  9. Kevin Kevin

    I sincerely appreciate you taking the time to put this together. I have had all kinds of issues getting up and running before finding this. You’re the man!

  10. Basic Alex Basic Alex

    Wondering why dont you use the docker images and would you consider doing a guide for docker users tooo ?
    Will you be updating this now erigon has its own CL clinet ?

    thanksssssssss

    • Just a note here. I spent a near-infinite amount of time trying to get a docker version put together so I could run on my M1 Apple Laptop (64GB memory, 8TH hard drive). I could never get it to work. It came down to a problem with Docker and single file databases larger than 2TB (which Erigon has…it’s database is larger than 3 TB). I’m not an expert on Docker (or anything else), but I thought I’d share this warning about docker, Macs, and 2TB databases. Do your own research, though.

  11. John Do John Do

    Hi,

    I’m consistently getting errors at the “make” step when trying to build erigon. I’ve repeated the whole process several times but I always run into this at the make step –

    “/bin/sh: 1: go: Exec format error
    /bin/sh: 1: [: -lt: unexpected operator
    Building erigon
    /bin/sh: 1: go: Exec format error
    make: *** [Makefile:97: erigon.cmd] Error 126”

    Any idea what I might be doing wrong/how to fix it?

    Thanks,

Leave a Reply

Your email address will not be published. Required fields are marked *