Skip to content

How to run an Erigon Archive Node on Polygon

We’re going to be building off my previous blog posts to build an Erigon Archive Node on Polygon (the PoS Chain).

We’re going to combine these into a new blog post with new information! Running Erigon instead of bor so we can serve full archive data for Matic/Polygon over on ArchiveNode.io!

I’m not going to go into a ton of details here, you can read the other blog posts for more information but I’ll hit the highlights and new information since writing those blog posts.

The first thing you need to know is that Erigon currently calls this network bor-mainnet don’t ask me why, that’s just what it is…

The second thing you need to know is that Erigon support for Matic/Polygon is very much “Alpha” and I haven’t even successfully sync’d a node myself yet! But I’m trying!

We’re going to build this “native” and without Docker this time.

Let’s begin

I start by installing Go 1.18.2

# Golang (https://golang.org/dl/)
GOVERSION="1.18.2"
wget -q -O go${GOVERSION}.linux-amd64.tar.gz https://golang.org/dl/go${GOVERSION}.linux-amd64.tar.gz
sudo rm -rf /usr/local/go
sudo tar -C /usr/local -xzf go${GOVERSION}.linux-amd64.tar.gz
rm go${GOVERSION}.linux-amd64.tar.gz
sudo ln -s /usr/local/go/bin/go /usr/local/bin/go

Next I create some directories to work with

mkdir -p /data/github
mkdir -p /data/heimdall/bin
mkdir -p /data/erigon/datadir
mkdir /data/erigon/ethash

Next I clone and build Heimdall (we still need this)

cd /data/github
git clone -b v0.2.9 https://github.com/maticnetwork/heimdall.git
cd heimdall && make build network=mainnet && cp build/* /data/heimdall/bin/

Notice that I moved the binaries into /data/heimdall/bin

Next I clone and build Erigon from it’s latest alpha release (note: Beta doesn’t work here)

cd /data/github
git clone -b v2022.05.03 https://github.com/ledgerwatch/erigon.git
cd erigon && make && cp -r build/bin /data/erigon/

Again, I moved the binaries over to /data/erigon/bin/

If you want to you can create non-interactive users for the processes to run under:

# Create users
adduser --system --group heimdall
adduser --system --group erigon

# Set ownership of folders we created ealier
chown -R heimdall:heimdall /data/heimdall
chown -R erigon:erigon /data/erigon

Setting up Heimdall (PoS Client)

Now we need to init heimdalld just like we did in the previous blog posts

# As heimdall user init with home directory set
sudo -u heimdall /data/heimdall/bin/heimdalld init --home /data/heimdall

# As hemidall user replace the genesis file
sudo -u heimdall wget -O /data/heimdall/config/genesis.json https://raw.githubusercontent.com/maticnetwork/launch/master/mainnet-v1/without-sentry/heimdall/config/genesis.json

Just like the previous blog post we need to modify the config files

# Set the seed servers
sed -i '/^seeds/c\seeds = "f4f605d60b8ffaaf15240564e58a81103510631c@159.203.9.164:26656,4fb1bc820088764a564d4f66bba1963d47d82329@44.232.55.71:26656,2eadba4be3ce47ac8db0a3538cb923b57b41c927@35.199.4.13:26656,3b23b20017a6f348d329c102ddc0088f0a10a444@35.221.13.28:26656,25f5f65a09c56e9f1d2d90618aa70cd358aa68da@35.230.116.151:26656"' /data/heimdall/config/config.toml

# Allow CORS (Optional)
sed -i "s#^cors_allowed_origins.*#cors_allowed_origins = [\"*\"]#" /data/heimdall/config/config.toml

Now, we want to download our snapshot (we don’t need a Heimdall archive node…). Same as before get the latest snapshot from https://snapshots.matic.today/

# Set snapshot URL
SNAPSHOT_URL=https://matic-blockchain-snapshots.s3-accelerate.amazonaws.com/matic-mainnet/heimdall-snapshot-2022-05-10.tar.gz

# Download and extract in one command
wget -c "${SNAPSHOT_URL}" -O - | tar -xz -C /data/heimdall/data

# Reset permissions again
chown -R heimdall:heimdall /data/heimdall

Now heimdalld and heimdallr (the rest server) are ready to run. You can create a systemd service file or use supervisor which I like to do.

apt install -y supervisor

# /etc/supervisor/conf.d/heimdalld.conf
[program:heimdalld]
command=bash -c '/data/heimdall/bin/heimdalld --home /data/heimdall start'
user=heimdall
group=heimdall
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/heimdalld.err.log
stderr_logfile_maxbytes=10000000
stderr_logfile_backups=10
stdout_logfile=/var/log/supervisor/heimdalld.out.log
stdout_logfile_maxbytes=10000000
stdout_logfile_backups=10
stopwaitsecs=300
# /etc/supervisor/conf.d/heimdallr.conf
[program:heimdallr]
command=bash -c '/data/heimdall/bin/heimdalld --home /data/heimdall rest-server --chain-id=137'
user=heimdall
group=heimdall
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/heimdallr.err.log
stderr_logfile_maxbytes=10000000
stderr_logfile_backups=10
stdout_logfile=/var/log/supervisor/heimdallr.out.log
stdout_logfile_maxbytes=10000000
stdout_logfile_backups=10
stopwaitsecs=30

Start heimdalld and heimdallr

systemctl daemon-reload
systemctl enable supervisor
systemctl start supervisor
supervisorctl update

Again if everything goes right heimdalld should sync from the snapshot to the current block, you can check with http://localhost:26657/status

You should see "network": "heimdall-137" in the output, if you don’t something went wrong with the init and replacing the genesis.json file… anyway wait for it to say "catching_up": false

Now that the Heimdall stuff is out of the way we can get to the Erigon stuff…

Setting up Erigon Archive Node

Since my previous blog post Erigon has ALSO integrated rpcdaemon directly into erigon so you don’t HAVE to run a secondary process (you still can…). But, regardless there are a few key flags required to sync Polygon…

--chain=bor-mainnet
--bor.heimdall="http://localhost:1317"

These should be obvious, but there’s a couple more considerations…

  1. There is a maximum size of 8TB on the Erigon database unless you bump the database page size with --db.pagesize
    • I suspect that the database is currently over 6TB in size…
    • The default is 4kb
    • The maximum size is 64kb
    • I do not know how severely these settings will impact performance
    • I recommend either picking 64kb and never worrying about it again, or picking something more like 16kb which is probably fine?
    • Look, I’m too lazy to benchmark alright?
  2. The current syncmode, for Polygon, is full this is because nobody has generated the “snapsync” files which can be distributed over BitTorrent, yet.

With that out of the way here’s how my supervisor config file looks:

# /etc/supervisor/conf.d/erigon.conf
[program:erigon]
command=bash -c '/data/erigon/bin/erigon --db.pagesize="64kb" --chain="bor-mainnet" --datadir="/data/erigon/datadir" --ethash.dagdir="/data/erigon/ethash" --syncmode="full" --bor.heimdall="http://localhost:1317" --http --http.addr="0.0.0.0" --http.port="8545" --http.compression --http.vhosts="*" --http.corsdomain="*" --http.api="eth,debug,net,trace,web3,erigon,bor" --ws --ws.compression --rpc.gascap="300000000"'
user=erigon
group=erigon
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/erigon.err.log
stderr_logfile_maxbytes=10000000
stderr_logfile_backups=10
stdout_logfile=/var/log/supervisor/erigon.out.log
stdout_logfile_maxbytes=10000000
stdout_logfile_backups=10
stopwaitsecs=300

supervisorctl update and we’re off to the races.

Log files can be seen via tail -f /var/log/supervisor/erigon.err.log etc.

Happy Archive Nodling.

Published inTech

Be First to Comment

Leave a Reply

Your email address will not be published.