Skip to content

How to run a Polygon (Matic) Mainnet Node with Docker

Running a Matic Node seems to consist of two four parts: The the PoS node, a REST API server, a RabbitMQ server, and an EVM node. The PoS node is called heimdall as is the REST API server, and the EVM Node is called bor. According to the documentation we need to run heimdall first and let it sync…so let’s start there.

Also, a word of warning, I’m gonna assume you’re behind some kinda firewall here, don’t expose your node’s RPC/API to the world, duh. I’ll call out what the ports are for below.

# Make a directory to work in
mkdir -p /data/github
cd /data/github

# Clone the repo
git clone https://github.com/maticnetwork/heimdall.git
cd heimdall

# Checkout the mainnet release version
git checkout v0.2.3

# Copy the Dockerfile up a directory
cp docker/Dockerfile .

# Edit their shitty Dockerfile
sed -i 's#./logs#/root/heimdall/logs#' Dockerfile

# Build the docker image
docker build --tag heimdall:latest .

# Make a directory for heimdall data
mkdir -p /data/volumes/heimdall

# Init configs and stuff
docker run -it -v /data/volumes/heimdall:/root/.heimdalld heimdall:latest heimdalld init

# Overwrite the genesis.json file with the mainnet launch genesis.json file
wget https://raw.githubusercontent.com/maticnetwork/launch/master/mainnet-v1/without-sentry/heimdall/config/genesis.json -O /data/volumes/heimdall/config/genesis.json

You might say to yourself “I don’t want to checkout the mainnet release version, I want to be on the latest and greatest version!” but you’d be wrong because it doesn’t work…at all…at the time of writing. :shrug:

Now you need to modify some config files. There are 2 config files of importance here.

config.toml and heimdall-config.toml they’re both located at /data/volumes/heimdall/config/

The key parts to modify in config.toml are:

  • cors_allowed_origins
    • Set it to ["*"]
    • cors_allowed_origins = ["*"]
  • seeds
    • Set it to the list of seeds found HERE
    • seeds = "f4f605d60b8ffaaf15240564e58a81103510631c@159.203.9.164:26656,4fb1bc820088764a564d4f66bba1963d47d82329@44.232.55.71:26656"

The key parts to modify in the heimdall-config.toml file are:

  • amqp_url
    • Point this to RabbitMQ server (we’ll run a docker called rabbitmq) so set it to:
      amqp_url = "amqp://guest:guest@rabbitmq:5672"

There’s also two other variables that I suggest editing LATER, after you’re up and running and in sync:

  • eth_rpc_url
    • This should point to an ETH1 endpoint, local or remote like Infura
    • I don’t think this does anything if you’re not validating though
  • bor_rpc_url
    • You can point this to your bor node after your bor node is in sync
    • I don’t think this does anything if you’re not validating though

You can pretty much ignore everything else, we’ll set some other stuff via command-line options. And obviously you haven’t created your bor node yet so you can come back to it later.

SNAPSHOTS

A quick word on sync time…it’s not fast. If you want to validate the entire chain (PoS (heimdall) and EVM (bor)) more power to you. If however, you just want to get up and running as fast as possible, the community offers database snapshot over on their new snapshot page.

https://snapshots.matic.today/

If you want to use a snapshot, now is the time to do that for heimdall. And while you’re waiting for heimdall to catch up, you can install the bor snapshot after you initialize it.

Heimdall Snapshot

Here is an example of how to download and extract the Heimdall snapshot in one line (this saves storage space by not having to download AND extract these large files). Otherwise just do your typical wget and then tar

# Download and tar extract via pipeline (linux magic)
wget -c https://matic-blockchain-snapshots.s3-accelerate.amazonaws.com/matic-mainnet/heimdall-snapshot-2021-09-24.tar.gz -O - | tar -xz -C /data/volumes/heimdall/data

Running Heimdall

# Run it / Make sure it's working
docker run -it -p "26656:26656" -p "26657:26657" -v /data/volumes/heimdall:/root/.heimdalld heimdall:latest heimdalld start

Even without having set proper eth_rpc_url or bor_rpc_url variables (or running the RabbitMQ server yet) it should start to sync the PoS network. However, you do need the other 2 service (REST API and RabbitMQ) for bor to work, so let’s set those up inside a docker-compose file.

Docker Compose

version: '3.4'

services:
  rabbitmq:
    container_name: rabbitmq
    image: rabbitmq:3-alpine
    ports:
      - "5672:5672" # RabbitMQ
    restart: unless-stopped
  
  heimdalld:
    container_name: heimdalld
    image: heimdall:latest
    build: /data/github/heimdall
    restart: unless-stopped
    volumes:
      - /data/volumes/heimdall:/root/.heimdalld
    ports:
      - "26656:26656" # P2P (TCP)
      - "26657:26657" # RPC (TCP)
    depends_on:
      - rabbitmq
    command:
      - heimdalld
      - start
      - --moniker=MyNodeName
      - --p2p.laddr=tcp://0.0.0.0:26656
      - --rpc.laddr=tcp://0.0.0.0:26657

  heimdallr:
    container_name: heimdallr
    image: heimdall:latest
    build: /data/github/heimdall
    restart: unless-stopped
    volumes:
      - /data/volumes/heimdall:/root/.heimdalld
    ports:
      - "1317:1317" # Heimdall REST API
    depends_on:
      - heimdalld
    command:
      - heimdalld
      - rest-server
      - --chain-id=137
      - --laddr=tcp://0.0.0.0:1317
      - --node=tcp://heimdalld:26657

Now if you run docker-compose up -d your docker container will start in the background. It’ll pull RabbitMQ and run that, then it will launch heimdalld the PoS node, and heimdallr the REST API server. Both heimdalld and heimdallr will read from the same config files so they’re mounted to the same location. As you can see from the config heimdallr points to heimdalld with the --node flag.

Wait until you’re in sync

How do you know if you’re in sync?

$ curl http://localhost:26657/status
{
  "jsonrpc": "2.0",
  "id": "",
  "result": {
    "node_info": {
      "protocol_version": {
        "p2p": "7",
        "block": "10",
        "app": "0"
      },
      "id": "d09d79f111a56284495dec4762056f801bdafd17",
      "listen_addr": "tcp://0.0.0.0:26656",
      "network": "heimdall-137",
      "version": "0.32.7",
      "channels": "4020212223303800",
      "moniker": "MysticRyuujin",
      "other": {
        "tx_index": "on",
        "rpc_address": "tcp://0.0.0.0:26657"
      }
    },
    "sync_info": {
      "latest_block_hash": "D081C8AEA93F78526E6088379819548D4ED780B4D2D24E405D744CB368817510",
      "latest_app_hash": "F43512329EC6E77C9FABC8D38FFD75747C4C787631829676AB55DB787399570D",
      "latest_block_height": "342083",
      "latest_block_time": "2020-06-21T09:43:49.695593592Z",
      "catching_up": true
    },
    "validator_info": {
      "address": "5FC7043EECEA93C19B55FD33760E45CE246DCB93",
      "pub_key": {
        "type": "tendermint/PubKeySecp256k1",
        "value": "BA5qrjvWfVGHa8TWH47gMIq3O0dlIl+VWENts7kN7fZcP06A7y6Lf/yXuFxdcQ6sfEX8eFsv6MmttQ2eG9XA3qk="
      },
      "voting_power": "0"
    }
  }
}

Looking at the above information we can see that our node has this info:

      "latest_block_time": "2020-06-21T09:43:49.695593592Z",
      "catching_up": true

When that says false and the timestamp makes sense, you’re in sync

bor? Boring

Bor is basically just a Geth clone, the setup is similar

# Re-use our github folder
cd /data/github

# Clone the repo
git clone https://github.com/maticnetwork/bor.git
cd bor

# This is the current recommended version
# git checkout v0.2.9

# Build the docker image
docker build --tag bor:latest .

# Make a directory for bor data
mkdir -p /data/volumes/bor

# We need to download the genesis.json file FIRST this time
wget https://raw.githubusercontent.com/maticnetwork/launch/master/mainnet-v1/without-sentry/bor/genesis.json -O /data/volumes/bor/genesis.json

# Initialize bor with genesis file
docker run -it -v /data/volumes/bor:/datadir bor:latest bor --datadir /datadir init /datadir/genesis.json

Now would be a good time to install the bor snapshot if you’re so inclined.

Bor Snapshot

Here’s an example of how to grab the bor snapshot, double check the target folder is correct e.g. /data/volumes/bor/bor/chaindata

# Download and tar extract via pipeline (linux magic)
wget -c https://matic-blockchain-snapshots.s3-accelerate.amazonaws.com/matic-mainnet/bor-pruned-snapshot-2021-10-11.tar.gz -O - | tar -xz -C /data/volumes/bor/bor/chaindata

Now it’s a matter of running bor node the same way you would any other Geth node, with a couple of extras.

  • They provide you with a list of bootnodes HERE
  • You need to provide the --bor.heimdall flag and point it to the REST API server
    • --bor.heimdall=http://heimdallr:1317
  • You need to specifically include --networkid=137
  • Turn on the bor API under --http.api and --ws.api if you use websocket
  • Set gas targets and limits (idk why but their documentation does this everywhere)
  • Newer versions of bor disable logs, use --bor.logs to re-enable it
  • YOU MUST RUN IN FULL SYNC MODE

Here’s what that all looks like in a Docker-Compose. Remember: HEIMDALLD NEEDS TO BE IN SYNC BEFORE YOU RUN BOR.

Docker Compose

version: '3.4'

services:
  bor:
    container_name: bor
    image: bor:latest
    build: /data/github/bor
    volumes:
        - /data/volumes/bor:/datadir
    ports:
        - "8545:8545" # RPC
        - "30303:30303" # Peers (TCP)
        - "30303:30303/udp" # Peers (UDP)
    command:
        - bor
        - --syncmode=full
        - --datadir=/datadir
        - --networkid=137
        - --bor.heimdall=http://heimdallr:1317
        - --bor.logs
        - --miner.gaslimit=200000000
        - --miner.gastarget=20000000
        - --http
        - --http.addr=0.0.0.0
        - --http.port=8545
        - --http.api=eth,net,web3,admin,debug,bor
        - --http.corsdomain=*
        - --http.vhosts=*
        - --ws
        - --ws.addr=0.0.0.0
        - --ws.port=8545
        - --ws.api=eth,net,web3,admin,debug,bor
        - --ws.origins=*
        - --ipcdisable
        - --nousb
        - --bootnodes=enode://0cb82b395094ee4a2915e9714894627de9ed8498fb881cec6db7c65e8b9a5bd7f2f25cc84e71e89d0947e51c76e85d0847de848c7782b13c0255247a6758178c@44.232.55.71:30303,enode://88116f4295f5a31538ae409e4d44ad40d22e44ee9342869e7d68bdec55b0f83c1530355ce8b41fbec0928a7d75a5745d528450d30aec92066ab6ba1ee351d710@159.203.9.164:30303

You can customize this to support whatever security you need or don’t need. Just remember to take care when exposing RPC and API ports.

A note about Docker-Compose Networking

By default docker-compose creates a single network for each “docker-compose.yml” file / app definition, whatever you want to call it. So if you’ve followed this tutorial exactly, you’d end up with a network containing bor and another network containing heimdalld, heimdallr, and rabbitmq. They won’t talk to each other by default. You can either a) place the bor service into the same docker-compose file as the other services or b) create an external network and put everything into that network.

https://docs.docker.com/compose/networking/

Option A is the easiest, however, if you want to do Option B, here’s a simple way to do that:

# Run this command to create a network called polygon-network
docker network create -d bridge polygon-network

# Add this to the bottom of both docker-compose files
networks:
  default:
    external: true
    name: polygon-network

Help – I didn’t understand any of that!

Yeah – Matic/Polygon is not the easiest network to run. Which is why there are RPC as a Service providers like QuickNode who can get you up and running in seconds without any of those pesky rate limits that the public RPC providers include (assuming you get a Pro or Scale plan).

I understood all of that and/or you’re doing it wrong!

Then you should come work for QuickNode! https://hire.clarify.so/company/quiknode-inc

Disclaimer: This is my personal blog, it is not actually affiliated with QuickNode - I just thought I'd give a shout out since my employer is hiring and we need super stars!
Published inTech

45 Comments

  1. lcgogo lcgogo

    i met followed error when run heimdalld

    E[2021-05-26|11:19:51.371] Error dialing seed module=p2p err=”incompatible: Peer is on a different network. Got heimdall-137, expected heimdall-yMfbLt” seed=f4f605d60b8ffaaf15240564e58a81103510631c@159.203.9.164:26656

    • lcgogo lcgogo

      This is caused bynot replace the correct genesis and toml after bor init.

  2. LiChao LiChao

    Hi Chase,

    you metioned that,
    YOU MUST RUN IN FULL SYNC MODE
    But in you bor docker-compose file, you don’t write syncmode full, but in bor –help

    –syncmode value Blockchain sync mode (“fast”, “full”, “snap” or “light”) (default: fast)

    BRs.
    Li Chao

  3. Nicola Nicola

    Hi, get error:

    root@Ubuntu-2004-focal-64-minimal ~ # docker run -it -v /data/volumes/heimdall:/root/.heimdalld heimdall:latest heimdalld init
    docker: Error response from daemon: OCI runtime create failed: invalid mount {Destination:logs Type:bind Source:/var/lib/docker/volumes/1096744ba8775701bb4706919ed3cf4f5a34ac6b894c6a82e92cc4b2c340c4a6/_data Options:[rbind]}: mount destination logs not absolute: unknown.
    ERRO[0000] error waiting for container: context canceled
    root@Ubuntu-2004-focal-64-minimal ~ #

    • Does the folder you’re trying to mount exist? Try `mkdir -p /data/volumes/heimdall`

      • Nicola Nicola

        Yes sure, the directory exist following your instructions.

        What distro did you use? I can’t fix this error.

        By the way have you understand what is a difference between a validator with or without sentry node?

        • I corrected the instructions. It was caused by their Dockerfile. See “Edit their shitty Dockerfile”

    • jack jack

      just change the docker version,use old version is ok.

  4. jack jack

    if i want to enable the bor ipc ,what should i do?

  5. Vali Vali

    Anything to do to speedup the heimdall sync? It is running for 2 days now and still “catching up”.?

    • Not that I know of. I assume you did the snapshots? Nothing to do but let it run. The Polygon chain has block times of like 1 second and millions of tx per day. It is what it is.

  6. Aris Aris

    thanks for sharing! I have two questions:
    1) Do we run all the scripts in root? (That’s what I did, just to confirm)
    2) In the “Running Heimdall” section is the /data/volumes/heimdalld correct? or is this supposed to be /data/volumes/heimdall without the d?

    thanks

    • Aris Aris

      Hello again,

      I managed to run both nodes, however bor seems to be out of sync. I made sure that heimdall was in full sync before running bor. I suspect that bor is in sync up to the snapshot. Any idea on why bor may stop syncing after snapshot?

  7. arch_mage arch_mage

    How much time does your Heimdall node sync take? Even with the snapshot is is showing me blocks from June 2020, is this normal. shouldn’t it checkpoint the snapshot and only sync new blocks from that point onwards.

  8. Thanks for the great guide! What is a Sentry node, btw? I see this referenced in their docs and also in some of your commands.

    Also, is it necessary to run Heimdall if only interested in the full-node RPCs? I don’t understand the Heimdall requirement, especially if not actually staking and validating.

  9. mike mike

    Any ideas how to resolve the following issue?

    INFO [06-22|01:32:13.726] Retrying again in 5 seconds for next Heimdall span path=clerk/event-record/list
    INFO [06-22|01:32:23.726] Retrying again in 5 seconds for next Heimdall span path=clerk/event-record/list

    Appreciate the guide!

    • Bobby Bobby

      Hi, I am getting the same issue, did you eventually manage to fix this?

    • HunterX HunterX

      The reason is, bor is not able to reach heimdallr because heimdall(r/d) and rabbitmq are on a different network “heimdall_default”.
      You have to change the network of bor to that of heimdallr and recreate bor with docker-compse.

      https://docs.docker.com/compose/networking/

      Check under “use pre-exisiting network”

      • Ahh, yeah, I run everything on an external host bridge. I should update this with networking info. Good call.

  10. Afanasii Afanasii

    Thank you! This saves a ton of time.

  11. Are you running a node where you are staking your own MATIC and allowing people to delegate their MATIC to your node?

    • No, I am not staking. I just wanted a full node to query.

  12. Fabrizio Fabrizio

    Hi, thanks for your amazing tutorial, i am running into an issue when composing docker file for heimdall, the issue “Docker-compose up failing because “port is already allocated”, do you know how can i solve that?

    • It just means that another process (say, Geth) is already listening on one of the mapped ports (e.g. 30303). You’ll need to modify the port(s) that are already in use. Probably 30303 and/or 8545

  13. Lucas Lucas

    Thanks for the great tutorial!. The node runs in the docker but no config files are created. Any idea what I am missing?

    • Well bor doesn’t create any “config files”, heimdall should create config files when you run the `heimdalld init` command – and place them in the volume you mount when you run that command.

      • Lucas Lucas

        Thanks Chase – I just made a silly mistake when setting up the volumes.

  14. Alex Alex

    Thank you for this tutorial, I couldn’t do it without. May I suggest you give the correct directory structure for the downloaded chain snapshot? I couldn’t get it to work until after a bit of sleuthing.

    Really weird, I have one tab of this tutorial open and it is missing ‘A note about Docker-Compose Networking’, one tab I opened later does not have it missing.
    For anyone reading in the future, if you have the error below, refresh the page and read this note.
    Retrying again in 5 seconds for next Heimdall span path=clerk/event-record/list

    • Yeah, sure, I’ll update the doc with more instructions for the snapshots.

  15. Lucas Lucas

    Thanks for the great tutorial. The nodes are now up and running. However, the bor node is after two days not fully synched and still lagging around 10 blocks behind the public node. Also, the geth synching request returns block values rather than just false, which a fully synched node would do. How long did the synch process take for you?

    • Lucas Lucas

      Also, knownStates and pulledStates are stuck at 0. I am not sure what the issue is. Could this be related to the missing ETH node RPC link? Example output is below.

      “AttributeDict({‘currentBlock’: 17267284, ‘highestBlock’: 17267290, ‘knownStates’: 0, ‘pulledStates’: 0, ‘startingBlock’: 17256497})”

  16. Remi Remi

    How do you check if bor is synced ? Or it doesn’t need to sync since heimdall is already synced ? Cause i can query the websocket for historical transactions but not recent ones.

  17. Max Max

    Hi my Heimdall etc is working correct and no longer catching up.

    However when I look at the logs of my bor client I see:

    INFO [07-29|13:49:59.446] Deep froze chain segment blocks=8 elapsed=832.639ms number=17,046,990 hash=35d1a9..8ea638
    INFO [07-29|13:50:07.141] Imported new chain segment blocks=2 txs=621 mgas=40.623 elapsed=10.183s mgasps=3.989 number=17,136,992 hash=b0383c..a2c59b age=1w6h23m dirty=231.00MiB
    WARN [07-29|13:50:10.383] Served miner_setEtherbase conn=195.123.222.16:39076 reqid=1 t=”58.829µs” err=”the method miner_setEtherbase does not exist/is not available”
    WARN [07-29|13:50:10.946] Served miner_start

    Any ideas what one has done wrong?

  18. Noah Noah

    Hi Chase,

    This article is excellent. Thank you so much for putting this together. I am currently trying to deploy several nodes within a cluster, but I am running into a few errors. I have followed your deployment.

    “`ERROR[08-15|13:23:00.866] Unavailable modules in HTTP API list unavailable=[bor] available=”[admin debug web3 eth txpool personal ethash miner net]”
    “`

    Bor and Heimdall seem to be running, but I have no idea how to tell if they are syncing correctly. Could you provide any insight into this error and/or the syncing dilemma?

    Thank you!

    • Afanasii Afanasii

      Hi Noah,

      I’m having the same issue – “Unavailable modules in HTTP API list unavailable=[bor]”. Have you found a solution by chance?

      Thank you!

      • Afanasii Afanasii

        After several hours I’ve figured that out:

        “Unavailable modules in HTTP API list unavailable=[bor]” error means that bor consensus engine was not initialised on startup, and node defaulted to ethash. You can see the current engine from the line:

        Initialised chain configuration config=”{ChainID: 137 … Engine: bor}”

        (should be bor).

        In my case it was because I’ve tried to upgrade my node to “arpit/v1.10.6″ branch, which corrupted my genesis block with message:

        Invalid chain config JSON hash=a9c28c..97de1b err=”json: cannot unmarshal number into Go struct field BorConfig.bor.period of type map[string]uint64”

        And the whole thing fas fixed by rolling back to original bor version and rewriting the genesis block (without resyncing the chain!):

        docker run -it -v /data/volumes/bor:/datadir bor:latest bor –datadir /datadir init /datadir/genesis.json

        (was a bit different command for me, since I’m not using docker, but idea is the same).

        Have a great day.

        • Sevenisalie Sevenisalie

          By “Rolling back to original Bor version” do you mean to say you checked out the original branch and rebuilt the docker image?

  19. Mimax Mimax

    Hi,
    Thanks for your tutorial. I have a question, I have set up and run successfully the Heimdall but the bor when I start docker-compose it seems to have an error. This is status when I call docker-compose could you help me to check these issues?
    Name Command State Ports
    ————————————————————————————————————–
    bor bor –syncmode=full –data … Exit 0
    heimdalld heimdalld start –moniker= … Up 1317/tcp, 0.0.0.0:26656->26656/tcp,:::26656->26656/tcp,
    0.0.0.0:26657->26657/tcp,:::26657->26657/tcp
    heimdallr heimdalld rest-server –ch … Up 0.0.0.0:1317->1317/tcp,:::1317->1317/tcp, 26656/tcp, 26657/tcp
    rabbitmq docker-entrypoint.sh rabbi … Restarting

    Thanks and regards,

  20. Dmise Dmise

    Where is log file is located? To check that all fine. What file tail to watch to see the logs?

    accordingly official instruction bor service write in to system journal and we can use command: journalctl -u bor -f to see what bor write to log. But when we use docker this command (journalctl -u bor -f) does not show log.

    I see LOG file in /data/volumes/bor/bor/chaindata directory. Is is bot logs?

  21. David David

    Thanks for the great guide.
    Could anyone give some pointers on the best way to update bor (in detail please) as there has been an update released today 24 Aug 2021.
    ( https://github.com/maticnetwork/bor/releases/tag/v0.2.7 )
    is it a simple matter of pulling from github and overwriting the current /data/github/bor directory?

Leave a Reply

Your email address will not be published. Required fields are marked *