Skip to content

Getting Started with TrueBlocks

Published by MysticRyuujin on February 25, 2022

What is TrueBlocks?

TrueBlocks is a lightweight, ultra performance indexing solution for EVM chains. It was originally created due to many limitations in the JSON-RPC which prevents easy access to simple data.

Have you ever wanted to get all of the transactions involving a single address out of an Ethereum node? Good luck. Have you ever wanted to grab all the logs from a single contract out of an Ethereum node? Again, good luck.

While the Erigon client has made great strides in improving the performance of the JSON-RPC, it still has its limits, and it’s still very inefficient to find some of this data. That’s where TrueBlocks comes in. TrueBlocks creates an offline index of every single appearance of every single address, ever, on the blockchain. It does this by running Trace calls against every single transaction in every single block and simply creating a record of any time an address appears. It’s even more accurate than Etherscan.

If you’re interesting in learning more or becoming a part of the TrueBlocks community head on over to their Discord.

Building TrueBlocks

There are two ways to build TrueBlocks, native and Docker, I’ll cover both real quick.

First head on over to the GitHub, clone the repo:

git clone -b develop https://github.com/trueblocks/trueblocks-core
cd trueblocks-core

Building with Docker

I’ll assume you already have docker installed here…

docker build . --tag=trueblocks-core:develop

Building Native

sudo apt-get install -y build-essential cmake python3 python3-dev libcurl3-dev clang-format jq

mkdir build && cd build
cmake ../src
make -j 5

That’s is, congrats, you built TrueBlocks. But the fun is just beginning. The first thing you should do is add trueblocks-core/bin to your path (if you’re running natively)

Configuring TrueBlocks

The first thing to note is that if you did the native build, TrueBlocks installed some executables on your system and generated some config files. The binaries are located at ~/.local/bin/chifra and then there are configuration files under ~/.local/share/trueblocks for Linux and ~/Library/Application\ Support/TrueBlocks on macs, but I’m on Linux right now so I’m just going to continue using that path.

The most important file is probably ~/.local/share/trueblocks/trueBlocks.toml this is your TrueBlocks configuration file. We need to populate some fields before it’s going to work.

I’m only going to focus on Mainnet for now:

cat ~/.local/share/trueblocks/trueBlocks.toml
[version]
current = "v0.40.0-beta"

[settings]
cachePath = ""
defaultChain = "mainnet"
defaultGateway = "https://ipfs.unchainedindex.io/ipfs/"
indexPath = ""

[keys]

[keys.etherscan]
apiKey = ""

[keys.pinata]
apiKey = ""
jwt = ""
secret = ""

#[dev]
#debug_curl=true

[chains]

[chains.mainnet]
apiProvider = "http://localhost:8080"
chainId = "1"
localExplorer = "http://localhost:1234"
remoteExplorer = "https://etherscan.io"
rpcProvider = "http://localhost:8545"
symbol = "ETH"

Here we can see a few different settings, let’s go over them starting with the most important ones:

  • rpcProvider – This is probably the most important setting, you need to point this at an Ethereum node that supports Trace methods (i.e. Erigon, Nethermind?)
  • keys.etherscan.apiKey – You’ll want to acquire an Etherscan API key to download contract ABI data if you want the --articulate flag to work. It’s not required but useful.
  • defaultGateway – If you have an IPFS gateway – use it here otherwise use the default!

The rest of the settings can be ignored but here’s a breakdown of the other settings:

  • cachePath – The folder location for caching of objects this is by default ~/.local/share/trueblocks/cache or ~/Library/Application\ Support/TrueBlocks/cache (empty string is default)
  • indexPath – The folder location for the main index, this is by default ~/.local/share/trueblocks/unchained or ~/Library/Application\ Support/TrueBlocks/unchained (empty string is default)
  • localExporer – This is used for the TrueBlocks-Explorer – We haven’t talked about this yet, you can ignore it for now. I might write another blog post later?
  • apiProvider – This setting points to the TrueBlocks server known as chifra serve – you can leave this as is for now, because we haven’t started the server yet.

How to use this file with Docker

One of the ways you can use/modify this file with docker is to simply generate it, I typically use something like /data/volumes/ to mount docker files, so I’d do something like this:

mkdir -p /data/volumes/trueblocks

docker run trueblocks-core:develop cat /root/.local/share/trueblocks/trueBlocks.toml > /data/volumes/trueblocks/trueBlocks.toml

Now I have the trueBlocks.toml file on my system to edit as I see fit and I can mount it to the container whenever I run it, e.g.

docker run -v /data/volumes/trueblocks/trueBlocks.toml:/root/.local/share/trueblocks/trueBlocks.toml trueblocks-core:develop chifra status

Initializing TrueBlocks

TrueBlocks has already done almost all of the work of indexing the chain for us, and hosts the bloom filters and the index itself on IPFS, so the easiest way to get up and running is with chifra init – which will download all of the bloom filters. You can also run chifra init --all to download the entire index as well. By only downloading the bloom filters, you only download what you need to locate appearances of addresses, which is like 3GB of bloom filters. However, if you download the entire index you will also download all of the actual appearance data as well, so instead of scanning the bloom filters, then downloading the parts of the index that interest you, you can just download the whole index. The whole index is like 80GB right now. So it’s up to you.

Running native you can just run chifra init

Running in Docker you’ll want to do something like this:

docker run -v /data/volumes/trueblocks/trueBlocks.toml:/root/.local/share/trueblocks/trueBlocks.toml -v /data/volumes/trueblocks/cache:/root/.local/share/trueblocks/cache -v /data/volumes/trueblocks/unchained:/root/.local/share/trueblocks/unchained trueblocks-core:develop chifra init

This command looks complicated but we’re just mounting the config, mounting a cache, and mounting an index folder so our data is persistent. You can of course use Docker Volumes if that’s more your style.

Using TrueBlocks

There are two primary ways to use TrueBlocks – the CLI and an API

CLI

You can just run CLI commands now like chifra list dude.eth and you will be presented with a list of all appearances of dude.eth on the blockchain. It’s like magic.

How long this takes the first time you run it depends on if you downloaded the entire index or not. If you didn’t download the entire index, it will scan the bloom filters and download the appropriate index pieces from IPFS. If you already downloaded the entire index then it will return a lot faster.

The second time you run this, it will all come from a local cache and it will be very fast. That’s the beauty of TrueBlocks – the more you use it the faster it becomes, but the more storage it will consume as well.

Check out chifra --help for more options.

Chifra Serve (API)

We can also run an API service via chifra serve see https://trueblocks.io/api/ for more details

chifra serve --port :8080

Now you can query stuff like:

curl 'http://localhost:8080/blocks?blocks=123'

Sick right?

Keeping Updated with the Blockchain

You’re probably wondering how TrueBlocks parses and indexes new blocks right? Well there’s a service for that too!

chifra scrape --sleep=12 --pin

This will run a block scraper that will talk to your tracing capable endpoint and continually update your cache and index. Once the merge takes place block times will be exactly 12 seconds, so it’s a solid number for the sleep timer on ETH Mainnet. The --pin option works if you’re running IPFS on localhost:5001

Doing it all in Docker / Docker-Compose

Because I like myself some Docker, I put all this together in Docker-Compose on my system and it looked a little something like this:

version: '3.4'

services:
  trueblocks-serve:
    container_name: trueblocks-serve
    image: trueblocks-core:develop
    build: ./github/trueblocks-core
    restart: unless-stopped
    volumes:
      - /data/volumes/trueblocks/trueBlocks.toml:/root/.local/share/trueblocks/trueBlocks.toml
      - /data/volumes/trueblocks/cache:/root/.local/share/trueblocks/cache
      - /data/volumes/trueblocks/unchained:/root/.local/share/trueblocks/unchained
    command:
      - chifra
      - serve
      - --port=:8080

  trueblocks-scrape:
    container_name: trueblocks-scrape
    image: trueblocks-core:develop
    build: ./github/trueblocks-core
    restart: unless-stopped
    volumes:
      - /data/volumes/trueblocks/trueBlocks.toml:/root/.local/share/trueblocks/trueBlocks.toml
      - /data/volumes/trueblocks/cache:/root/.local/share/trueblocks/cache
      - /data/volumes/trueblocks/unchained:/root/.local/share/trueblocks/unchained
    command:
      - chifra
      - scrape
      - --sleep=12

If you remember earlier I gave a couple of docker run commands to generate the the config file, and initialize with chifra init just run those before you docker-compose up -d this and you’re good to go. You now have a scraper keeping TrueBlocks updated, and you have chifra serve providing you the API.

How you deal with ports and firewalls, etc. is on you to figure out 🙂

Published inTech

Be First to Comment

Leave a Reply

Your email address will not be published. Required fields are marked *