What is TrueBlocks?
TrueBlocks is a lightweight, ultra performance indexing solution for EVM chains. It was originally created due to many limitations in the JSON-RPC which prevents easy access to simple data.
Have you ever wanted to get all of the transactions involving a single address out of an Ethereum node? Good luck. Have you ever wanted to grab all the logs from a single contract out of an Ethereum node? Again, good luck.
While the Erigon client has made great strides in improving the performance of the JSON-RPC, it still has its limits, and it’s still very inefficient to find some of this data. That’s where TrueBlocks comes in. TrueBlocks creates an offline index of every single appearance of every single address, ever, on the blockchain. It does this by running Trace calls against every single transaction in every single block and simply creating a record of any time an address appears. It’s even more accurate than Etherscan.
If you’re interesting in learning more or becoming a part of the TrueBlocks community head on over to their Discord.
There are two ways to build TrueBlocks, native and Docker, I’ll cover both real quick.
First head on over to the GitHub, clone the repo:
git clone -b develop https://github.com/trueblocks/trueblocks-core cd trueblocks-core
Building with Docker
I’ll assume you already have docker installed here…
docker build . --tag=trueblocks-core:develop
sudo apt-get install -y build-essential cmake python3 python3-dev libcurl3-dev clang-format jq mkdir build && cd build cmake ../src make -j 5
That’s is, congrats, you built TrueBlocks. But the fun is just beginning. The first thing you should do is add
trueblocks-core/bin to your path (if you’re running natively)
The first thing to note is that if you did the native build, TrueBlocks installed some executables on your system and generated some config files. The binaries are located at
~/.local/bin/chifra and then there are configuration files under
~/.local/share/trueblocks for Linux and
~/Library/Application\ Support/TrueBlocks on macs, but I’m on Linux right now so I’m just going to continue using that path.
The most important file is probably
~/.local/share/trueblocks/trueBlocks.toml this is your TrueBlocks configuration file. We need to populate some fields before it’s going to work.
I’m only going to focus on Mainnet for now:
cat ~/.local/share/trueblocks/trueBlocks.toml [version] current = "v0.40.0-beta" [settings] cachePath = "" defaultChain = "mainnet" defaultGateway = "https://ipfs.unchainedindex.io/ipfs/" indexPath = "" [keys] [keys.etherscan] apiKey = "" [keys.pinata] apiKey = "" jwt = "" secret = "" #[dev] #debug_curl=true [chains] [chains.mainnet] apiProvider = "http://localhost:8080" chainId = "1" localExplorer = "http://localhost:1234" remoteExplorer = "https://etherscan.io" rpcProvider = "http://localhost:8545" symbol = "ETH"
Here we can see a few different settings, let’s go over them starting with the most important ones:
- rpcProvider – This is probably the most important setting, you need to point this at an Ethereum node that supports Trace methods (i.e. Erigon, Nethermind?)
- keys.etherscan.apiKey – You’ll want to acquire an Etherscan API key to download contract ABI data if you want the
--articulateflag to work. It’s not required but useful.
- defaultGateway – If you have an IPFS gateway – use it here otherwise use the default!
The rest of the settings can be ignored but here’s a breakdown of the other settings:
- cachePath – The folder location for caching of objects this is by default
~/Library/Application\ Support/TrueBlocks/cache(empty string is default)
- indexPath – The folder location for the main index, this is by default
~/Library/Application\ Support/TrueBlocks/unchained(empty string is default)
- localExporer – This is used for the TrueBlocks-Explorer – We haven’t talked about this yet, you can ignore it for now. I might write another blog post later?
- apiProvider – This setting points to the TrueBlocks server known as
chifra serve– you can leave this as is for now, because we haven’t started the server yet.
How to use this file with Docker
One of the ways you can use/modify this file with docker is to simply generate it, I typically use something like
/data/volumes/ to mount docker files, so I’d do something like this:
mkdir -p /data/volumes/trueblocks docker run trueblocks-core:develop cat /root/.local/share/trueblocks/trueBlocks.toml > /data/volumes/trueblocks/trueBlocks.toml
Now I have the
trueBlocks.toml file on my system to edit as I see fit and I can mount it to the container whenever I run it, e.g.
docker run -v /data/volumes/trueblocks/trueBlocks.toml:/root/.local/share/trueblocks/trueBlocks.toml trueblocks-core:develop chifra status
TrueBlocks has already done almost all of the work of indexing the chain for us, and hosts the bloom filters and the index itself on IPFS, so the easiest way to get up and running is with
chifra init – which will download all of the bloom filters. You can also run
chifra init --all to download the entire index as well. By only downloading the bloom filters, you only download what you need to locate appearances of addresses, which is like 3GB of bloom filters. However, if you download the entire index you will also download all of the actual appearance data as well, so instead of scanning the bloom filters, then downloading the parts of the index that interest you, you can just download the whole index. The whole index is like 80GB right now. So it’s up to you.
Running native you can just run
Running in Docker you’ll want to do something like this:
docker run -v /data/volumes/trueblocks/trueBlocks.toml:/root/.local/share/trueblocks/trueBlocks.toml -v /data/volumes/trueblocks/cache:/root/.local/share/trueblocks/cache -v /data/volumes/trueblocks/unchained:/root/.local/share/trueblocks/unchained trueblocks-core:develop chifra init
This command looks complicated but we’re just mounting the config, mounting a cache, and mounting an index folder so our data is persistent. You can of course use Docker Volumes if that’s more your style.
There are two primary ways to use TrueBlocks – the CLI and an API
You can just run CLI commands now like
chifra list dude.eth and you will be presented with a list of all appearances of
dude.eth on the blockchain. It’s like magic.
How long this takes the first time you run it depends on if you downloaded the entire index or not. If you didn’t download the entire index, it will scan the bloom filters and download the appropriate index pieces from IPFS. If you already downloaded the entire index then it will return a lot faster.
The second time you run this, it will all come from a local cache and it will be very fast. That’s the beauty of TrueBlocks – the more you use it the faster it becomes, but the more storage it will consume as well.
chifra --help for more options.
Chifra Serve (API)
We can also run an API service via
chifra serve see https://trueblocks.io/api/ for more details
chifra serve --port :8080
Now you can query stuff like:
Keeping Updated with the Blockchain
You’re probably wondering how TrueBlocks parses and indexes new blocks right? Well there’s a service for that too!
chifra scrape --sleep=12 --pin
This will run a block scraper that will talk to your tracing capable endpoint and continually update your cache and index. Once the merge takes place block times will be exactly 12 seconds, so it’s a solid number for the sleep timer on ETH Mainnet. The
--pin option works if you’re running IPFS on
Doing it all in Docker / Docker-Compose
Because I like myself some Docker, I put all this together in Docker-Compose on my system and it looked a little something like this:
version: '3.4' services: trueblocks-serve: container_name: trueblocks-serve image: trueblocks-core:develop build: ./github/trueblocks-core restart: unless-stopped volumes: - /data/volumes/trueblocks/trueBlocks.toml:/root/.local/share/trueblocks/trueBlocks.toml - /data/volumes/trueblocks/cache:/root/.local/share/trueblocks/cache - /data/volumes/trueblocks/unchained:/root/.local/share/trueblocks/unchained command: - chifra - serve - --port=:8080 trueblocks-scrape: container_name: trueblocks-scrape image: trueblocks-core:develop build: ./github/trueblocks-core restart: unless-stopped volumes: - /data/volumes/trueblocks/trueBlocks.toml:/root/.local/share/trueblocks/trueBlocks.toml - /data/volumes/trueblocks/cache:/root/.local/share/trueblocks/cache - /data/volumes/trueblocks/unchained:/root/.local/share/trueblocks/unchained command: - chifra - scrape - --sleep=12
If you remember earlier I gave a couple of
docker run commands to generate the the config file, and initialize with
chifra init just run those before you
docker-compose up -d this and you’re good to go. You now have a scraper keeping TrueBlocks updated, and you have
chifra serve providing you the API.
How you deal with ports and firewalls, etc. is on you to figure out 🙂