Load Balancing Freemium Ethereum Endpoints Pt.2

When we last left off on Part 1 we had a very basic Dshackle instance running in docker that could do some basic caching and load balance between Infura and Chainstack.

Now we’ll dig into a more complex setup, the setup I have at home! Now, I’m pretty new to Docker and containerization, and this certainly isn’t my day job so if you’re an expert at this stuff, let me know what I’m doing wrong 🙂

Docker-Compose

In the Part 1 we went though installing Docker, but I’m going to go a step further here and also install Docker Compose. It’s really complicated so follow along:

# Latest Docker Compose Version
# https://docs.docker.com/compose/install/
COMPOSEVERSION="1.28.5"

# Download the binary
sudo curl -L "https://github.com/docker/compose/releases/download/${COMPOSEVERSION}/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

# Make binary executable
sudo chmod +x /usr/local/bin/docker-compose

Done.

Expanding Our Dshackle Config

One of the first things you’ll notice when you try to use Dshackle is that by default it will only accept a specific set of JSON-RPC methods. You can find that list HERE.

Luckily, Dshackle allows us to both enable and disable methods! None of the default methods are all that dangerous, so there’s not a lot of reasons to disable a method (unless it’s not supported by your client).

The problem is that each client supports different methods, and each of these freemium services has different endpoints they support. Fear not, for I have created custom configs for all of the providers I know about. It might not be 100% accurate (free free to submit a pull request), but it should get you started.

You can find the configs over here: https://github.com/MysticRyuujin/dshackle-configs

So let’s assume you downloaded those files and look at docker-compose.yml

version: '3'
services:
  dshackle:
    container_name: dshackle
    image: emeraldpay/dshackle:0.9.1
    restart: unless-stopped
    volumes:
      - /data/dshackle:/etc/dshackle
    ports:
      - "8080:8080"
    environment:
      - ALCHEMY_KEY=
      - ANYBLOCK_KEY=
      - CHAINSTACK_NODE=
      - CHAINSTACK_USERNAME=
      - CHAINSTACK_PASSWORD=
      - INFURA_KEY=
      - RIVET_KEY=
  redis:
    container_name: redis
    image: redis:alpine
    restart: unless-stopped

What’s that?? A Redis container too!? Yupppp, why not. In case you’re not familiar with Redis, it’s an in-memory caching service. Which let’s dshackle cache even more data. Be warned though that it does consume memory, so that 2GB VPS might not be enough if you run with Redis. You totally don’t have to.

The key thing to note above is I have a volume mounted that is /data/dshackle which is where my dshackle.yaml file sits as well as my custom configs for each of my nodes. It’s basically just the clone of the repo linked above.

Also, I have Environment variable placeholders for the various API keys that the Freemium services require. Be sure to fill those in.

So what does the dshackle.yaml file look like? This:

host: 0.0.0.0
port: 2449
tls:
  enabled: false

cache:
  redis:
    enabled: true
    host: redis

proxy:
  host: 0.0.0.0
  port: 8080
  tls:
    enabled: false
  routes:
    - id: eth
      blockchain: ethereum

cluster:
  defaults:
    - chains:
        - ethereum
      options:
        min-peers: 10
  include:
    # Local Nodes
    - "geth.yaml"
    - "nethermind.yaml"
    - "turbogeth.yaml"
    # Freemium Nodes
    - "alchemy.yaml"
    - "anyblock.yaml"
    - "chainstack.yaml"
    - "infura.yaml"
    - "rivet.yaml"
    # Public Nodes
    - "cloudflare.yaml"
    - "avado.yaml"

Now, the Local Nodes are just placeholders here, you can remove them if you don’t have any nodes of your own and you only want to rely on the Freemium providers. But I thought I would include them in here because I put in a lot of work figuring out all the methods to enable or disable and I thought I’d share it with ya’ll.

WARNING: Some of the methods exposed on your the Local Nodes can be dangerous. Don’t expose it to the internet, or disable all of the methods that might be dangerous. I’ll touch more on this in Part 3 if you want know details!

I’ve added a special key to the configuration files of all the non-local nodes role: fallback which instructs Dshackle to only use these nodes as a last resort. If you have local nodes, I would assume you’d always want to talk to them first, but if you’re only using Freemium services, they’ll all be treated equally anyway.

You’ll also notice that I’ve included 2 free public endpoints. Cloudflare and Avado. Because, why not?

Dshackle will look in /etc/dshackle/ folder for both its config and these included files.

That’s pretty much it…that’s a fully configured Dshackle with a redis cache. You can start it up with docker-compose up -d

You can check the logs with docker logs dshackle

I think this post is long enough for now. I’ll come back with a Part 3 and I’ll cover some of the pitfalls and tackle some of the limitations of Dshackle with NGINX.

PART 3 HERE