GitHub - ethpandaops/eth-client-docker-image-builder: automates some docker builds for ethereum clients

Automates docker builds for ethereum clients. The build process is scheduled every hour to check source repositories for new commits.

Build image on demand

Run the Build client workflow;

Run the Build tooling workflow;

Build Eth-DAS-Guardian [source]
Build Flashbots Builder [source]
Build tx-fuzz [source]
Build consesnus-monitor [source]
Build execution-monitor [source]
Build beacon-metrics-gazer [source]
Build goomy-blob [source]
Build ethereum-genesis-generator [source]
Build mev-rs [source]
Build reth-rbuilder [source]
Build Syncoor [source]
Build Slashoor [source]
Build ere-server-zisk [source]

Adding a new image to build on schedule

Add a new image to config.yaml file and it will be built on schedule from this workflow.

- source:
    repository: sigp/lighthouse # source repository to build from
    ref: stable # source repository branch/tag/commit to build from
  build_script: ./teku/build.sh # optional build script to run INSTEAD of the docker build & push (see below)
  target:
    tag: stable # tag to add to the docker image tag, this must be unique for each docker hub repository
    repository: ethpandaops/lighthouse # dockerhub target to deploy the built image
    dockerfile: ./lighthouse/Dockerfile # optional docker file to use, defaults to the source repository's Dockerfile

Output image tags

Take the following config;

- source:
    repository: sigp/lighthouse
    ref: stable
  target:
    tag: banana
    repository: ethpandaops/lighthouse

This would produce the following docker image tags;

# the tag by itself to have the latest build
ethpandaops/lighthouse:banana
# the tag and the source repository's commit hash
ethpandaops/lighthouse:banana-abcd123

How does the `build_script` work?

The build_script is a bash script that is run INSTEAD of the docker build & push. This is useful for clients that have a custom build process.

When the build_script is set, you must build and push the docker image yourself! Docker will already be logged in to the target repository. You should try to use the target_tag and target_repository environment variables to tag your image.

The following environment variables are available to the build_script;

source_repository - source repository to build from
source_ref - source repository branch/tag/commit to build from
target_tag - tag to add to the docker image tag
target_repository - dockerhub target to deploy the built image
target_dockerfile - optional docker file to use, defaults to the source repository's Dockerfile
source_git_commit_hash - the source repository's short commit hash
source_git_commit_hash_full - the source repository's full commit hash

Example build_script file;

#!/bin/bash

# helper to get source directory
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
cd ${SCRIPT_DIR}/../source

# do something here that requires this custom build script
# ...

# finally build with the tags from the dockerfile
docker build -t "${target_repository}:${target_tag}" -t "${target_repository}:${target_tag}-${source_git_commit_hash}" -f "../${target_dockerfile}" .

# push the image tags
docker push "${target_repository}:${target_tag}"
docker push "${target_repository}:${target_tag}-${source_git_commit_hash}"

Additional Configuration Files

Our image building process utilizes two additional configuration files: platforms.yaml and runners.yaml. These files help in determining the platforms for which docker images should be built and specifying the runners to use for those platforms, respectively.

`platforms.yaml`

This configuration determines the platforms for which each client will have a Docker image built.

Sample Content:

besu:
  - linux/amd64
lighthouse:
  - linux/amd64
  - linux/arm64

In the example above, the client 'besu' and 'lighthouse' are both configured to have Docker images built for the linux/amd64 platform. While 'lighthouse' is also configured to have Docker images built for the linux/arm64 platform.

`runners.yaml`

This configuration maps platforms to GitHub Action runners. It tells our workflow which runner should be used when building a Docker image for a specific platform.

Sample Content:

linux/amd64: ubuntu-latest
linux/arm64: self-hosted

In this example, the platform linux/amd64 will use the ubuntu-latest runner, while darwin/arm64 will use the self-hosted runner.

Lint locally

Requirements;

Python 3.6+
Yamale
yq

# make sure yamale is installed
pip install yamale

# yamale lint
yamale -s schema.yaml config.yaml

# check unique target tag, should return []
yq 'group_by(.target.repository + ":" + .target.tag) | map(select(length>1))' config.yaml