Noob Guide for Backtesting

Prerequisite: Rust

Install Rust following this guide: https://www.rust-lang.org/tools/install

Prerequisite: Reth node

You need a synced reth (beta) node (preferably with a CL client, i.e. Lighthouse). Please see the respective manuals to set that up:

You'll need local access to the Reth database files.

We recommend to run Reth + CL on a server, and setup a remote development environment there (VS Code remote development works very well for this). You just need SSH access to the server.

Working on a remote server

Clone the rbuilder repository: git clone git@github.com:flashbots/rbuilder.git rbuilder-YOURNAME
Open that directory with VS Code Remote Development
Install VS Code extensions
You should be able to run make build now

Compile rbuilder

make build

`make build` builds the codebase and produces a number of binaries in `target/debug/`.

# build it
$ make build

# show the artifacts
$ ls -alh target/debug/
total 14G
drwxrwxr-x    7 ubuntu ubuntu 4.0K Jun 20 20:59 .
drwxrwxr-x    3 ubuntu ubuntu 4.0K Jun 20 18:53 ..
-rw-rw-r--    1 ubuntu ubuntu    0 Jun 20 18:52 .cargo-lock
drwxrwxr-x 2520 ubuntu ubuntu 128K Jun 20 21:00 .fingerprint
-rwxrwxr-x    2 ubuntu ubuntu 1.2G Jun 20 20:59 backtest-build-block
-rw-rw-r--    1 ubuntu ubuntu 6.4K Jun 20 18:53 backtest-build-block.d
-rwxrwxr-x    2 ubuntu ubuntu 1.2G Jun 20 20:59 backtest-build-range
-rw-rw-r--    1 ubuntu ubuntu 6.4K Jun 20 18:53 backtest-build-range.d
-rwxrwxr-x    2 ubuntu ubuntu 1.3G Jun 20 20:59 backtest-fetch
-rw-rw-r--    1 ubuntu ubuntu 6.4K Jun 20 18:53 backtest-fetch.d
drwxrwxr-x  278 ubuntu ubuntu  20K Jun 20 21:00 build
-rwxrwxr-x    2 ubuntu ubuntu 1.2G Jun 20 20:59 debug-bench-machine
-rw-rw-r--    1 ubuntu ubuntu 6.4K Jun 20 18:53 debug-bench-machine.d
-rwxrwxr-x    2 ubuntu ubuntu 1.2G Jun 20 20:59 debug-fake-order-sim
-rw-rw-r--    1 ubuntu ubuntu 6.4K Jun 20 18:53 debug-fake-order-sim.d
-rwxrwxr-x    2 ubuntu ubuntu 1.2G Jun 20 20:59 debug-order-input
-rw-rw-r--    1 ubuntu ubuntu 6.4K Jun 20 18:53 debug-order-input.d
-rwxrwxr-x    2 ubuntu ubuntu 1.2G Jun 20 20:59 debug-order-sim
-rw-rw-r--    1 ubuntu ubuntu 6.4K Jun 20 18:53 debug-order-sim.d
-rwxrwxr-x    2 ubuntu ubuntu 1.2G Jun 20 20:59 debug-slot-data-generator
-rw-rw-r--    1 ubuntu ubuntu 6.4K Jun 20 18:53 debug-slot-data-generator.d
drwxrwxr-x    3 ubuntu ubuntu 596K Jun 20 21:02 deps
-rwxrwxr-x    2 ubuntu ubuntu 1.2G Jun 20 20:59 dummy-builder
-rw-rw-r--    1 ubuntu ubuntu 6.4K Jun 20 18:53 dummy-builder.d
drwxrwxr-x    2 ubuntu ubuntu 4.0K Jun 20 18:52 examples
drwxrwxr-x  109 ubuntu ubuntu  12K Jun 20 21:00 incremental
-rw-rw-r--    1 ubuntu ubuntu 6.3K Jun 20 18:53 librbuilder.d
-rw-rw-r--    2 ubuntu ubuntu 153M Jun 20 20:59 librbuilder.rlib
-rwxrwxr-x    2 ubuntu ubuntu 1.1G Jun 20 20:59 misc-relays-slot
-rw-rw-r--    1 ubuntu ubuntu 6.4K Jun 20 18:53 misc-relays-slot.d
-rwxrwxr-x    2 ubuntu ubuntu 1.2G Jun 20 20:59 rbuilder
-rw-rw-r--    1 ubuntu ubuntu 6.4K Jun 20 18:53 rbuilder.d
-rwxrwxr-x    2 ubuntu ubuntu 1.1G Jun 20 20:59 relay-sender
-rw-rw-r--    1 ubuntu ubuntu 6.4K Jun 20 18:53 relay-sender.d

Config file

All settings are configured in a toml file, for instance config-backtest-example.toml.

Make a copy of config-backtest-example.toml:

cp config-backtest-example.toml config-backtest.toml

The only thing you need to change is the reth database location (set to path used with reth --datadir <path>):

reth_datadir = "/mnt/data/reth"

Running the backtest

First, choose a block number you want to backtest for.

The backtesting setup uses Mempool Dumpster to download mempool transactions.
Mempool Dumpster updates the archive nightly, so you can't download today's transactions.
Pick a block number that's a day or two ago:
- go to https://etherscan.io, pick the latest block number, and subtract 20k from it (a bit more than 2 days)

Then download the transaction archive:

./target/debug/backtest-fetch --config config-backtest.toml fetch 20114954

Now you can run the backtest like this:

./target/debug/backtest-build-block --config config-backtest.toml 20114954 
./target/debug/backtest-build-block --config config-backtest.toml 20114954 --help

You can also use backtest-build-range to backtest for many blocks at once.

Next steps

Changing the building algorithm

(and other settings)

todo