feat: add weighted rpc steering by DaMandal0rian · Pull Request #6090 · graphprotocol/graph-node

@DaMandal0rian DaMandal0rian changed the base branch from master to feature/add-weighted-random-steering-load-balancing

August 11, 2025 00:30

@DaMandal0rian

…ements

This commit introduces a complete weighted load balancing system for RPC endpoints
with traffic distribution based on configurable provider weights (0.0-1.0).

## Core Features

### Weighted Load Balancing Algorithm
- Implements probabilistic selection using WeightedIndex from rand crate
- Supports decimal weights (0.0-1.0) for precise traffic distribution
- Weights are relative and don't need to sum to 1.0 (normalized internally)
- Graceful fallback to random selection if weights are invalid

### Enhanced Error Handling & Resilience
- Improved error retesting logic that preserves weight distribution
- Error retesting now occurs AFTER weight-based selection to minimize skew
- Maintains existing failover capabilities while respecting configured weights
- Robust handling of edge cases (all zero weights, invalid configurations)

### Configuration & Validation
- Added `weighted_rpc_steering` flag to enable/disable weighted selection
- Provider weight validation ensures values are between 0.0 and 1.0
- Validation prevents all-zero weight configurations
- Comprehensive configuration documentation with usage examples

## Implementation Details

### Network Layer Changes (chain/ethereum/src/network.rs)
- Refactored adapter selection into modular, well-documented functions:
  - `select_best_adapter()`: Chooses between weighted/random strategies
  - `select_weighted_adapter()`: Implements WeightedIndex-based selection
  - `select_random_adapter()`: Enhanced random selection with error consideration
- Added comprehensive inline documentation explaining algorithms
- Maintains thread safety with proper Arc usage and thread-safe RNG
- Added test coverage for weighted selection with statistical validation

### Configuration System (node/src/config.rs)
- Extended Provider struct with f64 weight field (default: 1.0)
- Added weight validation in Provider::validate() method
- Added Chain-level validation to prevent all-zero weight configurations
- Integrated with existing configuration validation pipeline

### CLI & Setup Integration
- Added --weighted-rpc-steering command line flag (node/src/opt.rs)
- Integrated weighted flag through network setup pipeline (node/src/network_setup.rs)
- Updated chain configuration to pass weight values to adapters (node/src/chain.rs)

### Documentation & Examples
- Added comprehensive configuration documentation in full_config.toml
- Includes weight range explanation, distribution examples, and usage guidelines
- Clear examples showing relative weight calculations and traffic distribution

## Technical Improvements

### Dependency Management
- Updated rand dependency to use appropriate version with WeightedIndex support
- Proper import paths for rand 0.9 distribution modules
- Fixed compilation issues with correct trait imports (Distribution)

### Code Quality & Maintenance
- Comprehensive inline documentation for all weight-related methods
- Clear separation of concerns with single-responsibility functions
- Maintained backward compatibility with existing random selection
- Added statistical test validation for weight distribution accuracy

## Validation & Testing

- Comprehensive test suite validates weight distribution over 1000 iterations
- Statistical validation with 10% tolerance for weight accuracy
- All existing tests continue to pass, ensuring no regression
- Build verification across all affected packages

## Configuration Example

```toml
weighted_rpc_steering = true

[chains.mainnet]
provider = [
  { label = "primary", url = "http://rpc1.io", weight = 0.7 },   # 70% traffic
  { label = "backup", url = "http://rpc2.io", weight = 0.3 },    # 30% traffic
]
```

This implementation provides production-ready weighted load balancing with
robust error handling, comprehensive validation, and excellent maintainability.

🤖 Generated with Claude Code

DaMandal0rian added a commit that referenced this pull request

Jan 31, 2026
…ements (#6090)

This commit introduces a complete weighted load balancing system for RPC endpoints
with traffic distribution based on configurable provider weights (0.0-1.0).

- Implements probabilistic selection using WeightedIndex from rand crate
- Supports decimal weights (0.0-1.0) for precise traffic distribution
- Weights are relative and don't need to sum to 1.0 (normalized internally)
- Graceful fallback to random selection if weights are invalid

- Improved error retesting logic that preserves weight distribution
- Error retesting now occurs AFTER weight-based selection to minimize skew
- Maintains existing failover capabilities while respecting configured weights
- Robust handling of edge cases (all zero weights, invalid configurations)

- Added `weighted_rpc_steering` flag to enable/disable weighted selection
- Provider weight validation ensures values are between 0.0 and 1.0
- Validation prevents all-zero weight configurations
- Comprehensive configuration documentation with usage examples

- Refactored adapter selection into modular, well-documented functions:
  - `select_best_adapter()`: Chooses between weighted/random strategies
  - `select_weighted_adapter()`: Implements WeightedIndex-based selection
  - `select_random_adapter()`: Enhanced random selection with error consideration
- Added comprehensive inline documentation explaining algorithms
- Maintains thread safety with proper Arc usage and thread-safe RNG
- Added test coverage for weighted selection with statistical validation

- Extended Provider struct with f64 weight field (default: 1.0)
- Added weight validation in Provider::validate() method
- Added Chain-level validation to prevent all-zero weight configurations
- Integrated with existing configuration validation pipeline

- Added --weighted-rpc-steering command line flag (node/src/opt.rs)
- Integrated weighted flag through network setup pipeline (node/src/network_setup.rs)
- Updated chain configuration to pass weight values to adapters (node/src/chain.rs)

- Added comprehensive configuration documentation in full_config.toml
- Includes weight range explanation, distribution examples, and usage guidelines
- Clear examples showing relative weight calculations and traffic distribution

- Updated rand dependency to use appropriate version with WeightedIndex support
- Proper import paths for rand 0.9 distribution modules
- Fixed compilation issues with correct trait imports (Distribution)

- Comprehensive inline documentation for all weight-related methods
- Clear separation of concerns with single-responsibility functions
- Maintained backward compatibility with existing random selection
- Added statistical test validation for weight distribution accuracy

- Comprehensive test suite validates weight distribution over 1000 iterations
- Statistical validation with 10% tolerance for weight accuracy
- All existing tests continue to pass, ensuring no regression
- Build verification across all affected packages

```toml
weighted_rpc_steering = true

[chains.mainnet]
provider = [
  { label = "primary", url = "http://rpc1.io", weight = 0.7 },   # 70% traffic
  { label = "backup", url = "http://rpc2.io", weight = 0.3 },    # 30% traffic
]
```

This implementation provides production-ready weighted load balancing with
robust error handling, comprehensive validation, and excellent maintainability.

🤖 Generated with Claude Code

DaMandal0rian added a commit that referenced this pull request

Feb 25, 2026
…ements (#6090)

This commit introduces a complete weighted load balancing system for RPC endpoints
with traffic distribution based on configurable provider weights (0.0-1.0).

- Implements probabilistic selection using WeightedIndex from rand crate
- Supports decimal weights (0.0-1.0) for precise traffic distribution
- Weights are relative and don't need to sum to 1.0 (normalized internally)
- Graceful fallback to random selection if weights are invalid

- Improved error retesting logic that preserves weight distribution
- Error retesting now occurs AFTER weight-based selection to minimize skew
- Maintains existing failover capabilities while respecting configured weights
- Robust handling of edge cases (all zero weights, invalid configurations)

- Added `weighted_rpc_steering` flag to enable/disable weighted selection
- Provider weight validation ensures values are between 0.0 and 1.0
- Validation prevents all-zero weight configurations
- Comprehensive configuration documentation with usage examples

- Refactored adapter selection into modular, well-documented functions:
  - `select_best_adapter()`: Chooses between weighted/random strategies
  - `select_weighted_adapter()`: Implements WeightedIndex-based selection
  - `select_random_adapter()`: Enhanced random selection with error consideration
- Added comprehensive inline documentation explaining algorithms
- Maintains thread safety with proper Arc usage and thread-safe RNG
- Added test coverage for weighted selection with statistical validation

- Extended Provider struct with f64 weight field (default: 1.0)
- Added weight validation in Provider::validate() method
- Added Chain-level validation to prevent all-zero weight configurations
- Integrated with existing configuration validation pipeline

- Added --weighted-rpc-steering command line flag (node/src/opt.rs)
- Integrated weighted flag through network setup pipeline (node/src/network_setup.rs)
- Updated chain configuration to pass weight values to adapters (node/src/chain.rs)

- Added comprehensive configuration documentation in full_config.toml
- Includes weight range explanation, distribution examples, and usage guidelines
- Clear examples showing relative weight calculations and traffic distribution

- Updated rand dependency to use appropriate version with WeightedIndex support
- Proper import paths for rand 0.9 distribution modules
- Fixed compilation issues with correct trait imports (Distribution)

- Comprehensive inline documentation for all weight-related methods
- Clear separation of concerns with single-responsibility functions
- Maintained backward compatibility with existing random selection
- Added statistical test validation for weight distribution accuracy

- Comprehensive test suite validates weight distribution over 1000 iterations
- Statistical validation with 10% tolerance for weight accuracy
- All existing tests continue to pass, ensuring no regression
- Build verification across all affected packages

```toml
weighted_rpc_steering = true

[chains.mainnet]
provider = [
  { label = "primary", url = "http://rpc1.io", weight = 0.7 },   # 70% traffic
  { label = "backup", url = "http://rpc2.io", weight = 0.3 },    # 30% traffic
]
```

This implementation provides production-ready weighted load balancing with
robust error handling, comprehensive validation, and excellent maintainability.

🤖 Generated with Claude Code