feat: add weighted rpc steering by DaMandal0rian · Pull Request #6090 · graphprotocol/graph-node
DaMandal0rian
changed the base branch from
master
to
feature/add-weighted-random-steering-load-balancing
…ements
This commit introduces a complete weighted load balancing system for RPC endpoints
with traffic distribution based on configurable provider weights (0.0-1.0).
## Core Features
### Weighted Load Balancing Algorithm
- Implements probabilistic selection using WeightedIndex from rand crate
- Supports decimal weights (0.0-1.0) for precise traffic distribution
- Weights are relative and don't need to sum to 1.0 (normalized internally)
- Graceful fallback to random selection if weights are invalid
### Enhanced Error Handling & Resilience
- Improved error retesting logic that preserves weight distribution
- Error retesting now occurs AFTER weight-based selection to minimize skew
- Maintains existing failover capabilities while respecting configured weights
- Robust handling of edge cases (all zero weights, invalid configurations)
### Configuration & Validation
- Added `weighted_rpc_steering` flag to enable/disable weighted selection
- Provider weight validation ensures values are between 0.0 and 1.0
- Validation prevents all-zero weight configurations
- Comprehensive configuration documentation with usage examples
## Implementation Details
### Network Layer Changes (chain/ethereum/src/network.rs)
- Refactored adapter selection into modular, well-documented functions:
- `select_best_adapter()`: Chooses between weighted/random strategies
- `select_weighted_adapter()`: Implements WeightedIndex-based selection
- `select_random_adapter()`: Enhanced random selection with error consideration
- Added comprehensive inline documentation explaining algorithms
- Maintains thread safety with proper Arc usage and thread-safe RNG
- Added test coverage for weighted selection with statistical validation
### Configuration System (node/src/config.rs)
- Extended Provider struct with f64 weight field (default: 1.0)
- Added weight validation in Provider::validate() method
- Added Chain-level validation to prevent all-zero weight configurations
- Integrated with existing configuration validation pipeline
### CLI & Setup Integration
- Added --weighted-rpc-steering command line flag (node/src/opt.rs)
- Integrated weighted flag through network setup pipeline (node/src/network_setup.rs)
- Updated chain configuration to pass weight values to adapters (node/src/chain.rs)
### Documentation & Examples
- Added comprehensive configuration documentation in full_config.toml
- Includes weight range explanation, distribution examples, and usage guidelines
- Clear examples showing relative weight calculations and traffic distribution
## Technical Improvements
### Dependency Management
- Updated rand dependency to use appropriate version with WeightedIndex support
- Proper import paths for rand 0.9 distribution modules
- Fixed compilation issues with correct trait imports (Distribution)
### Code Quality & Maintenance
- Comprehensive inline documentation for all weight-related methods
- Clear separation of concerns with single-responsibility functions
- Maintained backward compatibility with existing random selection
- Added statistical test validation for weight distribution accuracy
## Validation & Testing
- Comprehensive test suite validates weight distribution over 1000 iterations
- Statistical validation with 10% tolerance for weight accuracy
- All existing tests continue to pass, ensuring no regression
- Build verification across all affected packages
## Configuration Example
```toml
weighted_rpc_steering = true
[chains.mainnet]
provider = [
{ label = "primary", url = "http://rpc1.io", weight = 0.7 }, # 70% traffic
{ label = "backup", url = "http://rpc2.io", weight = 0.3 }, # 30% traffic
]
```
This implementation provides production-ready weighted load balancing with
robust error handling, comprehensive validation, and excellent maintainability.
🤖 Generated with Claude Code
DaMandal0rian added a commit that referenced this pull request
Jan 31, 2026…ements (#6090) This commit introduces a complete weighted load balancing system for RPC endpoints with traffic distribution based on configurable provider weights (0.0-1.0). - Implements probabilistic selection using WeightedIndex from rand crate - Supports decimal weights (0.0-1.0) for precise traffic distribution - Weights are relative and don't need to sum to 1.0 (normalized internally) - Graceful fallback to random selection if weights are invalid - Improved error retesting logic that preserves weight distribution - Error retesting now occurs AFTER weight-based selection to minimize skew - Maintains existing failover capabilities while respecting configured weights - Robust handling of edge cases (all zero weights, invalid configurations) - Added `weighted_rpc_steering` flag to enable/disable weighted selection - Provider weight validation ensures values are between 0.0 and 1.0 - Validation prevents all-zero weight configurations - Comprehensive configuration documentation with usage examples - Refactored adapter selection into modular, well-documented functions: - `select_best_adapter()`: Chooses between weighted/random strategies - `select_weighted_adapter()`: Implements WeightedIndex-based selection - `select_random_adapter()`: Enhanced random selection with error consideration - Added comprehensive inline documentation explaining algorithms - Maintains thread safety with proper Arc usage and thread-safe RNG - Added test coverage for weighted selection with statistical validation - Extended Provider struct with f64 weight field (default: 1.0) - Added weight validation in Provider::validate() method - Added Chain-level validation to prevent all-zero weight configurations - Integrated with existing configuration validation pipeline - Added --weighted-rpc-steering command line flag (node/src/opt.rs) - Integrated weighted flag through network setup pipeline (node/src/network_setup.rs) - Updated chain configuration to pass weight values to adapters (node/src/chain.rs) - Added comprehensive configuration documentation in full_config.toml - Includes weight range explanation, distribution examples, and usage guidelines - Clear examples showing relative weight calculations and traffic distribution - Updated rand dependency to use appropriate version with WeightedIndex support - Proper import paths for rand 0.9 distribution modules - Fixed compilation issues with correct trait imports (Distribution) - Comprehensive inline documentation for all weight-related methods - Clear separation of concerns with single-responsibility functions - Maintained backward compatibility with existing random selection - Added statistical test validation for weight distribution accuracy - Comprehensive test suite validates weight distribution over 1000 iterations - Statistical validation with 10% tolerance for weight accuracy - All existing tests continue to pass, ensuring no regression - Build verification across all affected packages ```toml weighted_rpc_steering = true [chains.mainnet] provider = [ { label = "primary", url = "http://rpc1.io", weight = 0.7 }, # 70% traffic { label = "backup", url = "http://rpc2.io", weight = 0.3 }, # 30% traffic ] ``` This implementation provides production-ready weighted load balancing with robust error handling, comprehensive validation, and excellent maintainability. 🤖 Generated with Claude Code
DaMandal0rian added a commit that referenced this pull request
Feb 25, 2026…ements (#6090) This commit introduces a complete weighted load balancing system for RPC endpoints with traffic distribution based on configurable provider weights (0.0-1.0). - Implements probabilistic selection using WeightedIndex from rand crate - Supports decimal weights (0.0-1.0) for precise traffic distribution - Weights are relative and don't need to sum to 1.0 (normalized internally) - Graceful fallback to random selection if weights are invalid - Improved error retesting logic that preserves weight distribution - Error retesting now occurs AFTER weight-based selection to minimize skew - Maintains existing failover capabilities while respecting configured weights - Robust handling of edge cases (all zero weights, invalid configurations) - Added `weighted_rpc_steering` flag to enable/disable weighted selection - Provider weight validation ensures values are between 0.0 and 1.0 - Validation prevents all-zero weight configurations - Comprehensive configuration documentation with usage examples - Refactored adapter selection into modular, well-documented functions: - `select_best_adapter()`: Chooses between weighted/random strategies - `select_weighted_adapter()`: Implements WeightedIndex-based selection - `select_random_adapter()`: Enhanced random selection with error consideration - Added comprehensive inline documentation explaining algorithms - Maintains thread safety with proper Arc usage and thread-safe RNG - Added test coverage for weighted selection with statistical validation - Extended Provider struct with f64 weight field (default: 1.0) - Added weight validation in Provider::validate() method - Added Chain-level validation to prevent all-zero weight configurations - Integrated with existing configuration validation pipeline - Added --weighted-rpc-steering command line flag (node/src/opt.rs) - Integrated weighted flag through network setup pipeline (node/src/network_setup.rs) - Updated chain configuration to pass weight values to adapters (node/src/chain.rs) - Added comprehensive configuration documentation in full_config.toml - Includes weight range explanation, distribution examples, and usage guidelines - Clear examples showing relative weight calculations and traffic distribution - Updated rand dependency to use appropriate version with WeightedIndex support - Proper import paths for rand 0.9 distribution modules - Fixed compilation issues with correct trait imports (Distribution) - Comprehensive inline documentation for all weight-related methods - Clear separation of concerns with single-responsibility functions - Maintained backward compatibility with existing random selection - Added statistical test validation for weight distribution accuracy - Comprehensive test suite validates weight distribution over 1000 iterations - Statistical validation with 10% tolerance for weight accuracy - All existing tests continue to pass, ensuring no regression - Build verification across all affected packages ```toml weighted_rpc_steering = true [chains.mainnet] provider = [ { label = "primary", url = "http://rpc1.io", weight = 0.7 }, # 70% traffic { label = "backup", url = "http://rpc2.io", weight = 0.3 }, # 30% traffic ] ``` This implementation provides production-ready weighted load balancing with robust error handling, comprehensive validation, and excellent maintainability. 🤖 Generated with Claude Code
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters