feat: add weight-based rate limiting support to HttpRequestRegexMatcher by darynaishchenko · Pull Request #966

feat: add weight-based rate limiting support to HttpRequestRegexMatcher by darynaishchenko · Pull Request #966 · airbytehq/airbyte-python-cdk

Add optional 'cost' field to HttpRequestRegexMatcher that allows specifying
per-request cost/weight in rate limiting policies. This enables cost-based
rate limiting where different endpoints consume different amounts from a
shared budget (e.g., Amplitude's Dashboard REST API).

Changes:
- YAML schema: add 'cost' field (int or string) to HttpRequestRegexMatcher
- Python model: add 'cost' field to HttpRequestRegexMatcher model
- call_rate.py: store cost on matcher, add get_cost() to BaseCallRatePolicy,
  update APIBudget._do_acquire() to use cost as weight instead of hardcoded 1
- model_to_component_factory.py: wire up cost field with interpolation support
- Tests: 8 new tests for cost-based rate limiting behavior

Backward compatible: cost defaults to None (treated as 1).

Co-Authored-By: Daryna Ishchenko <darina.ishchenko17@gmail.com>

Co-Authored-By: Daryna Ishchenko <darina.ishchenko17@gmail.com>

…quire(weight=...)

Co-Authored-By: Daryna Ishchenko <darina.ishchenko17@gmail.com>

devin-ai-integration bot changed the title ~~feat: add cost-based rate limiting support to HttpRequestRegexMatcher~~ feat: add weight-based rate limiting support to HttpRequestRegexMatcher

Mar 27, 2026

Address CodeRabbit review: get_weight() now stops at the first matcher
that matches the request, consistent with matches() behavior. If the
first matching matcher has a weight, use it; otherwise return 1.

Co-Authored-By: Daryna Ishchenko <darina.ishchenko17@gmail.com>

Reject non-positive weight values in the constructor to prevent
weight=0 (free requests) or negative weights (budget restoration)
from bypassing rate limits. Adds two tests for validation.

Co-Authored-By: Daryna Ishchenko <darina.ishchenko17@gmail.com>

Validates weight >= 1 after interpolation/casting in the factory,
catching invalid weights from manifest config before they reach
the HttpRequestRegexMatcher constructor.

Co-Authored-By: Daryna Ishchenko <darina.ishchenko17@gmail.com>

Co-Authored-By: Daryna Ishchenko <darina.ishchenko17@gmail.com>

…Policy

Co-Authored-By: Daryna Ishchenko <darina.ishchenko17@gmail.com>