Unable to configure retry behavior for status codes 400-499

In BasicCrawler._should_retry_request, we always return False for status codes 400-499.

    def _should_retry_request(self, context: BasicCrawlingContext, error: Exception) -> bool:
        if context.request.no_retry:
            return False

        # Do not retry on client errors.
        if isinstance(error, HttpStatusCodeError) and is_status_code_client_error(error.status_code):
            return False

The is_status_code_client_error function enforces this behavior:

def is_status_code_client_error(value: int) -> bool:
    """Returns `True` for 4xx status codes, `False` otherwise."""
    return 400 <= value <= 499  # noqa: PLR2004

This makes it impossible to configure retry behavior for these codes.