CurlImpersonateHttpClient | API | Crawlee for Python · Fast, reliable Python web crawlers.
- HttpClient
- CurlImpersonateHttpClient
Index
Methods
- async __aenter__(): HttpClient
- async __aexit__(exc_type, exc_value, traceback): None
Parameters
exc_type: BaseException | None
exc_value: BaseException | None
traceback: TracebackType | None
Returns None
- __init__(*, persist_cookies_per_session, async_session_kwargs): None
Parameters
optionalkeyword-onlypersist_cookies_per_session: bool = True
async_session_kwargs: Any
Returns None
- async cleanup(): None
Returns None
- async crawl(request, *, session, proxy_info, statistics, timeout): HttpCrawlingResult
Parameters
request: Request
optionalkeyword-onlysession: Session | None = None
optionalkeyword-onlyproxy_info: ProxyInfo | None = None
optionalkeyword-onlystatistics: Statistics | None = None
optionalkeyword-onlytimeout: timedelta | None = None
Returns HttpCrawlingResult
- async send_request(url, *, method, headers, payload, session, proxy_info, timeout): HttpResponse
Parameters
url: str
optionalkeyword-onlymethod: HttpMethod = 'GET'
optionalkeyword-onlyheaders: (HttpHeaders | dict[str, str]) | None = None
optionalkeyword-onlypayload: HttpPayload | None = None
optionalkeyword-onlysession: Session | None = None
optionalkeyword-onlyproxy_info: ProxyInfo | None = None
optionalkeyword-onlytimeout: timedelta | None = None
Returns HttpResponse
- stream(url, *, method, headers, payload, session, proxy_info, timeout): AbstractAsyncContextManager[HttpResponse]
Parameters
url: str
optionalkeyword-onlymethod: HttpMethod = 'GET'
optionalkeyword-onlyheaders: (HttpHeaders | dict[str, str]) | None = None
optionalkeyword-onlypayload: HttpPayload | None = None
optionalkeyword-onlysession: Session | None = None
optionalkeyword-onlyproxy_info: ProxyInfo | None = None
optionalkeyword-onlytimeout: timedelta | None = None
Returns AbstractAsyncContextManager[HttpResponse]