Home Original page

Request | API | Crawlee for Python · Fast, reliable Python web crawlers.

Index

Methods

  • crawl_depth(new_value): None

  • Parameters

    • new_value: int

    Returns None

  • enqueue_strategy(new_enqueue_strategy): None
  • forefront(new_value): None

  • Parameters

    • new_value: bool

    Returns None

  • from_url(url, *, method, headers, payload, label, session_id, unique_key, keep_url_fragment, use_extended_unique_key, always_enqueue, enqueue_strategy, max_retries, kwargs): Self

  • Parameters

    • url: str
    • optionalkeyword-onlymethod: HttpMethod = 'GET'
    • optionalkeyword-onlyheaders: (HttpHeaders | dict[str, str]) | None = None
    • optionalkeyword-onlypayload: (HttpPayload | str) | None = None
    • optionalkeyword-onlylabel: str | None = None
    • optionalkeyword-onlysession_id: str | None = None
    • optionalkeyword-onlyunique_key: str | None = None
    • optionalkeyword-onlykeep_url_fragment: bool = False
    • optionalkeyword-onlyuse_extended_unique_key: bool = False
    • optionalkeyword-onlyalways_enqueue: bool = False
    • optionalkeyword-onlyenqueue_strategy: EnqueueStrategy | None = None
    • optionalkeyword-onlymax_retries: int | None = None
    • kwargs: Any

    Returns Self

  • get_query_param_from_url(param, *, default): str | None

  • Parameters

    • param: str
    • optionalkeyword-onlydefault: str | None = None

    Returns str | None

  • last_proxy_tier(new_value): None

  • Parameters

    • new_value: int

    Returns None

  • session_rotation_count(new_session_rotation_count): None

  • Parameters

    • new_session_rotation_count: int

    Returns None

  • state(new_state): None

Properties

handled_at: datetime | None

last_proxy_tier: int | None

session_rotation_count: int | None

was_already_handled: bool