refactor!: сhange default `incognito context` to `persistent context` for `Playwright` by Mantisus · Pull Request #985 · apify/crawlee-python

@Mantisus

@Mantisus Mantisus changed the title refactor!: сhange default 'incognito context' to 'persistent context' for Playwright refactor!: сhange default incognito context to persistent context for Playwright

Feb 15, 2025

vdusek

Pijukatel

Pijukatel

Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>
Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>
Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>

@renovate @Mantisus

@renovate @Mantisus

@renovate @Mantisus

### Description

- fix public imports in `__init__` files
- Add `rich` to direct dependencies. It is one of `cookiecutter`'s
dependencies, but we use it directly in `statistics._models.py`

---------

Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>

@renovate @Mantisus

### Description

Add adaptive context helpers and documentation for
AdaptivePlaywrightCrawler.

### Issues

- Closes: apify#249

---------

Co-authored-by: Jan Buchar <Teyras@gmail.com>
Co-authored-by: Jan Buchar <jan.buchar@apify.com>

@Mantisus

…#988)

### Description

- change custom `LRUCache` to `cachetools.LRUCache`. In my opinion,
`functools.lru_cache's` logic isn't well-suited for this use case.
Therefore, if we want to modify our caching approach, using `cachetools`
appears to be a better option.

### Issues

- Closes: apify#86
### Description

- update curl-cffi version requirement to >=0.9.0.
- update default `impersonate` from `chrome124` to `chrome131`
- Migrate from `poetry` to `uv`.
- Relates: apify#628
- The update of templates to use `uv` will be implemented separately.

@renovate @Mantisus

- `project.urls`
- python 3.13 in ci
- unify name "Set up uv package manager"
- fix contributing guide
- add all extra, remove dev extra (move to dev deps)
- relates: apify#628

@vdusek @Mantisus

@Mantisus

…pify#959)

Add `additional_http_error_status_codes` and
`ignore_http_error_status_codes` to PlaywrightCrawler.
Since they exist now on all crawlers, move them to `BasicCrawler` level.
Do not use `_http_client` attributes for getting additional status codes
related variables.

**Breaking:** Remove `HttpCrawlerOptions` -> No unique options compared
to `BasicCrawlerOptions` anymore.

- Closes: apify#953

@Mantisus

@Mantisus

@Mantisus

@Mantisus

vdusek

@Mantisus

@Mantisus

Pijukatel

@Mantisus

@Mantisus

Pijukatel

vdusek

Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>

@Mantisus

@Mantisus

@vdusek

@vdusek vdusek added the t-tooling

Issues with this label are in the ownership of the tooling team.

label

Feb 25, 2025