feat: blocking detection for playwright crawler by vdusek · Pull Request #328 · apify/crawlee-python

Skip to content

Navigation Menu

Sign in

Appearance settings

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up

Appearance settings

Conversation

@vdusek

Copy link

Collaborator

@vdusek vdusek commented

Jul 19, 2024

edited

Loading

Description

  • Add blocking detection for PlaywrightCrawler.
  • Improve documentation of the PlaywrightCrawler and related components.

Issues

Testing

Checklist

  • Changes are described in the CHANGELOG.md
  • CI passed

@vdusek vdusek requested a review from janbuchar

July 19, 2024 11:19

@vdusek vdusek self-assigned this

Jul 19, 2024

@github-actions github-actions bot added this to the 94th sprint - Tooling team milestone

Jul 19, 2024

@github-actions github-actions bot added the t-tooling

Issues with this label are in the ownership of the tooling team.

label

Jul 19, 2024

Comment on lines +16 to +19

Args:
page: The Playwright `Page` object.
response: The Playwright `Response` object.
enqueue_links: The `PlaywrightCrawler` implementation of the `EnqueueLinksFunction` function.
Copy link

Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this correctly rendered in the documentation?

Copy link

Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, args in API docs currently do not work - #324.

@vdusek vdusek merged commit 49ff6e2 into master

Jul 19, 2024

@vdusek vdusek deleted the playwright-blocking branch

July 19, 2024 12:08

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@janbuchar janbuchar janbuchar approved these changes

Assignees

@vdusek vdusek

Labels

t-tooling

Issues with this label are in the ownership of the tooling team.

Projects

None yet

Milestone

94th sprint - Tooling team

Development

Successfully merging this pull request may close these issues.

Check for selectors that indicate blocking in PlaywrightCrawler

2 participants

@vdusek @janbuchar