Wrong type of crawler shown when using crawler/context logging
Reproduction
import asyncio from crawlee.crawlers import ParselCrawler, ParselCrawlingContext async def main() -> None: crawler = ParselCrawler() crawler.log.info('Hello from crawler') @crawler.router.default_handler async def request_handler(context: ParselCrawlingContext) -> None: context.log.info('Hello from handler') await crawler.run(['https://crawlee.dev']) if __name__ == '__main__': asyncio.run(main())
Output:
$ python crawl.py
[crawlee.crawlers._abstract_http._abstract_http_crawler] INFO Hello from crawler
[crawlee.crawlers._abstract_http._abstract_http_crawler] INFO Current request statistics:
┌───────────────────────────────┬──────────┐
│ requests_finished │ 0 │
│ requests_failed │ 0 │
│ retry_histogram │ [0] │
│ request_avg_failed_duration │ None │
│ request_avg_finished_duration │ None │
│ requests_finished_per_minute │ 0 │
│ requests_failed_per_minute │ 0 │
│ request_total_duration │ 0.0 │
│ requests_total │ 0 │
│ crawler_runtime │ 0.082711 │
└───────────────────────────────┴──────────┘
[crawlee._autoscaling.autoscaled_pool] INFO current_concurrency = 0; desired_concurrency = 2; cpu = 0; mem = 0; event_loop = 0.0; client_info = 0.0
[crawlee.crawlers._abstract_http._abstract_http_crawler] INFO Hello from handler
[crawlee._autoscaling.autoscaled_pool] INFO Waiting for remaining tasks to finish
[crawlee.crawlers._abstract_http._abstract_http_crawler] INFO Final request statistics:
┌───────────────────────────────┬──────────┐
│ requests_finished │ 1 │
│ requests_failed │ 0 │
│ retry_histogram │ [1] │
│ request_avg_failed_duration │ None │
│ request_avg_finished_duration │ 0.598955 │
│ requests_finished_per_minute │ 87 │
│ requests_failed_per_minute │ 0 │
│ request_total_duration │ 0.598955 │
│ requests_total │ 1 │
│ crawler_runtime │ 0.693418 │
└───────────────────────────────┴──────────┘
Expected behaviour
- Display the specific crawler type instead of
crawlee.crawlers._abstract_http._abstract_http_crawler.