TypeError in Crawler when local storage is not purged
When running a simple BeautifulSoupCrawler / PlaywrightCrawler script without purging local storage, a TypeError is raised.
The exception traceback indicates that the Statistics._persist_state() method is receiving an unexpected number of arguments. This issue occurs in the listener_wrapper function within crawlee.events.event_manager.
Script to reproduce
import asyncio from crawlee.beautifulsoup_crawler import BeautifulSoupCrawler, BeautifulSoupCrawlingContext async def main() -> None: crawler = BeautifulSoupCrawler() @crawler.router.default_handler async def request_handler(context: BeautifulSoupCrawlingContext) -> None: data = { 'url': context.request.url, 'title': context.soup.title.string if context.soup.title else None, } await context.push_data(data) await crawler.run(['https://crawlee.dev']) if __name__ == '__main__': asyncio.run(main())
Full error message:
$ python run_beautifulsoup_crawler.py
Exception in callback AsyncIOEventEmitter._emit_run.<locals>.callback(<Task finishe... were given')>) at /home/vdusek/Projects/crawlee-py/.venv/lib/python3.12/site-packages/pyee/asyncio.py:69
handle: <Handle AsyncIOEventEmitter._emit_run.<locals>.callback(<Task finishe... were given')>) at /home/vdusek/Projects/crawlee-py/.venv/lib/python3.12/site-packages/pyee/asyncio.py:69>
Traceback (most recent call last):
File "/usr/lib64/python3.12/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/home/vdusek/Projects/crawlee-py/.venv/lib/python3.12/site-packages/pyee/asyncio.py", line 77, in callback
self.emit("error", exc)
File "/home/vdusek/Projects/crawlee-py/.venv/lib/python3.12/site-packages/pyee/base.py", line 211, in emit
self._emit_handle_potential_error(event, args[0] if args else None)
File "/home/vdusek/Projects/crawlee-py/.venv/lib/python3.12/site-packages/pyee/base.py", line 169, in _emit_handle_potential_error
raise error
File "/home/vdusek/Projects/crawlee-py/src/crawlee/events/event_manager.py", line 108, in listener_wrapper
listener(event_data)
TypeError: Statistics._persist_state() takes 1 positional argument but 2 were given