Standardize cookie handling
Currently we have 3 main cookie handling mechanisms depending on the HTTP client or browser, and none work correctly.
-
HttpxHttpClient.
This solution is closest to expected. Cookies are stored inSession. However, we usedictwhich loses the cookie-domain relationship. This can cause issues during cross-domain crawling. -
CurlImpersonateHttpClient.
Sessionknows nothing about cookies and all cookies are stored at theAsyncSessionlevel. As a result, if we don't use proxies, all sessions have identical cookies. If we work with proxies, cookies become tied to the proxy. -
Playwright.
Sessionknows nothing about cookies and all cookies are stored at thePlaywrightContextlevel, meaning all sessions working from one context will operate with the same cookies.