Storage | API | Crawlee for Python · Fast, reliable Python web crawlers.
Hierarchy
Index
Methods
drop
- async drop(): None
Drop the storage, removing it from the underlying storage client and clearing the cache.
Returns None
get_metadata
- async get_metadata(): (DatasetMetadata | KeyValueStoreMetadata) | RequestQueueMetadata
Get the storage metadata.
Returns (DatasetMetadata | KeyValueStoreMetadata) | RequestQueueMetadata
open
- async open(*, id, name, alias, configuration, storage_client): Storage
Open a storage, either restore existing or create a new one.
Parameters
optionalkeyword-onlyid: str | None = None
optionalkeyword-onlyname: str | None = None
The storage name (global scope, persists across runs). Name can only contain letters "a" through "z", the digits "0" through "9", and the hyphen ("-") but only in the middle of the string (e.g. "my-value-1").
optionalkeyword-onlyalias: str | None = None
The storage alias (run scope, creates unnamed storage).
optionalkeyword-onlyconfiguration: Configuration | None = None
Configuration object used during the storage creation or restoration process.
optionalkeyword-onlystorage_client: StorageClient | None = None
Underlying storage client to use. If not provided, the default global storage client from the service locator will be used.
Returns Storage
purge
- async purge(): None
Purge the storage, removing all items from the underlying storage client.
This method does not remove the storage itself, e.g. don't remove the metadata, but clears all items within it.
Returns None