From 1769e5730b1470dbd14385d1913282b0c4c0cf45 Mon Sep 17 00:00:00 2001 From: Vlada Dusek Date: Mon, 22 Jun 2026 11:41:12 +0200 Subject: [PATCH 1/2] docs: clarify and surface single vs. shared request queue Add a pointer from the request queue section to the single vs. shared explanation, and give a concrete use case for the shared client so readers can tell when they need it. --- docs/02_concepts/03_storages.mdx | 2 ++ docs/02_concepts/12_storage_clients.mdx | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/02_concepts/03_storages.mdx b/docs/02_concepts/03_storages.mdx index bb616813..15a03947 100644 --- a/docs/02_concepts/03_storages.mdx +++ b/docs/02_concepts/03_storages.mdx @@ -151,6 +151,8 @@ you can use the `KeyValueStore. In this section we will show you how to work with [request queues](https://docs.apify.com/platform/storage/request-queue). +By default, a request queue is optimized for a single consumer. To let multiple Actor runs process the same queue at once, see [single vs. shared request queue](./storage-clients#single-vs-shared-request-queue). + ### Adding requests to a queue To add a request into the queue, you can use the `RequestQueue.add_request` method. diff --git a/docs/02_concepts/12_storage_clients.mdx b/docs/02_concepts/12_storage_clients.mdx index 221af2a0..7b953fd9 100644 --- a/docs/02_concepts/12_storage_clients.mdx +++ b/docs/02_concepts/12_storage_clients.mdx @@ -43,8 +43,8 @@ For details, see the [Crawlee storage clients guide](https://crawlee.dev/python/ `ApifyStorageClient` supports two ways of accessing the Apify request queue, selected via its `request_queue_access` argument: -- **`'single'`** (default) - optimized for a single consumer. It makes fewer API calls, so it is cheaper and faster, but it doesn't support multiple clients consuming the same queue concurrently. This is the right choice for the majority of Actors. -- **`'shared'`** - supports multiple consumers working on the same queue at the same time, at the cost of more API calls. +- **`'single'`** (default) - optimized for a single consumer. It makes fewer API calls, so it's cheaper and faster, but it doesn't support multiple clients consuming the same queue concurrently. This is the right choice for the majority of Actors. +- **`'shared'`** - supports multiple consumers working on the same queue at the same time, at the cost of more API calls. Use it when several Actor runs share one named request queue. For example, you can split a large crawl across parallel runs, or feed the queue from a producer run while worker runs consume it. To opt into the shared client, set it as the cloud client of the `SmartApifyStorageClient` in the [service locator](https://crawlee.dev/python/docs/guides/service-locator) before entering the Actor context: From 98a6d463aa4952fa6a18b2219e31056cc1861446 Mon Sep 17 00:00:00 2001 From: Vlada Dusek Date: Mon, 22 Jun 2026 11:42:53 +0200 Subject: [PATCH 2/2] docs: link single vs. shared request queue from storages conclusion --- docs/02_concepts/03_storages.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/02_concepts/03_storages.mdx b/docs/02_concepts/03_storages.mdx index 15a03947..f44a98c4 100644 --- a/docs/02_concepts/03_storages.mdx +++ b/docs/02_concepts/03_storages.mdx @@ -189,6 +189,6 @@ Behind the scenes, the SDK uses storage clients to communicate with the storage ## Conclusion -This page has covered the three storage types (datasets, key-value stores, and request queues): how they are emulated on the local filesystem, how to open named and unnamed storages, and how to read from and write to each through the `Actor` shortcuts and the storage classes. +This page has covered the three storage types (datasets, key-value stores, and request queues): how they are emulated on the local filesystem, how to open named and unnamed storages, and how to read from and write to each through the `Actor` shortcuts and the storage classes. When multiple Actor runs need to process the same request queue, choose between a [single and shared request queue](./storage-clients#single-vs-shared-request-queue). For comprehensive information about storage on the Apify platform, see the [storage documentation](https://docs.apify.com/platform/storage), including the pages on [datasets](https://docs.apify.com/platform/storage/dataset), [key-value stores](https://docs.apify.com/platform/storage/key-value-store), and [request queues](https://docs.apify.com/platform/storage/request-queue).