Toolcog home
Search ⌘ + K

Command Palette

Search for a command to run...

Sign In

All operations here run the same website-content-crawler Actor but expose three different behaviors: two synchronous variants that wait for the Actor to finish and return its outputs directly, and one start-only variant that returns run metadata immediately. Every call requires the caller's Apify token and a request body that conforms to the operation's InputSchema (the Actor reference and the Actor input are supplied inside that body). Keep those two top-level arguments in every request: token and body.

How the domain is organized

A single Actor run can produce two places of output relevant to these operations: the Actor dataset (an array of items) and the Actor key-value store (named keys, commonly a key called OUTPUT). The three operations differ only in whether they wait for the run to finish and which of those outputs they return:

  • Synchronous dataset variant: executes the Actor, waits for completion, and returns the Actor's dataset items in the response body.
  • Synchronous key-value variant: executes the Actor, waits for completion, and returns the value the Actor wrote to the key-value store under the OUTPUT key in the response body.
  • Start-only variant: starts the run and returns run metadata (IDs and status) immediately; it does not wait for the run to finish.

Understanding these relationships is the most important operational fact: a run can produce both dataset items and key-value entries, but only the specific synchronous endpoint that was called will return one of those outputs directly.

Which operation to call first (entry points)

Choose an operation based on whether the user needs the crawl results immediately and which output they expect:

  • If the user asks to crawl a site and get structured items (the Actor writes results into a dataset), call run-sync-get-dataset-items-apify-website-content-crawler. The response body contains the dataset items.

  • If the user asks to crawl a site and expects a single exported payload the Actor writes to the key-value store (typically under the OUTPUT key), call run-sync-apify-website-content-crawler. The response body contains that key-value value.

  • If the user wants to start the crawl, monitor progress, or retrieve outputs later (for example to inspect run logs, fetch by run/dataset IDs, or trigger downstream processing), call runs-sync-apify-website-content-crawler. The response returns run metadata (IDs and initial status) that you can use with other Apify platform endpoints or tools that operate on runs, datasets, and key-value stores.

All three operations accept the same top-level shape: { token: <apify-token>, body: <InputSchema> }. The body must match the Actor's InputSchema (specify the target URL(s), crawling limits, and any crawler options the Actor expects).

Common tasks and the exact operation to use

  • Crawl and return final dataset items in one step: call run-sync-get-dataset-items-apify-website-content-crawler. When it completes successfully, the returned body is the dataset items array.

  • Crawl and return the Actor's key-value OUTPUT value in one step: call run-sync-apify-website-content-crawler. The returned body is the value stored under the OUTPUT key. If the Actor writes to a different key, this operation will not return that other key.

  • Start a run and get IDs to operate on later: call runs-sync-apify-website-content-crawler. Inspect the run metadata in the returned body for the runId, and for storage IDs such as datasetId or keyValueStoreId (if present). Use those IDs with the platform's storage/query endpoints when you need to retrieve outputs later.

  • When the desired output format is unclear, prefer the dataset variant. Many crawlers write structured records to the dataset while the OUTPUT key is used for a single export file; requesting the dataset is more likely to give you item-level structure you can act on.

What to check in responses

  • status (top-level): a high-level indicator of the request result. Synchronous calls will often set this to indicate completion state; check it before trusting the body contents.

  • body shape varies by operation:

    • For run-sync-get-dataset-items-apify-website-content-crawler, body is the dataset items (usually an array).
    • For run-sync-apify-website-content-crawler, body is the key-value value (the content stored under OUTPUT).
    • For runs-sync-apify-website-content-crawler, body is structured run metadata (look for runId, status, and any store IDs).

Treat these differences as intentional: calls that wait for completion return outputs directly; the start-only call returns metadata for later use.

Non-obvious behaviors and gotchas

  • Synchronous calls block until the Actor finishes. For long or unbounded crawls the call may take a long time or fail if the Actor never completes; prefer the start-only call when the run is expected to be long or when monitoring is required.

  • The synchronous key-value variant returns only the value stored under the OUTPUT key. If the Actor writes its main results to the dataset or to a differently named key, that synchronous key-value call will not return those other outputs.

  • The start-only response is the reliable place to get run and storage IDs. If a synchronous call returns an empty or unexpected body, use the run metadata (from the start-only variant) to inspect which stores the Actor created and to fetch contents by ID.

  • The body argument you pass must match the Actor's InputSchema. Typical fields include the target URL(s), crawl limits, and other crawler options; confirm the exact field names in the InputSchema before constructing body so the Actor receives the expected inputs.

  • Every request requires a valid Apify token in the token parameter. Tokens must have permission to run the Actor and to access any resulting stores.

Quick selection guide

When a user request arrives, pick an operation as follows:

  • Needs immediate dataset items → run-sync-get-dataset-items-apify-website-content-crawler (returns dataset items in body).
  • Needs immediate single-export/output value → run-sync-apify-website-content-crawler (returns OUTPUT value in body).
  • Needs to start run and get run/store IDs for later checks or external retrieval → runs-sync-apify-website-content-crawler (returns run metadata in body).

Use the returned status and the specific body shape for the chosen operation to determine the next action: parse dataset items if present, inspect OUTPUT contents if returned, or extract runId/datasetId/keyValueStoreId from run metadata for downstream steps.