Elasticsearch is organized around a cluster of nodes that store and serve indices (and data streams). Most operations target one of these core entities: indices/data streams, documents (by id), snapshots/repositories, long‑running tasks (identified by task_id), transforms, ML jobs, and inference endpoints. Understanding which ID each family needs and which call returns it is the single most important operational pattern.

Key entities and relationships

• Indices and data streams: indices (strings) are the primary storage unit. Data streams are collections of backing indices; many APIs accept either index or data_stream names and special options for expand/ignore behavior. Aliases map names to one or more indices and are commonly used for rollover workflows.

• Documents: documents live in an index and are addressed by index + id. Most write/read flows need the index name and optionally an id.

• Templates / component templates: templates define mappings/settings/aliases applied when new indices/data streams are created. Component templates are composed into index templates.

• Ingest pipelines: named pipelines preprocess documents during indexing. Create with a pipeline API and reference by pipeline when indexing or in bulk operations.

• Snapshots & repositories: snapshots are stored in repositories. You must create/configure a repository before creating snapshots. Snapshots are referred to by repository + snapshot name.

• Long-running jobs and tasks: many heavy operations (reindex, update-by-query, delete-by-query, reindexing, snapshot restore, ML jobs, transforms) either accept a wait_for_completion flag or return a task_id you can inspect and rethrottle. Async search and async ES|QL follow a submit→poll→delete lifecycle and return their own id.

• Machine Learning and transforms: ML jobs (job_id, datafeed_id) and trained models (model_id) have their own create/start/stop/status flows. Transforms are created/started/stopped by transform_id.

• Inference endpoints: inference endpoints are created with an inference_id and then invoked for embeddings, completions, rerank, etc. Many endpoint types (OpenAI, Anthropic, etc.) share the same semantics but require appropriate task_type when creating.

What to call first — common entry points

Start by fetching the cluster and index identifiers you'll need: use cluster-info/cluster-health to check cluster status and indices-get or cat-indices to list indices and get canonical index names. To find data stream names use indices-get-data-stream.

• Need an index name for most operations: call indices-get or indices-resolve-index.
• Need a repository or snapshot name: call snapshot-get-repository then snapshot-get.
• Need a transform / job / model id: call the transform-get-transform, ml-get-jobs, or ml-get-trained-models endpoints respectively to list available IDs.
• For connectors or inference endpoints: list them (connector-list, inference-get) to obtain connector_id or inference_id.

Knowing where to obtain the small set of IDs the rest of your flow requires prevents dead ends.

Common capabilities and practical sequences

Below are the real tasks users ask agents to perform and the practical API sequences to accomplish them. Each step uses the API parameter names you must supply (index, id, repository, snapshot, inference_id, etc.).

Search & fetch documents

• Quick document lookup by id: call get with index and id.
• Freeform query: call search with index (or omit for all) and a query DSL in body.query. If you need multiple queries in one request use msearch.
• Count results: count accepts a query body or q string.
• Paging and long-lived cursors: prefer open-point-in-time (PIT) then search with pit_id for consistent paging; close with close-point-in-time.

Index, update, delete, bulk ingest

• Index a single document: index (or create if you require create-only semantics) with index, id, and body.
• Update partial document: update with index, id, and body.doc or script.
• Delete document: delete with index and id.
• High throughput ingest: bulk with body as an array of action lines. Check errors and the items array in the response for per-operation failures. Use pipeline to route through an ingest pipeline.

Bulk and indexing gotcha: bulk responses don’t throw for partial failures — always inspect errors and the items list.

Reindex / transform / update-by-query

• Reindex to copy data: reindex with body.source and body.dest. For large jobs, use wait_for_completion=false to run async, inspect task in response, then use tasks APIs to monitor or reindex-rethrottle to change speed.
• Update or delete by query: update-by-query / delete-by-query; these accept wait_for_completion and produce task if async. Use *_rethrottle endpoints to adjust throughput.

Snapshots and restore

• Create repository: snapshot-create-repository (must exist before snapshots).
• Create snapshot: snapshot-create with repository + snapshot name. Use wait_for_completion when you want synchronous completion.
• List snapshots: snapshot-get and snapshot-status.
• Restore: snapshot-restore with repository + snapshot, optionally indices, rename_pattern, rename_replacement, and include_global_state (see gotchas).

Snapshot gotcha: include_global_state=true will restore cluster-level objects (templates, ingest pipelines, ILM) and can overwrite cluster persistent settings—do not set it unless you intend to restore cluster state.

Async searches, EQL, ES|QL, and SQL

• Async search (search saved on cluster): use async-search-submit (or client-side search with wait_for_completion_timeout and keep_on_completion) which returns an id. Poll with async-search-get or check async-search-status; delete with async-search-delete when finished.
• EQL async: eql-search (or eql-search with wait_for_completion_timeout) returns similar async id; use eql-get, eql-get-status, eql-delete.
• ES|QL async: esql-async-query returns an ID; retrieve with esql-async-query-get and stop with esql-async-query-stop.
• SQL: sql-query supports sync or becomes async depending on wait_for_completion_timeout; sql-translate converts SQL to DSL; sql-get-async and sql-delete-async manage async SQL.

Async gotchas: many async endpoints accept keep_alive (how long results are stored) and wait_for_completion_timeout (how long to wait synchronously). If you get an id back, poll the appropriate GET/status endpoint to obtain results.

Ingest pipelines and simulation

• Create pipeline: ingest-put-pipeline with id and body.processors.
• Test pipeline: ingest-simulate with docs and optional pipeline or pipeline_substitutions.

Index template lifecycle

• Create component templates first if composing templates; then indices-put-index-template composes component templates into index template. If you need to preview template application, use indices-simulate-index-template/indices-simulate-template or indices-create with settings/mappings explicitly.

Security, roles, and users (privileged operations)

• Manage security objects via security-* endpoints (security-put-user, security-put-role, security-get-user, etc.). These APIs require appropriate privileges; many agent tasks in multi-tenant environments will not have rights to call them.

Inference and ML workflows

• Inference endpoints: create with inference-put (or provider-specific inference-put-<provider>) using an inference_id, then invoke inference with inference-inference, inference-embedding, inference-text-embedding, inference-completion, or streaming variants. To list endpoints use inference-get.
• Trained models: create with ml-put-trained-model and start deployment with ml-start-trained-model-deployment; call ml-infer-trained-model to run inference directly.

ML gotchas: model deployment has number_of_allocations, queue_capacity, and wait_for semantics; deployments may be long-running and you must check ml-get-trained-models-stats or ml-start-trained-model-deployment response assignment details.

Transforms and rollups

• Create and manage transforms with transform-put-transform, start/stop with transform-start-transform / transform-stop-transform, and preview with transform-preview-transform.
• Rollups: create jobs with rollup-put-job, start/stop via rollup-start-job / rollup-stop-job, and query rolled-up data with rollup-rollup-search.

Gotchas and non-obvious behaviors

• Wildcards and missing indices: many index APIs accept expand_wildcards, allow_no_indices, and ignore_unavailable. If you expect wildcard patterns that might match nothing, supply allow_no_indices=true or the call may return 404. Conversely, some APIs default to strict behavior—verify the allow_no_match/allow_no_indices behavior for the specific operation.

• require_alias and require_data_stream: write APIs (bulk, index, create) accept flags forcing target to be an alias or data stream. If your target is a data stream, set require_data_stream=true otherwise the request fails.

• Long-running ops: many resource-intensive operations default to synchronous behavior unless you set wait_for_completion=false. When they return task or task_id, use the tasks endpoints to inspect progress and *_rethrottle to adjust throughput.

• Templates vs existing indices: updating templates does not retroactively change existing indices. To apply mapping/setting changes you may need to create a new index and reindex or use composable templates and rollover workflows.

• Bulk behavior: bulk accepts a mix of index/update/delete actions. The response is per-item; errors=true means at least one item failed. Retry only the failed items; inspect each item’s response.

• Snapshot restore cluster state: include_global_state=true restores cluster-level objects (templates/pipelines/ILM). This can overwrite cluster settings and templates—do not set unless intended.

• Index mappings and breaking changes: some mapping changes are non-dynamic and require closing an index and re-creating it (or using reindex into a new index). Use indices-put-mapping for dynamic updates; otherwise create a new index with desired mappings and reindex.

• Scroll vs PIT: scroll uses scroll IDs and requires explicit clear-scroll. PIT offers a simpler, stateless paging pattern (open-point-in-time, search with pit_id) and must be closed with close-point-in-time.

• Ingest pipelines and versioning: pipelines include version and are stored cluster-wide. When testing, use ingest-simulate to validate behavior before indexing.

• Async search lifetime: keep_alive controls how long the async search and results are retained. If you poll after the TTL, the id is gone.

Practical checklist for the most common user requests

• Run a search: 1) ensure index name via indices-get if needed; 2) call search with index and body.query; 3) if the query is long-running, supply wait_for_completion_timeout and keep_on_completion to persist results.

• Index / bulk ingest: 1) if pipeline required, ensure pipeline exists (ingest-get-pipeline); 2) call bulk with action lines; 3) check errors and retry failed items.

• Create an index from template: 1) create component templates if using composed_of; 2) call indices-put-index-template; 3) create index (indices-create) or rely on rollover/data stream creation.

• Take a snapshot: 1) verify/create repository (snapshot-create-repository); 2) call snapshot-create with repository + snapshot; 3) use wait_for_completion=true if you need completion before continuing.

• Restore a snapshot safely: 1) review snapshot contents (snapshot-get); 2) if you must not alter cluster state, set include_global_state=false; 3) use rename_pattern/rename_replacement to avoid name collisions.

• Start a heavy job (reindex/update-by-query): 1) call with wait_for_completion=false to get task; 2) poll with tasks-get or use the returned task to check progress; 3) adjust speed with *-rethrottle if supported.

When to inspect more metadata

• If a write appears missing, check index settings/mappings via indices-get and the cluster health/shard allocation via cluster-health and cat-shards/cat-allocation.
• When debugging ingest pipeline behavior, run ingest-simulate and inspect docs results for per-processor output.

Final note

Elasticsearch exposes many specialized APIs, but most user-facing tasks reduce to a few patterns: identify the index/data stream or resource id, call the specific create/update/start API, and for long-running actions choose sync (wait_for_completion) or async (capture task or id and poll the corresponding status endpoint). When in doubt about an ID, list the parent resource (indices, snapshots, transforms, jobs, models, connectors, inference endpoints) and use that canonical value in subsequent calls.