Perform inference on the service

POST /_inference/{task_type}/{inference_id}

Copy endpoint

This API enables you to use machine learning models to perform specific tasks on data that you provide as an input. It returns a response with the results of the tasks. The inference endpoint you use can perform one specific task that has been defined when the endpoint was created with the create inference API.

For details about using this API with a service, such as Amazon Bedrock, Anthropic, or HuggingFace, refer to the service-specific documentation.

info The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

Required authorization

Cluster privileges: monitor_inference

Parameters

path Path Parameters

Name	Type
`task_type` required The type of inference task that the model performs.	type InferenceTypesTaskType = "sparse_embedding" \| "text_embedding" \| "rerank" \| "completion" \| "chat_completion" \| "embedding"
`inference_id` required The unique identifier for the inference endpoint.	type TypesId = string

query Query Parameters

Name	Type
`timeout` The amount of time to wait for the inference request to complete.	type TypesDuration = string \| "-1" \| "0"

Request Body

application/json required

{
query?: string;
input: string[] | string;
input_type?: string;
task_settings?:

InferenceTypesTaskSettings

interface InferenceTypesTaskSettings {}

;
}

Responses

200 application/json

interface InferenceTypesInferenceResult {
embeddings_bytes?:

InferenceTypesDenseEmbeddingByteResult

The dense embedding result object for byte representation

interface InferenceTypesDenseEmbeddingByteResult {
embedding: InferenceTypesDenseByteVector;
}

[];
embeddings_bits?:

InferenceTypesDenseEmbeddingByteResult

The dense embedding result object for byte representation

interface InferenceTypesDenseEmbeddingByteResult {
embedding: InferenceTypesDenseByteVector;
}

[];
embeddings?:

InferenceTypesDenseEmbeddingResult

The dense embedding result object for float representation

interface InferenceTypesDenseEmbeddingResult {
embedding: InferenceTypesDenseVector;
}

[];
text_embedding_bytes?:

InferenceTypesDenseEmbeddingByteResult

The dense embedding result object for byte representation

interface InferenceTypesDenseEmbeddingByteResult {
embedding: InferenceTypesDenseByteVector;
}

[];
text_embedding_bits?:

InferenceTypesDenseEmbeddingByteResult

The dense embedding result object for byte representation

interface InferenceTypesDenseEmbeddingByteResult {
embedding: InferenceTypesDenseByteVector;
}

[];
text_embedding?:

InferenceTypesDenseEmbeddingResult

The dense embedding result object for float representation

interface InferenceTypesDenseEmbeddingResult {
embedding: InferenceTypesDenseVector;
}

[];
sparse_embedding?:

InferenceTypesSparseEmbeddingResult

interface InferenceTypesSparseEmbeddingResult {
is_truncated: boolean;
embedding: InferenceTypesSparseVector;
}

[];
completion?:

InferenceTypesCompletionResult

The completion result object

interface InferenceTypesCompletionResult {
result: string;
}

[];
rerank?:

InferenceTypesRankedDocument

The rerank result object representing a single ranked document id: the original index of the document in the request relevance_score: the relevance_score of the document relative to the query text: Optional, the text of the document, if requested

interface InferenceTypesRankedDocument {
index: number;
relevance_score: number;
text?: string;
}

[];
}