Create an inference endpoint

PUT /_inference/{task_type}/{inference_id}

Copy endpoint

IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Mistral, Azure OpenAI, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

The following integrations are available through the inference API. You can find the available task types next to the integration name:

AI21 (chat_completion, completion)
AlibabaCloud AI Search (completion, rerank, sparse_embedding, text_embedding)
Amazon Bedrock (chat_completion, completion, text_embedding)
Amazon SageMaker (chat_completion, completion, rerank, sparse_embedding, text_embedding)
Anthropic (completion)
Azure AI Studio (completion, rerank, text_embedding)
Azure OpenAI (chat_completion, completion, text_embedding)
Cohere (completion, rerank, text_embedding)
DeepSeek (chat_completion, completion)
Elasticsearch (rerank, sparse_embedding, text_embedding - this service is for built-in models and models uploaded through Eland)
ELSER (sparse_embedding)
Google AI Studio (completion, text_embedding)
Google Vertex AI (chat_completion, completion, rerank, text_embedding)
Groq (chat_completion)
Hugging Face (chat_completion, completion, rerank, text_embedding)
JinaAI (embedding, rerank, text_embedding)
Llama (chat_completion, completion, text_embedding)
Mistral (chat_completion, completion, text_embedding)
Nvidia (chat_completion, completion, text_embedding, rerank)
OpenAI (chat_completion, completion, text_embedding)
OpenShift AI (chat_completion, completion, rerank, text_embedding)
VoyageAI (rerank, text_embedding)
Watsonx (chat_completion, completion, rerank, text_embedding)

Required authorization

Cluster privileges: manage_inference

Parameters

path Path Parameters

Name	Type
`task_type` required The task type. Refer to the integration list in the API description for the available task types.	type InferenceTypesTaskType = "sparse_embedding" \| "text_embedding" \| "rerank" \| "completion" \| "chat_completion" \| "embedding"
`inference_id` required The inference Id	type TypesId = string

query Query Parameters

Name	Type
`timeout` Specifies the amount of time to wait for the inference endpoint to be created.	type TypesDuration = string \| "-1" \| "0"

Request Body

application/json required

interface InferenceTypesInferenceEndpoint {
chunking_settings?:

InferenceTypesInferenceChunkingSettings

Chunking configuration object

interface InferenceTypesInferenceChunkingSettings {
max_chunk_size?: number;
overlap?: number;
sentence_overlap?: number;
separator_group?: string;
separators?: string[];
strategy?: string;
}

;
service: string;
service_settings:

InferenceTypesServiceSettings

interface InferenceTypesServiceSettings {}

;
task_settings?:

InferenceTypesTaskSettings

interface InferenceTypesTaskSettings {}

;
}

Responses

200 application/json

type InferenceTypesInferenceEndpointInfo = interface InferenceTypesInferenceEndpoint {
chunking_settings?: