OpenAI Evals API
Manage and run evals in the OpenAI platform.
OpenAI Evals API is one of 62 APIs that OpenAI publishes on the APIs.io network, described by a machine-readable OpenAPI specification.
This API exposes 2 JSON Schema definitions.
Tagged areas include Evals. The published artifact set on APIs.io includes an OpenAPI specification, API documentation, a JSON-LD context, and 2 JSON Schemas.
This API exposes 12 operations across 6 paths, and defines 119 schemas. It is described by OpenAPI 3.2.0, at version 2.3.0.
Requests are made against a single base URL, https://api.openai.com/v1.
12 operations
6 paths
119 schemas
2 DELETE6 GET4 POST
Metadata
The identity and technical contract details declared by the specification.
Authentication & Security 2
OpenAI Evals API declares
2 security schemes
for authenticating requests.
It accepts HTTP bearer tokens (ApiKeyAuth).
It accepts HTTP bearer tokens (AdminApiKeyAuth).
By default, every request must be authenticated.
Paths & Operations 12
Across 6 paths, the API surfaces 12 operations — 2 DELETE, 6 GET, 4 POST. Each is listed below with its method, path, parameters, and response codes.
Evals 12
Manage and run evals in the OpenAI platform.
GET
/evals
List evals
POST
/evals
Create eval
GET
/evals/{eval_id}
Get an eval
POST
/evals/{eval_id}
Update an eval
DELETE
/evals/{eval_id}
Delete an eval
GET
/evals/{eval_id}/runs
Get eval runs
POST
/evals/{eval_id}/runs
Create eval run
GET
/evals/{eval_id}/runs/{run_id}
Get an eval run
POST
/evals/{eval_id}/runs/{run_id}
Cancel eval run
DELETE
/evals/{eval_id}/runs/{run_id}
Delete eval run
GET
/evals/{eval_id}/runs/{run_id}/output_items
Get eval run output items
GET
/evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}
Get an output item of an eval run
Schemas 119
The contract defines 119 schemas that model the data the API accepts and returns. The most detailed are EvalRun (14 properties), ImageGenTool (12 properties), MCPTool (12 properties), EvalResponsesSource (11 properties). Each schema is shown below with its type and property counts.
ResponseFormatText
object
Default response format. Used to generate text responses.
1 property
1 required
InlineSkillSourceParam
object
Inline skill payload
3 properties
3 required
EvalItemContent
Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
CustomToolParam
object
A custom tool that processes input using a specified format. Learn more about [custom tools](https://developers.openai.com/api/docs/guides/function-callingcust…
7 properties
2 required
ComputerUsePreviewTool
object
A tool that controls a virtual computer. Learn more about the [computer tool](https://developers.openai.com/api/docs/guides/tools-computer-use).
4 properties
4 required
EvalApiError
object
An object representing an error response from the Eval API.
2 properties
2 required
CreateEvalRequest
object
4 properties
2 required
EvalLogsDataSourceConfig
object
A LogsDataSourceConfig which specifies the metadata property of your logs query. This is usually metadata like usecase=chatbot or prompt-version=v2, etc. The s…
3 properties
2 required
HybridSearchOptions
object
2 properties
2 required
ChatCompletionTool
object
A function tool that can be used to generate a response.
2 properties
2 required
EvalGraderStringCheck
object
InlineSkillParam
object
4 properties
4 required
EvalItemContentItem
A single content item: input text, output text, input image, or input audio.
CreateEvalCompletionsRunDataSource
object
A CompletionsRunDataSource object describing a model sampling configuration.
5 properties
2 required
EvalRunOutputItemList
object
An object representing a list of output items for an evaluation run.
5 properties
5 required
FunctionObject
object
4 properties
1 required
ContainerMemoryLimit
string
InputMessageContentList
array
A list of one or many input items to the model, containing different content types.
EvalRunList
object
An object representing a list of runs for an evaluation.
5 properties
5 required
FunctionTool
object
Defines a function in your own code the model can choose to call. Learn more about [function calling](https://developers.openai.com/api/docs/guides/function-ca…
9 properties
4 required
EvalCustomDataSourceConfig
object
A CustomDataSourceConfig which specifies the schema of your item and optionally sample namespaces. The response schema defines the shape of the data that will…
2 properties
2 required
ContainerReferenceParam
object
2 properties
2 required
EasyInputMessage
object
A message input to the model with a role indicating instruction following hierarchy. Instructions given with the developer or system role take precedence over…
4 properties
2 required
FunctionToolParam
object
9 properties
2 required
InputFileContent
object
A file input to the model.
7 properties
1 required
ImageGenTool
object
A tool that generates images using the GPT image models.
12 properties
1 required
ErrorResponse
object
1 property
1 required
CodeInterpreterTool
object
A tool that runs Python code to help generate a response to a prompt.
3 properties
2 required
InputAudio
object
An audio input to the model.
2 properties
2 required
ContainerNetworkPolicyDisabledParam
object
1 property
1 required
CreateEvalLabelModelGrader
object
A LabelModelGrader object which uses a model to assign labels to each item in the evaluation.
6 properties
6 required
CustomGrammarFormatParam
object
A grammar defined by the user.
3 properties
3 required
EvalStoredCompletionsSource
object
A StoredCompletionsRunDataSource configuration describing a set of filters
6 properties
1 required
EvalJsonlFileContentSource
object
2 properties
2 required
EvalItemContentOutputText
object
A text output from the model.
2 properties
2 required
EvalList
object
An object representing a list of evals.
5 properties
5 required
EvalGraderLabelModel
object
EvalItemContentArray
array
A list of inputs, each of which may be either an input text, output text, input image, or input audio object.
CreateEvalStoredCompletionsDataSourceConfig
object
Deprecated in favor of LogsDataSourceConfig.
2 properties
1 required
GrammarSyntax1
string
CreateEvalJsonlRunDataSource
object
A JsonlRunDataSource object with that specifies a JSONL file that matches the eval
2 properties
2 required
MisalignmentErrorDetailsResource
object
3 properties
EvalItem
object
A message input to the model with a role indicating instruction following hierarchy. Instructions given with the developer or system role take precedence over…
3 properties
2 required
FunctionParameters
object
The parameters the functions accepts, described as a JSON Schema object. See the [guide](https://developers.openai.com/api/docs/guides/function-calling) for ex…
Tool
A tool that can be used to generate a response.
TextResponseFormatConfiguration
An object specifying the format that the model must output. Configuring { "type": "jsonschema" } enables Structured Outputs, which ensures the model will match…
FunctionShellToolParam
object
A tool that allows the model to execute shell commands.
3 properties
1 required
InputImageContent
object
An image input to the model. Learn about [image inputs](https://developers.openai.com/api/docs/guides/images-vision).
5 properties
2 required
CreateEvalRunRequest
object
3 properties
1 required
ApplyPatchToolParam
object
Allows the assistant to create, delete, or update files using unified diffs.
2 properties
1 required
ComputerEnvironment
string
NamespaceToolParam
object
Groups function/custom tools under a shared namespace.
4 properties
4 required
LocalSkillParam
object
3 properties
3 required
FileSearchTool
object
A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://developers.openai.com/api/docs/guides/tools-file…
5 properties
2 required
ResponseFormatJsonSchema
object
JSON Schema response format. Used to generate structured JSON responses. Learn more about [Structured Outputs](https://developers.openai.com/api/docs/guides/st…
2 properties
2 required
RankerVersionType
string
CreateEvalResponsesRunDataSource
object
A ResponsesRunDataSource object describing a model sampling configuration.
5 properties
2 required
EvalGraderPython
object
LocalShellToolParam
object
A tool that allows the model to execute shell commands in a local environment.
1 property
1 required
_MisalignmentSteer
object
1 property
1 required
CreateEvalCustomDataSourceConfig
object
A CustomDataSourceConfig object that defines the schema for the data source used for the evaluation runs. This schema is used to define the shape of the data t…
3 properties
2 required
ReasoningEffort
TextResponseFormatJsonSchema
object
JSON Schema response format. Used to generate structured JSON responses. Learn more about [Structured Outputs](https://developers.openai.com/api/docs/guides/st…
5 properties
3 required
ComparisonFilter
object
A filter used to compare a specified attribute key to a given value using a defined comparison operation.
3 properties
3 required
EvalJsonlFileIdSource
object
2 properties
2 required
EvalRun
object
A schema representing an evaluation run.
14 properties
14 required
EvalItemContentText
string
A text input to the model.
SearchContextSize
string
CreateEvalItem
object
A chat message that makes up the prompt or context. May include variable references to the item namespace, ie {{item.name}}.
CallableToolAllowedCaller
string
EvalGraderScoreModel
object
ToolSearchToolParam
object
Hosted or BYOT tool search configuration for deferred tools.
4 properties
1 required
EvalResponsesSource
object
A EvalResponsesSource object describing a run data source configuration.
11 properties
1 required
GraderPython
object
A PythonGrader object that runs a python script on the input.
4 properties
3 required
MCPToolFilter
object
A filter object to specify which tools are allowed.
2 properties
GraderLabelModel
object
A LabelModelGrader object which uses a model to assign labels to each item in the evaluation.
6 properties
6 required
ApproximateLocation
object
5 properties
1 required
RankingOptions
object
3 properties
Error
object
5 properties
4 required
Filters
GraderTextSimilarity
object
A TextSimilarityGrader object which grades text based on similarity metrics.
5 properties
5 required
SkillReferenceParam
object
3 properties
2 required
InputContent
EmptyModelParam
object
GraderStringCheck
object
A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
5 properties
5 required
EvalItemInputImage
object
An image input block used within EvalItem content arrays.
3 properties
2 required
Metadata
InputTextContent
object
A text input to the model.
3 properties
2 required
ResponseFormatJsonSchemaSchema
object
The schema for the response format, described as a JSON Schema object. Learn how to build JSON schemas [here](https://json-schema.org/).
EvalGraderTextSimilarity
object
EvalRunOutputItem
object
A schema representing an evaluation run output item.
10 properties
10 required
ContainerNetworkPolicyAllowlistParam
object
3 properties
2 required
PromptCacheBreakpointConfig
object
Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request's promptcacheoptions.ttl; the boundary is not rounded to a to…
1 property
1 required
WebSearchApproximateLocation
CreateEvalLogsDataSourceConfig
object
A data source config which specifies the metadata property of your logs query. This is usually metadata like usecase=chatbot or prompt-version=v2, etc.
2 properties
1 required
MCPTool
object
Give the model access to additional tools via remote Model Context Protocol (MCP) servers. [Learn more about MCP](https://developers.openai.com/api/docs/guides…
12 properties
2 required
LocalEnvironmentParam
object
2 properties
1 required
InputFidelity
string
Control how much effort the model will exert to match the style and features, especially facial features, of input images. This parameter is only supported for…
ContainerNetworkPolicyDomainSecretParam
object
3 properties
3 required
ImageDetail
string
MessagePhase
string
Labels an assistant message as intermediate commentary (commentary) or the final answer (finalanswer). For models like gpt-5.3-codex and beyond, when sending f…
_MisalignmentErrorType
CompoundFilter
object
Combine multiple filters using and or or.
2 properties
2 required
ProgrammaticToolCallingParam
object
1 property
1 required
CustomTextFormatParam
object
Unconstrained free-form text.
1 property
1 required
ResponseFormatJsonObject
object
JSON object response format. An older method of generating JSON responses. Using jsonschema is recommended for models that support it. Note that the model will…
1 property
1 required
ImageGenActionEnum
string
WebSearchTool
object
Search the Internet for sources related to the prompt. Learn more about the [web search tool](https://developers.openai.com/api/docs/guides/tools-web-search).
5 properties
1 required
EvalRunOutputItemResult
object
A single grader result for an evaluation run output item.
5 properties
3 required
ComputerTool
object
A tool that controls a virtual computer. Learn more about the [computer tool](https://developers.openai.com/api/docs/guides/tools-computer-use).
1 property
1 required
ToolSearchExecutionType
string
ContainerAutoParam
object
5 properties
1 required
Eval
object
An Eval object with a data source config and testing criteria. An Eval represents a task to be done for your LLM integration. Like: - Improve the quality of my…
7 properties
7 required
EvalStoredCompletionsDataSourceConfig
object
Deprecated in favor of LogsDataSourceConfig.
3 properties
2 required
WebSearchPreviewTool
object
This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](https://developers.openai.com/api/docs/guides/tool…
4 properties
1 required
AutoCodeInterpreterToolParam
object
Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.
4 properties
1 required
GraderScoreModel
object
A ScoreModelGrader object that uses a model to assign a score to the input.
6 properties
4 required
SearchContentType
string
FileInputDetail
string
Specification
The full machine-readable OpenAPI contract behind this narrative.
Source
More from OpenAI 12
Other APIs OpenAI publishes across the network.
OpenAI Responses API
OpenAI Moderations API
OpenAI Batch API
OpenAI Vector Stores API
OpenAI Uploads API
OpenAI Realtime API
OpenAI Videos API
OpenAI Conversations API
OpenAI Containers API
OpenAI ChatKit API
OpenAI Skills API
OpenAI Agents SDK
#Evals
Where this information came from
This is an independent, third-party profile of OpenAI Evals API, published by API Evangelist. We do not operate, host, resell, or support these APIs, and we are not affiliated with or endorsed by the company unless stated above. Everything here is built from publicly available information — the company's own site, developer portal, documentation, public repositories, and the specifications it publishes for public use. Nothing is obtained by breaching a system, defeating an access control, or using credentials.
The Kin Score and Agent Readiness rating are independently calculated assessments of a company's public API artifacts, scored against a published rubric. They are not certifications, endorsements, security assessments, or audits.
Corrections, re-scores, and removal are free — no partnership or purchase required, and you do not need to justify the request. A removed company is recorded as unrated, never scored zero for having asked. Acknowledgement within one business day; removal within two.
info@apievangelist.com
·
Read the full data-sourcing policy →
On a security or compliance team? Put security in the subject line and
you will get a person, not a form — we will tell you exactly which public URLs this profile was built from.