OpenAI Evals API
Manage and run evals in the OpenAI platform.
OpenAI Evals API is one of 51 APIs that OpenAI publishes on the APIs.io network, described by a machine-readable OpenAPI specification.
This API exposes 2 JSON Schema definitions.
Tagged areas include Evals. The published artifact set on APIs.io includes an OpenAPI specification, API documentation, a JSON-LD context, and 2 JSON Schemas.
This API exposes 12 operations across 6 paths, and defines 112 schemas. It is described by OpenAPI 3.0.0, at version 2.0.0.
Requests are made against a single base URL, https://api.openai.com/v1.
12 operations
6 paths
112 schemas
2 DELETE6 GET4 POST
Metadata
The identity and technical contract details declared by the specification.
Authentication & Security 1
OpenAI Evals API declares
1 security scheme
for authenticating requests.
It accepts HTTP bearer tokens (ApiKeyAuth).
By default, every request must be authenticated.
Paths & Operations 12
Across 6 paths, the API surfaces 12 operations — 2 DELETE, 6 GET, 4 POST. Each is listed below with its method, path, parameters, and response codes.
Evals 12
Manage and run evals in the OpenAI platform.
GET
/evals
List evaluations for a project.
POST
/evals
Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the dat…
GET
/evals/{eval_id}
Get an evaluation by ID.
POST
/evals/{eval_id}
Update certain properties of an evaluation.
DELETE
/evals/{eval_id}
Delete an evaluation.
GET
/evals/{eval_id}/runs
Get a list of runs for an evaluation.
POST
/evals/{eval_id}/runs
Kicks off a new run for a given evaluation, specifying the data source, and what model configuration to use to test. The datasource will be validated against the schema specified in the config of the…
GET
/evals/{eval_id}/runs/{run_id}
Get an evaluation run by ID.
POST
/evals/{eval_id}/runs/{run_id}
Cancel an ongoing evaluation run.
DELETE
/evals/{eval_id}/runs/{run_id}
Delete an eval run.
GET
/evals/{eval_id}/runs/{run_id}/output_items
Get a list of output items for an evaluation run.
GET
/evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}
Get an evaluation run output item by ID.
Schemas 112
The contract defines 112 schemas that model the data the API accepts and returns. The most detailed are EvalRun (14 properties), ImageGenTool (12 properties), EvalResponsesSource (11 properties), EvalRunOutputItem (10 properties). Each schema is shown below with its type and property counts.
EvalRunOutputItemList
object
An object representing a list of output items for an evaluation run.
5 properties
5 required
EvalRunOutputItemResult
object
A single grader result for an evaluation run output item.
5 properties
3 required
EvalItemInputImage
object
An image input block used within EvalItem content arrays.
3 properties
2 required
WebSearchTool
object
Search the Internet for sources related to the prompt. Learn more about the [web search tool](/docs/guides/tools-web-search).
4 properties
1 required
InlineSkillParam
object
4 properties
4 required
ResponseFormatJsonSchema
object
JSON Schema response format. Used to generate structured JSON responses. Learn more about [Structured Outputs](/docs/guides/structured-outputs).
2 properties
2 required
ImageGenActionEnum
string
EvalGraderScoreModel
object
FileInputDetail
string
EvalList
object
An object representing a list of evals.
5 properties
5 required
SearchContentType
string
Eval
object
An Eval object with a data source config and testing criteria. An Eval represents a task to be done for your LLM integration. Like: - Improve the quality of my…
7 properties
7 required
EvalRun
object
A schema representing an evaluation run.
14 properties
14 required
ToolSearchToolParam
object
Hosted or BYOT tool search configuration for deferred tools.
4 properties
1 required
GrammarSyntax1
string
ChatCompletionTool
object
A function tool that can be used to generate a response.
2 properties
2 required
Filters
GraderLabelModel
object
A LabelModelGrader object which uses a model to assign labels to each item in the evaluation.
6 properties
6 required
GraderPython
object
A PythonGrader object that runs a python script on the input.
4 properties
3 required
CreateEvalStoredCompletionsDataSourceConfig
object
Deprecated in favor of LogsDataSourceConfig.
2 properties
1 required
ContainerNetworkPolicyDisabledParam
object
1 property
1 required
ContainerAutoParam
object
5 properties
1 required
RankerVersionType
string
MessagePhase
string
Labels an assistant message as intermediate commentary (commentary) or the final answer (finalanswer). For models like gpt-5.3-codex and beyond, when sending f…
ResponseFormatJsonObject
object
JSON object response format. An older method of generating JSON responses. Using jsonschema is recommended for models that support it. Note that the model will…
1 property
1 required
EvalCustomDataSourceConfig
object
A CustomDataSourceConfig which specifies the schema of your item and optionally sample namespaces. The response schema defines the shape of the data that will…
2 properties
2 required
EvalItem
object
A message input to the model with a role indicating instruction following hierarchy. Instructions given with the developer or system role take precedence over…
3 properties
2 required
FunctionParameters
object
The parameters the functions accepts, described as a JSON Schema object. See the [guide](/docs/guides/function-calling) for examples, and the [JSON Schema refe…
EmptyModelParam
object
InputMessageContentList
array
A list of one or many input items to the model, containing different content types.
ComputerUsePreviewTool
object
A tool that controls a virtual computer. Learn more about the [computer tool](https://platform.openai.com/docs/guides/tools-computer-use).
4 properties
4 required
InlineSkillSourceParam
object
Inline skill payload
3 properties
3 required
ApplyPatchToolParam
object
Allows the assistant to create, delete, or update files using unified diffs.
1 property
1 required
CreateEvalRunRequest
object
3 properties
1 required
ComparisonFilter
object
A filter used to compare a specified attribute key to a given value using a defined comparison operation.
3 properties
3 required
LocalSkillParam
object
3 properties
3 required
ComputerEnvironment
string
EvalResponsesSource
object
A EvalResponsesSource object describing a run data source configuration.
11 properties
1 required
ContainerNetworkPolicyAllowlistParam
object
3 properties
2 required
MCPToolFilter
object
A filter object to specify which tools are allowed.
2 properties
EvalApiError
object
An object representing an error response from the Eval API.
2 properties
2 required
NamespaceToolParam
object
Groups function/custom tools under a shared namespace.
4 properties
4 required
EvalItemContentItem
A single content item: input text, output text, input image, or input audio.
ResponseFormatText
object
Default response format. Used to generate text responses.
1 property
1 required
EasyInputMessage
object
A message input to the model with a role indicating instruction following hierarchy. Instructions given with the developer or system role take precedence over…
4 properties
2 required
Tool
A tool that can be used to generate a response.
LocalShellToolParam
object
A tool that allows the model to execute shell commands in a local environment.
1 property
1 required
EvalItemContent
Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items.
CustomTextFormatParam
object
Unconstrained free-form text.
1 property
1 required
MCPTool
object
Give the model access to additional tools via remote Model Context Protocol (MCP) servers. [Learn more about MCP](/docs/guides/tools-remote-mcp).
10 properties
2 required
InputImageContent
object
An image input to the model. Learn about [image inputs](/docs/guides/vision).
4 properties
2 required
LocalEnvironmentParam
object
2 properties
1 required
SearchContextSize
string
ApproximateLocation
object
5 properties
1 required
FunctionObject
object
4 properties
1 required
GraderScoreModel
object
A ScoreModelGrader object that uses a model to assign a score to the input.
6 properties
4 required
InputTextContent
object
A text input to the model.
2 properties
2 required
EvalGraderTextSimilarity
object
EvalGraderLabelModel
object
EvalItemContentOutputText
object
A text output from the model.
2 properties
2 required
RankingOptions
object
3 properties
Error
object
4 properties
4 required
FileSearchTool
object
A tool that searches for relevant content from uploaded files. Learn more about the [file search tool](https://platform.openai.com/docs/guides/tools-file-searc…
5 properties
2 required
EvalStoredCompletionsDataSourceConfig
object
Deprecated in favor of LogsDataSourceConfig.
3 properties
2 required
ContainerMemoryLimit
string
GraderStringCheck
object
A StringCheckGrader object that performs a string comparison between input and reference using a specified operation.
5 properties
5 required
CreateEvalCustomDataSourceConfig
object
A CustomDataSourceConfig object that defines the schema for the data source used for the evaluation runs. This schema is used to define the shape of the data t…
3 properties
2 required
FunctionShellToolParam
object
A tool that allows the model to execute shell commands.
2 properties
1 required
SkillReferenceParam
object
3 properties
2 required
EvalItemContentText
string
A text input to the model.
InputContent
CreateEvalCompletionsRunDataSource
object
A CompletionsRunDataSource object describing a model sampling configuration.
5 properties
2 required
WebSearchPreviewTool
object
This tool searches the web for relevant results to use in a response. Learn more about the [web search tool](https://platform.openai.com/docs/guides/tools-web-…
4 properties
1 required
EvalRunOutputItem
object
A schema representing an evaluation run output item.
10 properties
10 required
CreateEvalRequest
object
4 properties
2 required
CompoundFilter
object
Combine multiple filters using and or or.
2 properties
2 required
CodeInterpreterTool
object
A tool that runs Python code to help generate a response to a prompt.
2 properties
2 required
CreateEvalJsonlRunDataSource
object
A JsonlRunDataSource object with that specifies a JSONL file that matches the eval
2 properties
2 required
ComputerTool
object
A tool that controls a virtual computer. Learn more about the [computer tool](https://platform.openai.com/docs/guides/tools-computer-use).
1 property
1 required
EvalJsonlFileContentSource
object
2 properties
2 required
EvalItemContentArray
array
A list of inputs, each of which may be either an input text, output text, input image, or input audio object.
AutoCodeInterpreterToolParam
object
Configuration for a code interpreter container. Optionally specify the IDs of the files to run the code on.
4 properties
1 required
EvalGraderPython
object
CustomToolParam
object
A custom tool that processes input using a specified format. Learn more about [custom tools](/docs/guides/function-callingcustom-tools)
5 properties
2 required
InputAudio
object
An audio input to the model.
2 properties
2 required
FunctionTool
object
Defines a function in your own code the model can choose to call. Learn more about [function calling](https://platform.openai.com/docs/guides/function-calling).
6 properties
4 required
ContainerNetworkPolicyDomainSecretParam
object
3 properties
3 required
InputFileContent
object
A file input to the model.
6 properties
1 required
HybridSearchOptions
object
2 properties
2 required
CreateEvalResponsesRunDataSource
object
A ResponsesRunDataSource object describing a model sampling configuration.
5 properties
2 required
CreateEvalLogsDataSourceConfig
object
A data source config which specifies the metadata property of your logs query. This is usually metadata like usecase=chatbot or prompt-version=v2, etc.
2 properties
1 required
EvalRunList
object
An object representing a list of runs for an evaluation.
5 properties
5 required
CreateEvalLabelModelGrader
object
A LabelModelGrader object which uses a model to assign labels to each item in the evaluation.
6 properties
6 required
EvalGraderStringCheck
object
ReasoningEffort
EvalLogsDataSourceConfig
object
A LogsDataSourceConfig which specifies the metadata property of your logs query. This is usually metadata like usecase=chatbot or prompt-version=v2, etc. The s…
3 properties
2 required
TextResponseFormatJsonSchema
object
JSON Schema response format. Used to generate structured JSON responses. Learn more about [Structured Outputs](/docs/guides/structured-outputs).
5 properties
3 required
ToolSearchExecutionType
string
FunctionToolParam
object
6 properties
2 required
EvalStoredCompletionsSource
object
A StoredCompletionsRunDataSource configuration describing a set of filters
6 properties
1 required
GraderTextSimilarity
object
A TextSimilarityGrader object which grades text based on similarity metrics.
5 properties
5 required
Metadata
InputFidelity
string
Control how much effort the model will exert to match the style and features, especially facial features, of input images. This parameter is only supported for…
TextResponseFormatConfiguration
An object specifying the format that the model must output. Configuring { "type": "jsonschema" } enables Structured Outputs, which ensures the model will match…
ImageDetail
string
WebSearchApproximateLocation
EvalJsonlFileIdSource
object
2 properties
2 required
ResponseFormatJsonSchemaSchema
object
The schema for the response format, described as a JSON Schema object. Learn how to build JSON schemas [here](https://json-schema.org/).
ContainerReferenceParam
object
2 properties
2 required
CreateEvalItem
object
A chat message that makes up the prompt or context. May include variable references to the item namespace, ie {{item.name}}.
CustomGrammarFormatParam
object
A grammar defined by the user.
3 properties
3 required
ImageGenTool
object
A tool that generates images using the GPT image models.
12 properties
1 required
Specification
The full machine-readable OpenAPI contract behind this narrative.
Source
More from OpenAI 12
Other APIs OpenAI publishes across the network.
OpenAI Responses API
OpenAI Moderations API
OpenAI Batch API
OpenAI Vector Stores API
OpenAI Uploads API
OpenAI Realtime API
OpenAI Videos API
OpenAI Conversations API
OpenAI Containers API
OpenAI ChatKit API
OpenAI Skills API
OpenAI Agents SDK
#Evals
Where this information came from
This is an independent, third-party profile of OpenAI Evals API, published by API Evangelist. We do not operate, host, resell, or support these APIs, and we are not affiliated with or endorsed by the company unless stated above. Everything here is built from publicly available information — the company's own site, developer portal, documentation, public repositories, and the specifications it publishes for public use. Nothing is obtained by breaching a system, defeating an access control, or using credentials.
The Kin Score and Agent Readiness rating are independently calculated assessments of a company's public API artifacts, scored against a published rubric. They are not certifications, endorsements, security assessments, or audits.
Corrections, re-scores, and removal are free — no partnership or purchase required, and you do not need to justify the request. A removed company is recorded as unrated, never scored zero for having asked. Acknowledgement within one business day; removal within two.
info@apievangelist.com
·
Read the full data-sourcing policy →
On a security or compliance team? Put security in the subject line and
you will get a person, not a form — we will tell you exactly which public URLs this profile was built from.