Vellum AI subpackage_testSuiteRuns API
The subpackage_testSuiteRuns API from Vellum AI — 3 operation(s) for subpackage_testsuiteruns.
Vellum AI subpackage_testSuiteRuns API is one of 16 APIs that Vellum AI publishes on the APIs.io network, described by a machine-readable OpenAPI specification.
Tagged areas include subpackage_testSuiteRuns. The published artifact set on APIs.io includes an OpenAPI specification.
This API exposes 3 operations across 3 paths, and defines 178 schemas. It is described by OpenAPI 3.1.0, at version 1.0.0.
Requests are made against 3 base URLs: https://predict.vellum.ai, https://api.vellum.ai, https://documents.vellum.ai.
3 operations
3 paths
178 schemas
2 GET1 POST
Metadata
The identity and technical contract details declared by the specification.
Authentication & Security 1
Vellum AI subpackage_testSuiteRuns API declares
1 security scheme
for authenticating requests.
An API key is passed in the header as X-API-KEY (default).
Paths & Operations 3
Across 3 paths, the API surfaces 3 operations — 2 GET, 1 POST. Each is listed below with its method, path, parameters, and response codes.
subpackage_testSuiteRuns 3
POST
/v1/test-suite-runs
Create
GET
/v1/test-suite-runs/{id}
Retrieve
GET
/v1/test-suite-runs/{id}/executions
List Executions
Schemas 178
The contract defines 178 schemas that model the data the API accepts and returns. The most detailed are TestSuiteRunRead (6 properties), SearchResultRequest (5 properties), SearchResult (5 properties), ChatMessageRequest (5 properties). Each schema is shown below with its type and property counts.
TestSuiteRunRead
object
6 properties
4 required
ArrayChatMessageContentRequest
object
A list of chat message content items.
2 properties
2 required
ErrorVellumValue
object
A value representing an Error.
2 properties
2 required
PdfEnum
string
VideoVellumValueRequest
object
A base Vellum primitive value representing a video.
2 properties
2 required
TestSuiteRunExternalExecConfigDataRequest
object
1 property
1 required
FunctionCallRequest
object
The final resolved function call value.
3 properties
2 required
TestSuiteRunMetricErrorOutput
object
Output for a test suite run metric that is of type ERROR
3 properties
2 required
TestSuiteRunExecutionStringOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type STRING
4 properties
4 required
DocumentVellumValueRequest
object
A base Vellum primitive value representing a document.
2 properties
2 required
SearchResultsVellumValue
object
A value representing Search Results.
2 properties
2 required
ChatHistoryVellumValue
object
A value representing Chat History.
2 properties
2 required
TestSuiteRunMetricStringOutputTypeEnum
string
STRING - STRING
NamedTestCaseVideoVariableValue
object
3 properties
3 required
TestSuiteRunDeploymentReleaseTagExecConfig
object
Execution configuration for running a Test Suite against a Prompt Deployment
3 properties
1 required
NamedTestCaseJsonVariableValueRequest
object
Named Test Case value that is of type JSON
3 properties
3 required
TestSuiteRunExecutionJsonOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type JSON
4 properties
4 required
ArrayVellumValue
object
A value representing an array of Vellum variable values.
2 properties
2 required
TestSuiteRunExecutionMetricResult
object
4 properties
2 required
TestSuiteRunExecutionNumberOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type NUMBER
4 properties
4 required
VideoEnum
string
DocumentChatMessageContentRequest
object
A document value that is used in a chat message.
2 properties
2 required
NamedTestCaseSearchResultsVariableValue
object
Named Test Case value that is of type SEARCHRESULTS
3 properties
3 required
VideoChatMessageContent
object
A video value that is used in a chat message.
2 properties
2 required
TestSuiteRunWorkflowReleaseTagExecConfigDataRequest
object
2 properties
1 required
ErrorVellumValueRequest
object
A value representing an Error.
2 properties
2 required
FunctionCallChatMessageContentValueRequest
object
The final resolved function call value.
3 properties
2 required
SearchResultMeta
object
1 property
VideoChatMessageContentRequest
object
A video value that is used in a chat message.
2 properties
2 required
AudioVellumValue
object
A base Vellum primitive value representing audio.
2 properties
2 required
TestSuiteRunWorkflowSandboxExecConfigTypeEnum
string
WORKFLOWSANDBOX - WORKFLOWSANDBOX
TestSuiteRunMetricNumberOutputTypeEnum
string
NUMBER - NUMBER
TestSuiteRunExecutionChatHistoryOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type CHATHISTORY
4 properties
4 required
ArrayChatMessageContent
object
A list of chat message content items.
2 properties
2 required
ExternalTestCaseExecution
object
2 properties
2 required
ChatMessageRole
string
SYSTEM - System ASSISTANT - Assistant USER - User FUNCTION - Function
TestSuiteRunProgress
object
2 properties
2 required
NumberEnum
string
TestSuiteRunExternalExecConfig
object
Execution configuration for running a Vellum Test Suite against an external callable
3 properties
1 required
TestSuiteRunWorkflowSandboxHistoryItemExecConfig
object
Execution configuration for running a Test Suite against a Workflow Sandbox History Item
3 properties
1 required
SearchResultMetaRequest
object
1 property
FunctionCallVellumValueRequest
object
A value representing a Function Call.
2 properties
2 required
ArrayChatMessageContentItemRequest
TestSuiteRunWorkflowSandboxHistoryItemExecConfigData
object
2 properties
2 required
NamedTestCaseFunctionCallVariableValueRequest
object
Named Test Case value that is of type FUNCTIONCALL
3 properties
3 required
TestSuiteRunWorkflowReleaseTagExecConfigTypeEnum
string
WORKFLOWRELEASETAG - WORKFLOWRELEASETAG
TestSuiteRunPromptSandboxHistoryItemExecConfigData
object
2 properties
2 required
StringChatMessageContentRequest
object
A string value that is used in a chat message.
2 properties
2 required
ChatHistoryVellumValueRequest
object
A value representing Chat History.
2 properties
2 required
VellumImage
object
2 properties
1 required
NamedTestCaseChatHistoryVariableValueRequest
object
Named Test Case value that is of type CHATHISTORY
3 properties
3 required
TestSuiteRunDeploymentReleaseTagExecConfigRequest
object
Execution configuration for running a Test Suite against a Prompt Deployment
3 properties
1 required
TestSuiteRunMetricErrorOutputTypeEnum
string
ERROR - ERROR
StringVellumValue
object
A value representing a string.
2 properties
2 required
NamedTestCaseDocumentVariableValueRequest
object
3 properties
3 required
TestSuiteRunWorkflowSandboxExecConfigDataRequest
object
1 property
1 required
NamedTestCaseStringVariableValue
object
Named Test Case value that is of type STRING
3 properties
3 required
TestSuiteRunPromptSandboxHistoryItemExecConfigDataRequest
object
2 properties
2 required
TestSuiteRunExecutionOutput
VellumDocumentRequest
object
2 properties
1 required
ImageVellumValueRequest
object
A base Vellum primitive value representing an image.
2 properties
2 required
VellumDocument
object
2 properties
1 required
VideoVellumValue
object
A base Vellum primitive value representing a video.
2 properties
2 required
PdfSearchResultMetaSource
object
The source of a search result from a PDF document.
3 properties
3 required
TestSuiteRunExecConfig
ArrayVellumValueRequest
object
A value representing an array of Vellum variable values.
2 properties
2 required
ExternalTestCaseExecutionRequest
object
2 properties
2 required
ThinkingVellumValueRequest
object
A value representing Thinking mode output.
2 properties
2 required
ImageChatMessageContentRequest
object
An image value that is used in a chat message.
2 properties
2 required
TestSuiteRunWorkflowSandboxHistoryItemExecConfigDataRequest
object
2 properties
2 required
AudioEnum
string
ImageEnum
string
FunctionCall
object
The final resolved function call value.
3 properties
2 required
JsonVellumValue
object
A value representing a JSON object.
2 properties
2 required
StringChatMessageContent
object
A string value that is used in a chat message.
2 properties
2 required
VellumError
object
3 properties
2 required
TestSuiteRunDeploymentReleaseTagExecConfigDataRequest
object
2 properties
1 required
TestSuiteRunMetricStringOutput
object
Output for a test suite run metric that is of type STRING
3 properties
2 required
NamedTestCaseFunctionCallVariableValue
object
Named Test Case value that is of type FUNCTIONCALL
3 properties
3 required
FunctionCallChatMessageContentValue
object
The final resolved function call value.
3 properties
2 required
TestSuiteRunExecConfigRequest
DocumentChatMessageContent
object
A document value that is used in a chat message.
2 properties
2 required
FunctionCallChatMessageContentRequest
object
A function call value that is used in a chat message.
2 properties
2 required
NamedTestCaseArrayVariableValue
object
Named Test Case value that is of type ARRAY
3 properties
3 required
TestSuiteRunPromptSandboxExecConfigTypeEnum
string
PROMPTSANDBOX - PROMPTSANDBOX
ChatMessageContent
PaginatedTestSuiteRunExecutionList
object
4 properties
4 required
DocumentVellumValue
object
A base Vellum primitive value representing a document.
2 properties
2 required
StringEnum
string
VellumVideoRequest
object
2 properties
1 required
NumberVellumValue
object
A value representing a number.
2 properties
2 required
NamedTestCaseNumberVariableValue
object
Named Test Case value that is of type NUMBER
3 properties
3 required
NamedTestCaseAudioVariableValueRequest
object
Named Test Case value that is of type AUDIO
3 properties
3 required
NamedTestCaseArrayVariableValueRequest
object
Named Test Case value that is of type ARRAY
3 properties
3 required
TestSuiteRunPromptSandboxHistoryItemExecConfigRequest
object
Execution configuration for running a Test Suite against a Prompt Sandbox History Item
3 properties
1 required
SearchResultRequest
object
5 properties
4 required
TestSuiteRunExternalExecConfigTypeEnum
string
EXTERNAL - EXTERNAL
NamedTestCaseSearchResultsVariableValueRequest
object
Named Test Case value that is of type SEARCHRESULTS
3 properties
3 required
ThinkingEnum
string
NamedTestCaseNumberVariableValueRequest
object
Named Test Case value that is of type NUMBER
3 properties
3 required
NamedTestCaseErrorVariableValueRequest
object
Named Test Case value that is of type ERROR
3 properties
3 required
TestSuiteRunWorkflowReleaseTagExecConfigRequest
object
Execution configuration for running a Test Suite against a Workflow Deployment
3 properties
1 required
FunctionCallVellumValue
object
A value representing a Function Call.
2 properties
2 required
NamedTestCaseErrorVariableValue
object
Named Test Case value that is of type ERROR
3 properties
3 required
SearchResultsVellumValueRequest
object
A value representing Search Results.
2 properties
2 required
TestSuiteRunExecutionSearchResultsOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type SEARCHRESULTS
4 properties
4 required
AudioChatMessageContentRequest
object
An audio value that is used in a chat message.
2 properties
2 required
NamedTestCaseChatHistoryVariableValue
object
Named Test Case value that is of type CHATHISTORY
3 properties
3 required
AudioVellumValueRequest
object
A base Vellum primitive value representing audio.
2 properties
2 required
ImageVellumValue
object
A base Vellum primitive value representing an image.
2 properties
2 required
JsonEnum
string
NamedTestCaseImageVariableValueRequest
object
3 properties
3 required
FunctionCallEnum
string
SearchResultsEnum
string
SearchResultDocumentRequest
object
4 properties
1 required
VellumAudioRequest
object
2 properties
1 required
JsonVellumValueRequest
object
A value representing a JSON object.
2 properties
2 required
ArrayEnum
string
TestSuiteRunExecution
object
4 properties
4 required
VellumValueRequest
VellumErrorRequest
object
3 properties
2 required
VellumErrorCodeEnum
string
INVALIDREQUEST - INVALIDREQUEST INVALIDINPUTS - INVALIDINPUTS PROVIDERERROR - PROVIDERERROR PROVIDERCREDENTIALSUNAVAILABLE - PROVIDERCREDENTIALSUNAVAILABLE INT…
VellumVideo
object
2 properties
1 required
TestSuiteRunWorkflowSandboxHistoryItemExecConfigRequest
object
Execution configuration for running a Test Suite against a Workflow Sandbox History Item
3 properties
1 required
NamedTestCaseVideoVariableValueRequest
object
3 properties
3 required
TestSuiteRunMetricJSONOutput
object
Output for a test suite run metric that is of type NUMBER
3 properties
2 required
ErrorEnum
string
TestSuiteRunWorkflowSandboxExecConfigRequest
object
Execution configuration for running a Test Suite against a Workflow Sandbox
3 properties
1 required
TestSuiteRunExternalExecConfigData
object
1 property
1 required
NamedTestCaseDocumentVariableValue
object
3 properties
3 required
TestSuiteRunDeploymentReleaseTagExecConfigData
object
2 properties
1 required
NamedTestCaseVariableValue
PdfSearchResultMetaSourceRequest
object
The source of a search result from a PDF document.
3 properties
3 required
NamedTestCaseImageVariableValue
object
3 properties
3 required
ChatMessageRequest
object
5 properties
1 required
ChatMessage
object
5 properties
1 required
NamedTestCaseStringVariableValueRequest
object
Named Test Case value that is of type STRING
3 properties
3 required
NumberVellumValueRequest
object
A value representing a number.
2 properties
2 required
AudioChatMessageContent
object
An audio value that is used in a chat message.
2 properties
2 required
TestSuiteRunExecutionArrayOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type ARRAY
4 properties
4 required
NamedTestCaseAudioVariableValue
object
Named Test Case value that is of type AUDIO
3 properties
3 required
DocumentEnum
string
TestSuiteRunTestSuite
object
3 properties
3 required
TestSuiteRunMetricJSONOutputTypeEnum
string
JSON - JSON
VellumImageRequest
object
2 properties
1 required
TestSuiteRunMetricNumberOutput
object
Output for a test suite run metric that is of type NUMBER
3 properties
2 required
NamedTestCaseJsonVariableValue
object
Named Test Case value that is of type JSON
3 properties
3 required
TestSuiteRunExternalExecConfigRequest
object
Execution configuration for running a Vellum Test Suite against an external callable
3 properties
1 required
TestSuiteRunMetricArrayOutput
object
Output for a test suite run metric that is of type ARRAY
3 properties
2 required
ChatHistoryEnum
string
TestSuiteRunMetricArrayOutputTypeEnum
string
ARRAY - ARRAY
FunctionCallChatMessageContent
object
A function call value that is used in a chat message.
2 properties
2 required
TestSuiteRunExecutionFunctionCallOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type FUNCTIONCALL
4 properties
4 required
ImageChatMessageContent
object
An image value that is used in a chat message.
2 properties
2 required
SearchResult
object
5 properties
4 required
TestSuiteRunExecutionErrorOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type ERROR
4 properties
4 required
TestSuiteRunWorkflowReleaseTagExecConfig
object
Execution configuration for running a Test Suite against a Workflow Deployment
3 properties
1 required
TestSuiteRunState
string
QUEUED - Queued RUNNING - Running COMPLETE - Complete FAILED - Failed CANCELLED - Cancelled
TestSuiteRunCreateRequest
object
3 properties
1 required
TestSuiteRunWorkflowSandboxHistoryItemExecConfigTypeEnum
string
WORKFLOWSANDBOXHISTORYITEM - WORKFLOWSANDBOXHISTORYITEM
ChatMessageContentRequest
ThinkingVellumValue
object
A value representing Thinking mode output.
2 properties
2 required
VellumAudio
object
2 properties
1 required
SearchResultDocument
object
4 properties
1 required
StringVellumValueRequest
object
A value representing a string.
2 properties
2 required
SearchResultMetaSource
TestSuiteRunPromptSandboxHistoryItemExecConfig
object
Execution configuration for running a Test Suite against a Prompt Sandbox History Item
3 properties
1 required
NamedTestCaseVariableValueRequest
VellumValue
ArrayChatMessageContentItem
TestSuiteRunPromptSandboxExecConfigDataRequest
object
2 properties
2 required
TestSuiteRunPromptSandboxHistoryItemExecConfigTypeEnum
string
PROMPTSANDBOXHISTORYITEM - PROMPTSANDBOXHISTORYITEM
TestSuiteRunPromptSandboxExecConfigRequest
object
Execution configuration for running a Test Suite against a Prompt Sandbox
3 properties
1 required
SearchResultMetaSourceRequest
TestSuiteRunWorkflowReleaseTagExecConfigData
object
2 properties
1 required
TestSuiteRunMetricOutput
TestSuiteRunDeploymentReleaseTagExecConfigTypeEnum
string
DEPLOYMENTRELEASETAG - DEPLOYMENTRELEASETAG
TestSuiteRunExecutionMetricDefinition
object
3 properties
Specification
The full machine-readable OpenAPI contract behind this narrative.
Source
More from Vellum AI 12
Other APIs Vellum AI publishes across the network.
Vellum LLM Platform API
Vellum AI Default API
Vellum AI subpackage_deployments API
Vellum AI subpackage_documentIndexes API
Vellum AI subpackage_documents API
Vellum AI subpackage_events API
Vellum AI subpackage_folderEntities API
Vellum AI subpackage_sandboxes API
Vellum AI subpackage_testSuites API
Vellum AI subpackage_uploadedFiles API
Vellum AI subpackage_workflowDeployments API
Vellum AI subpackage_workflowExecutions API
#subpackage_testSuiteRuns
Where this information came from
This is an independent, third-party profile of Vellum AI subpackage_testSuiteRuns API, published by API Evangelist. We do not operate, host, resell, or support these APIs, and we are not affiliated with or endorsed by the company unless stated above. Everything here is built from publicly available information — the company's own site, developer portal, documentation, public repositories, and the specifications it publishes for public use. Nothing is obtained by breaching a system, defeating an access control, or using credentials.
The Kin Score and Agent Readiness rating are independently calculated assessments of a company's public API artifacts, scored against a published rubric. They are not certifications, endorsements, security assessments, or audits.
Corrections, re-scores, and removal are free — no partnership or purchase required, and you do not need to justify the request. A removed company is recorded as unrated, never scored zero for having asked. Acknowledgement within one business day; removal within two.
info@apievangelist.com
·
Read the full data-sourcing policy →
On a security or compliance team? Put security in the subject line and
you will get a person, not a form — we will tell you exactly which public URLs this profile was built from.