Vellum AI Subpackage Test Suite Runs API
The subpackage_testSuiteRuns API from Vellum AI — 3 operation(s) for subpackage_testsuiteruns.
Vellum AI Subpackage Test Suite Runs API is one of 23 APIs that Vellum AI publishes on the APIs.io network, described by a machine-readable OpenAPI specification.
Tagged areas include subpackage_testSuiteRuns. The published artifact set on APIs.io includes an OpenAPI specification.
This API exposes 3 operations across 3 paths, and defines 178 schemas. It is described by OpenAPI 3.2.0, at version 1.0.0.
Requests are made against 3 base URLs: https://predict.vellum.ai, https://api.vellum.ai, https://documents.vellum.ai.
3 operations
3 paths
178 schemas
2 GET1 POST
Metadata
The identity and technical contract details declared by the specification.
Authentication & Security 1
Vellum AI Subpackage Test Suite Runs API declares
1 security scheme
for authenticating requests.
An API key is passed in the header as X-API-KEY (default).
Paths & Operations 3
Across 3 paths, the API surfaces 3 operations — 2 GET, 1 POST. Each is listed below with its method, path, parameters, and response codes.
subpackage_testSuiteRuns 3
POST
/v1/test-suite-runs
Create
GET
/v1/test-suite-runs/{id}
Retrieve
GET
/v1/test-suite-runs/{id}/executions
List Executions
Schemas 178
The contract defines 178 schemas that model the data the API accepts and returns. The most detailed are TestSuiteRunRead (6 properties), ChatMessage (5 properties), ChatMessageRequest (5 properties), SearchResult (5 properties). Each schema is shown below with its type and property counts.
SearchResultMetaRequest
object
1 property
TestSuiteRunState
string
QUEUED - Queued RUNNING - Running COMPLETE - Complete FAILED - Failed CANCELLED - Cancelled
VellumVideo
object
2 properties
1 required
NamedTestCaseStringVariableValue
object
Named Test Case value that is of type STRING
3 properties
3 required
ArrayChatMessageContentRequest
object
A list of chat message content items.
2 properties
2 required
NamedTestCaseSearchResultsVariableValueRequest
object
Named Test Case value that is of type SEARCHRESULTS
3 properties
3 required
StringVellumValue
object
A value representing a string.
2 properties
2 required
ChatMessageRole
string
SYSTEM - System ASSISTANT - Assistant USER - User FUNCTION - Function
VellumErrorRequest
object
3 properties
2 required
PdfEnum
string
ArrayVellumValue
object
A value representing an array of Vellum variable values.
2 properties
2 required
TestSuiteRunMetricErrorOutputTypeEnum
string
ERROR - ERROR
TestSuiteRunPromptSandboxExecConfigRequest
object
Execution configuration for running a Test Suite against a Prompt Sandbox
3 properties
1 required
ArrayChatMessageContentItemRequest
NamedTestCaseChatHistoryVariableValueRequest
object
Named Test Case value that is of type CHATHISTORY
3 properties
3 required
TestSuiteRunWorkflowSandboxExecConfigDataRequest
object
1 property
1 required
SearchResultMetaSource
TestSuiteRunRead
object
6 properties
4 required
JsonVellumValue
object
A value representing a JSON object.
2 properties
2 required
DocumentVellumValueRequest
object
A base Vellum primitive value representing a document.
2 properties
2 required
VellumVideoRequest
object
2 properties
1 required
NamedTestCaseImageVariableValue
object
3 properties
3 required
SearchResult
object
5 properties
4 required
ChatHistoryVellumValueRequest
object
A value representing Chat History.
2 properties
2 required
ErrorVellumValue
object
A value representing an Error.
2 properties
2 required
TestSuiteRunExternalExecConfig
object
Execution configuration for running a Vellum Test Suite against an external callable
3 properties
1 required
VideoEnum
string
TestSuiteRunExecutionMetricDefinition
object
3 properties
SearchResultsEnum
string
FunctionCallChatMessageContent
object
A function call value that is used in a chat message.
2 properties
2 required
TestSuiteRunExternalExecConfigDataRequest
object
1 property
1 required
TestSuiteRunExecutionOutput
VellumDocumentRequest
object
2 properties
1 required
TestSuiteRunWorkflowSandboxHistoryItemExecConfigTypeEnum
string
WORKFLOWSANDBOXHISTORYITEM - WORKFLOWSANDBOXHISTORYITEM
TestSuiteRunPromptSandboxHistoryItemExecConfigData
object
2 properties
2 required
VellumImageRequest
object
2 properties
1 required
ChatMessageContentRequest
TestSuiteRunExecutionArrayOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type ARRAY
4 properties
4 required
TestSuiteRunPromptSandboxExecConfigDataRequest
object
2 properties
2 required
NamedTestCaseNumberVariableValue
object
Named Test Case value that is of type NUMBER
3 properties
3 required
ChatMessageRequest
object
5 properties
1 required
NumberVellumValue
object
A value representing a number.
2 properties
2 required
NamedTestCaseDocumentVariableValueRequest
object
3 properties
3 required
PaginatedTestSuiteRunExecutionList
object
4 properties
4 required
ImageChatMessageContentRequest
object
An image value that is used in a chat message.
2 properties
2 required
TestSuiteRunWorkflowSandboxHistoryItemExecConfigRequest
object
Execution configuration for running a Test Suite against a Workflow Sandbox History Item
3 properties
1 required
ArrayChatMessageContentItem
ErrorEnum
string
TestSuiteRunMetricNumberOutputTypeEnum
string
NUMBER - NUMBER
NumberVellumValueRequest
object
A value representing a number.
2 properties
2 required
NamedTestCaseArrayVariableValueRequest
object
Named Test Case value that is of type ARRAY
3 properties
3 required
TestSuiteRunMetricJSONOutput
object
Output for a test suite run metric that is of type NUMBER
3 properties
2 required
TestSuiteRunDeploymentReleaseTagExecConfigData
object
2 properties
1 required
VellumErrorCodeEnum
string
INVALIDREQUEST - INVALIDREQUEST INVALIDINPUTS - INVALIDINPUTS PROVIDERERROR - PROVIDERERROR PROVIDERCREDENTIALSUNAVAILABLE - PROVIDERCREDENTIALSUNAVAILABLE INT…
VideoVellumValue
object
A base Vellum primitive value representing a video.
2 properties
2 required
TestSuiteRunWorkflowSandboxHistoryItemExecConfig
object
Execution configuration for running a Test Suite against a Workflow Sandbox History Item
3 properties
1 required
NamedTestCaseSearchResultsVariableValue
object
Named Test Case value that is of type SEARCHRESULTS
3 properties
3 required
StringChatMessageContentRequest
object
A string value that is used in a chat message.
2 properties
2 required
ChatMessage
object
5 properties
1 required
VellumValue
TestSuiteRunMetricStringOutput
object
Output for a test suite run metric that is of type STRING
3 properties
2 required
AudioVellumValue
object
A base Vellum primitive value representing audio.
2 properties
2 required
NamedTestCaseDocumentVariableValue
object
3 properties
3 required
ArrayEnum
string
TestSuiteRunMetricStringOutputTypeEnum
string
STRING - STRING
TestSuiteRunPromptSandboxHistoryItemExecConfig
object
Execution configuration for running a Test Suite against a Prompt Sandbox History Item
3 properties
1 required
NamedTestCaseVariableValueRequest
TestSuiteRunExecutionNumberOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type NUMBER
4 properties
4 required
TestSuiteRunMetricOutput
NamedTestCaseStringVariableValueRequest
object
Named Test Case value that is of type STRING
3 properties
3 required
TestSuiteRunExecutionSearchResultsOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type SEARCHRESULTS
4 properties
4 required
NamedTestCaseJsonVariableValue
object
Named Test Case value that is of type JSON
3 properties
3 required
PdfSearchResultMetaSource
object
The source of a search result from a PDF document.
3 properties
3 required
NamedTestCaseArrayVariableValue
object
Named Test Case value that is of type ARRAY
3 properties
3 required
TestSuiteRunPromptSandboxHistoryItemExecConfigDataRequest
object
2 properties
2 required
TestSuiteRunMetricArrayOutput
object
Output for a test suite run metric that is of type ARRAY
3 properties
2 required
TestSuiteRunWorkflowReleaseTagExecConfigDataRequest
object
2 properties
1 required
TestSuiteRunDeploymentReleaseTagExecConfigTypeEnum
string
DEPLOYMENTRELEASETAG - DEPLOYMENTRELEASETAG
ArrayChatMessageContent
object
A list of chat message content items.
2 properties
2 required
VellumAudio
object
2 properties
1 required
StringVellumValueRequest
object
A value representing a string.
2 properties
2 required
FunctionCallChatMessageContentValueRequest
object
The final resolved function call value.
3 properties
2 required
PdfSearchResultMetaSourceRequest
object
The source of a search result from a PDF document.
3 properties
3 required
TestSuiteRunDeploymentReleaseTagExecConfigDataRequest
object
2 properties
1 required
TestSuiteRunExecution
object
4 properties
4 required
FunctionCallRequest
object
The final resolved function call value.
3 properties
2 required
ArrayVellumValueRequest
object
A value representing an array of Vellum variable values.
2 properties
2 required
NamedTestCaseErrorVariableValue
object
Named Test Case value that is of type ERROR
3 properties
3 required
FunctionCall
object
The final resolved function call value.
3 properties
2 required
TestSuiteRunExecConfig
DocumentEnum
string
TestSuiteRunProgress
object
2 properties
2 required
ChatMessageContent
AudioChatMessageContent
object
An audio value that is used in a chat message.
2 properties
2 required
VideoVellumValueRequest
object
A base Vellum primitive value representing a video.
2 properties
2 required
VellumAudioRequest
object
2 properties
1 required
SearchResultsVellumValue
object
A value representing Search Results.
2 properties
2 required
SearchResultRequest
object
5 properties
4 required
AudioChatMessageContentRequest
object
An audio value that is used in a chat message.
2 properties
2 required
FunctionCallChatMessageContentValue
object
The final resolved function call value.
3 properties
2 required
ChatHistoryEnum
string
SearchResultMeta
object
1 property
FunctionCallVellumValue
object
A value representing a Function Call.
2 properties
2 required
NamedTestCaseNumberVariableValueRequest
object
Named Test Case value that is of type NUMBER
3 properties
3 required
TestSuiteRunWorkflowSandboxExecConfigRequest
object
Execution configuration for running a Test Suite against a Workflow Sandbox
3 properties
1 required
FunctionCallEnum
string
TestSuiteRunCreateRequest
object
3 properties
1 required
TestSuiteRunExecutionFunctionCallOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type FUNCTIONCALL
4 properties
4 required
ExternalTestCaseExecution
object
2 properties
2 required
TestSuiteRunWorkflowSandboxExecConfigTypeEnum
string
WORKFLOWSANDBOX - WORKFLOWSANDBOX
DocumentVellumValue
object
A base Vellum primitive value representing a document.
2 properties
2 required
TestSuiteRunWorkflowSandboxHistoryItemExecConfigDataRequest
object
2 properties
2 required
VellumImage
object
2 properties
1 required
NamedTestCaseAudioVariableValue
object
Named Test Case value that is of type AUDIO
3 properties
3 required
NumberEnum
string
VellumValueRequest
VellumDocument
object
2 properties
1 required
ThinkingEnum
string
NamedTestCaseErrorVariableValueRequest
object
Named Test Case value that is of type ERROR
3 properties
3 required
JsonEnum
string
ImageVellumValueRequest
object
A base Vellum primitive value representing an image.
2 properties
2 required
TestSuiteRunMetricErrorOutput
object
Output for a test suite run metric that is of type ERROR
3 properties
2 required
TestSuiteRunWorkflowReleaseTagExecConfig
object
Execution configuration for running a Test Suite against a Workflow Deployment
3 properties
1 required
SearchResultMetaSourceRequest
ErrorVellumValueRequest
object
A value representing an Error.
2 properties
2 required
VellumError
object
3 properties
2 required
TestSuiteRunDeploymentReleaseTagExecConfig
object
Execution configuration for running a Test Suite against a Prompt Deployment
3 properties
1 required
TestSuiteRunWorkflowReleaseTagExecConfigData
object
2 properties
1 required
TestSuiteRunExecutionStringOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type STRING
4 properties
4 required
NamedTestCaseFunctionCallVariableValueRequest
object
Named Test Case value that is of type FUNCTIONCALL
3 properties
3 required
NamedTestCaseImageVariableValueRequest
object
3 properties
3 required
TestSuiteRunDeploymentReleaseTagExecConfigRequest
object
Execution configuration for running a Test Suite against a Prompt Deployment
3 properties
1 required
TestSuiteRunWorkflowSandboxHistoryItemExecConfigData
object
2 properties
2 required
FunctionCallChatMessageContentRequest
object
A function call value that is used in a chat message.
2 properties
2 required
ThinkingVellumValue
object
A value representing Thinking mode output.
2 properties
2 required
VideoChatMessageContentRequest
object
A video value that is used in a chat message.
2 properties
2 required
DocumentChatMessageContentRequest
object
A document value that is used in a chat message.
2 properties
2 required
TestSuiteRunExecutionErrorOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type ERROR
4 properties
4 required
DocumentChatMessageContent
object
A document value that is used in a chat message.
2 properties
2 required
ImageChatMessageContent
object
An image value that is used in a chat message.
2 properties
2 required
NamedTestCaseFunctionCallVariableValue
object
Named Test Case value that is of type FUNCTIONCALL
3 properties
3 required
NamedTestCaseVideoVariableValueRequest
object
3 properties
3 required
ChatHistoryVellumValue
object
A value representing Chat History.
2 properties
2 required
NamedTestCaseVideoVariableValue
object
3 properties
3 required
StringChatMessageContent
object
A string value that is used in a chat message.
2 properties
2 required
TestSuiteRunPromptSandboxHistoryItemExecConfigRequest
object
Execution configuration for running a Test Suite against a Prompt Sandbox History Item
3 properties
1 required
ImageVellumValue
object
A base Vellum primitive value representing an image.
2 properties
2 required
TestSuiteRunMetricNumberOutput
object
Output for a test suite run metric that is of type NUMBER
3 properties
2 required
SearchResultsVellumValueRequest
object
A value representing Search Results.
2 properties
2 required
TestSuiteRunTestSuite
object
3 properties
3 required
FunctionCallVellumValueRequest
object
A value representing a Function Call.
2 properties
2 required
ExternalTestCaseExecutionRequest
object
2 properties
2 required
TestSuiteRunExternalExecConfigData
object
1 property
1 required
TestSuiteRunExecConfigRequest
TestSuiteRunWorkflowReleaseTagExecConfigTypeEnum
string
WORKFLOWRELEASETAG - WORKFLOWRELEASETAG
TestSuiteRunExternalExecConfigTypeEnum
string
EXTERNAL - EXTERNAL
TestSuiteRunExecutionChatHistoryOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type CHATHISTORY
4 properties
4 required
TestSuiteRunExternalExecConfigRequest
object
Execution configuration for running a Vellum Test Suite against an external callable
3 properties
1 required
ImageEnum
string
TestSuiteRunExecutionJsonOutput
object
Execution output of an entity evaluated during a Test Suite Run that is of type JSON
4 properties
4 required
TestSuiteRunMetricArrayOutputTypeEnum
string
ARRAY - ARRAY
TestSuiteRunExecutionMetricResult
object
4 properties
2 required
ThinkingVellumValueRequest
object
A value representing Thinking mode output.
2 properties
2 required
TestSuiteRunMetricJSONOutputTypeEnum
string
JSON - JSON
AudioEnum
string
VideoChatMessageContent
object
A video value that is used in a chat message.
2 properties
2 required
SearchResultDocumentRequest
object
4 properties
1 required
StringEnum
string
NamedTestCaseAudioVariableValueRequest
object
Named Test Case value that is of type AUDIO
3 properties
3 required
JsonVellumValueRequest
object
A value representing a JSON object.
2 properties
2 required
SearchResultDocument
object
4 properties
1 required
TestSuiteRunPromptSandboxHistoryItemExecConfigTypeEnum
string
PROMPTSANDBOXHISTORYITEM - PROMPTSANDBOXHISTORYITEM
NamedTestCaseVariableValue
NamedTestCaseChatHistoryVariableValue
object
Named Test Case value that is of type CHATHISTORY
3 properties
3 required
NamedTestCaseJsonVariableValueRequest
object
Named Test Case value that is of type JSON
3 properties
3 required
TestSuiteRunPromptSandboxExecConfigTypeEnum
string
PROMPTSANDBOX - PROMPTSANDBOX
AudioVellumValueRequest
object
A base Vellum primitive value representing audio.
2 properties
2 required
TestSuiteRunWorkflowReleaseTagExecConfigRequest
object
Execution configuration for running a Test Suite against a Workflow Deployment
3 properties
1 required
Specification
The full machine-readable OpenAPI contract behind this narrative.
Source
More from Vellum AI 12
Other APIs Vellum AI publishes across the network.
Vellum LLM Platform API
Vellum AI Subpackage Deployments API
Vellum AI Subpackage Document Indexes API
Vellum AI Subpackage Documents API
Vellum AI Subpackage Events API
Vellum AI Subpackage Folder Entities API
Vellum AI Subpackage Sandboxes API
Vellum AI Subpackage Test Suites API
Vellum AI Subpackage Uploaded Files API
Vellum AI Subpackage Workflow Deployments API
Vellum AI Subpackage Workflow Executions API
Vellum AI Subpackage Workflows API
#subpackage_testSuiteRuns
Where this information came from
This is an independent, third-party profile of Vellum AI Subpackage Test Suite Runs API, published by API Evangelist. We do not operate, host, resell, or support these APIs, and we are not affiliated with or endorsed by the company unless stated above. Everything here is built from publicly available information — the company's own site, developer portal, documentation, public repositories, and the specifications it publishes for public use. Nothing is obtained by breaching a system, defeating an access control, or using credentials.
The Kin Score and Agent Readiness rating are independently calculated assessments of a company's public API artifacts, scored against a published rubric. They are not certifications, endorsements, security assessments, or audits.
Corrections, re-scores, and removal are free — no partnership or purchase required, and you do not need to justify the request. A removed company is recorded as unrated, never scored zero for having asked. Acknowledgement within one business day; removal within two.
info@apievangelist.com
·
Read the full data-sourcing policy →
On a security or compliance team? Put security in the subject line and
you will get a person, not a form — we will tell you exactly which public URLs this profile was built from.