How AI is applied across API Evangelist and APIs.io. Read my AI disclosure →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

Relevance AI Evals API

The evals API from Relevance AI — 22 operation(s) for evals.

Relevance AI Evals API is one of 32 APIs that Relevance AI publishes on the APIs.io network, described by a machine-readable OpenAPI specification.

Tagged areas include Evals. The published artifact set on APIs.io includes an OpenAPI specification, API documentation, an API reference, and authentication docs.

This API exposes 37 operations across 22 paths, and defines 63 schemas. It is described by OpenAPI 3.2.0, at version latest.

Requests are made against a single base URL, /latest.

37 operations 22 paths 63 schemas 7 DELETE11 GET1 PATCH14 POST4 PUT

Metadata

The identity and technical contract details declared by the specification.

Specification
OpenAPI 3.2.0
API Version
latest
Base URL
https://api-{region}.stack.tryrelevance.com/latest
Authentication
API Key
Resource Areas
1

Authentication & Security 1

Relevance AI Evals API declares 1 security scheme for authenticating requests. An API key is passed in the header as Authorization (AuthorizationHeader). By default, every request must be authenticated.

  • AuthorizationHeader — Authorization credentials. Header authorization should be in the form of: project:apikey

Paths & Operations 37

Across 22 paths, the API surfaces 37 operations — 7 DELETE, 11 GET, 1 PATCH, 14 POST, 4 PUT. Each is listed below with its method, path, parameters, and response codes.

Evals 37
POST
/evals/{resource_type}/{resource_id}/test-cases
CreateTestCase
CreateTestCase 3 params body → 200
GET
/evals/{resource_type}/{resource_id}/test-cases
ListTestCases
ListTestCases 6 params → 200
GET
/evals/{resource_type}/{resource_id}/batches
ListEvalBatchByResourceNew
ListEvalBatchByResourceNew 7 params → 200
POST
/evals/{resource_type}/{resource_id}/rules
CreateEvalRuleForResource
CreateEvalRuleForResource 2 params body → 200
GET
/evals/{resource_type}/{resource_id}/rules
ListEvalRulesForResource
ListEvalRulesForResource 2 params → 200
POST
/evals/{resource_type}/{resource_id}/runs
ListEvalRunsForResource
ListEvalRunsForResource 2 params body → 200
POST
/evals/{resource_type}/{resource_id}/evaluate
EvaluateResource
EvaluateResource 2 params body → 200
GET
/evals/{resource_type}/{resource_id}/batches/{eval_batch_id}/summary
GetEvalBatchSummaryForResource
GetEvalBatchSummaryForResource 3 params → 200
DELETE
/evals/{resource_type}/{resource_id}/batches/{eval_batch_id}
DeleteEvalBatchForResource
DeleteEvalBatchForResource 3 params body → 200
POST
/evals/{resource_type}/{resource_id}/batches/{eval_batch_id}/cancel
CancelEvalBatchForResource
CancelEvalBatchForResource 3 params body → 200
POST
/evals/{resource_type}/{resource_id}/runs/{eval_run_id}/cancel
CancelEvalRunForResource
CancelEvalRunForResource 3 params body → 200
PATCH
/evals/{resource_type}/{resource_id}/rules/{eval_rule_id}
UpdateEvalRuleForResource
UpdateEvalRuleForResource 3 params body → 200
DELETE
/evals/{resource_type}/{resource_id}/rules/{eval_rule_id}
DeleteEvalRuleForResource
DeleteEvalRuleForResource 3 params body → 200
POST
/evals/{resource_type}/{resource_id}/observability/config
CreateObservabilityConfig
CreateObservabilityConfig 2 params body → 200
PUT
/evals/{resource_type}/{resource_id}/observability/config
UpdateObservabilityConfig
UpdateObservabilityConfig 2 params body → 200
GET
/evals/{resource_type}/{resource_id}/observability/config
GetObservabilityConfig
GetObservabilityConfig 2 params → 200
DELETE
/evals/{resource_type}/{resource_id}/observability/config
DeleteObservabilityConfig
DeleteObservabilityConfig 2 params body → 200
POST
/evals/{resource_type}/{resource_id}/observability/configs
CreateObservabilityConfigV2
CreateObservabilityConfigV2 2 params body → 200
GET
/evals/{resource_type}/{resource_id}/observability/configs
ListObservabilityConfigs
ListObservabilityConfigs 2 params → 200
PUT
/evals/{resource_type}/{resource_id}/observability/configs/{config_id}
UpdateObservabilityConfigV2
UpdateObservabilityConfigV2 3 params body → 200
DELETE
/evals/{resource_type}/{resource_id}/observability/configs/{config_id}
DeleteObservabilityConfigV2
DeleteObservabilityConfigV2 3 params body → 200
GET
/evals/{resource_type}/{resource_id}/pre-publish/config
GetPrePublishConfig
GetPrePublishConfig 2 params → 200
POST
/evals/{resource_type}/{resource_id}/pre-publish/config
UpsertPrePublishConfig
UpsertPrePublishConfig 2 params body → 200
DELETE
/evals/{resource_type}/{resource_id}/pre-publish/config
DeletePrePublishConfig
DeletePrePublishConfig 2 params body → 200
POST
/evals/{resource_type}/{resource_id}/pre-publish/trigger
TriggerPrePublishChecks
TriggerPrePublishChecks 2 params body → 200
GET
/evals/{resource_type}/{resource_id}/pre-publish/checks
ListPrePublishChecksV2
ListPrePublishChecksV2 6 params → 200
POST
/evals/{resource_type}/{resource_id}/observability/eval-scores/timeseries
GetObservabilityEvalScoreTimeseriesForResource
GetObservabilityEvalScoreTimeseriesForResource 2 params body → 200
GET
/evals/{resource_type}/{resource_id}/test-cases/{test_case_id}
GetTestCaseForResource
GetTestCaseForResource 3 params → 200
PUT
/evals/{resource_type}/{resource_id}/test-cases/{test_case_id}
UpdateTestCaseForResource
UpdateTestCaseForResource 3 params body → 200
DELETE
/evals/{resource_type}/{resource_id}/test-cases/{test_case_id}
DeleteTestCaseForResource
DeleteTestCaseForResource 3 params body → 200
POST
/evals/{resource_type}/{resource_id}/test-cases/copy
CopyTestCaseForResource
CopyTestCaseForResource 2 params body → 200
POST
/evals/{resource_type}/{resource_id}/test-cases/move
MoveTestCaseForResource
MoveTestCaseForResource 2 params body → 200
POST
/evals/{resource_type}/{resource_id}/test-sets
CreateTestSetForResource
CreateTestSetForResource 2 params body → 200
GET
/evals/{resource_type}/{resource_id}/test-sets
ListTestSetsForResource
ListTestSetsForResource 5 params → 200
GET
/evals/{resource_type}/{resource_id}/test-sets/{test_set_id}
GetTestSetForResource
GetTestSetForResource 3 params → 200
PUT
/evals/{resource_type}/{resource_id}/test-sets/{test_set_id}
UpdateTestSetForResource
UpdateTestSetForResource 3 params body → 200
DELETE
/evals/{resource_type}/{resource_id}/test-sets/{test_set_id}
DeleteTestSetForResource
DeleteTestSetForResource 4 params body → 200

Schemas 63

The contract defines 63 schemas that model the data the API accepts and returns. The most detailed are GetEvalBatchSummaryForResourceOutput (13 properties), CopyTestCaseForResourceOutput (13 properties), GetTestCaseForResourceOutput (13 properties), EvaluateResourceInput (10 properties). Each schema is shown below with its type and property counts.

DeleteEvalBatchForResourceOutput
object
2 properties 2 required
GetEvalBatchSummaryForResourceOutput
object
13 properties 7 required
DeleteTestCaseForResourceOutput
object
DeleteEvalBatchForResourceInput
UpsertPrePublishConfigInput
object
1 property 1 required
MoveTestCaseForResourceInput
object
3 properties 3 required
CopyTestCaseForResourceInput
object
2 properties 2 required
ListObservabilityConfigsOutput
object
1 property 1 required
UpdateTestSetForResourceInput
object
2 properties 1 required
ListTestCasesOutput
object
2 properties 2 required
UpsertPrePublishConfigOutput
object
1 property 1 required
UpdateTestCaseForResourceOutput
object
CreateObservabilityConfigV2Input
object
6 properties 1 required
DeleteTestSetForResourceOutput
object
CancelEvalBatchForResourceInput
DeleteTestCaseForResourceInput
CancelEvalRunForResourceInput
UpdateObservabilityConfigInput
object
6 properties 1 required
CancelEvalBatchForResourceOutput
object
3 properties 3 required
CreateTestCaseInput
object
4 properties 3 required
CreateTestSetForResourceInput
object
3 properties 1 required
ListEvalRulesForResourceOutput
object
1 property 1 required
UpdateEvalRuleForResourceInput
object
1 property 1 required
ListPrePublishChecksV2Output
object
2 properties 2 required
UpdateObservabilityConfigV2Output
object
1 property 1 required
UpdateObservabilityConfigV2Input
object
6 properties 1 required
DeleteObservabilityConfigV2Output
object
1 property 1 required
DeleteObservabilityConfigOutput
object
1 property 1 required
CopyTestCaseForResourceOutput
object
13 properties 9 required
DeleteEvalRuleForResourceInput
TriggerPrePublishChecksInput
object
1 property 1 required
UpdateObservabilityConfigOutput
object
2 properties 1 required
UpdateEvalRuleForResourceOutput
object
1 property 1 required
GetTestCaseForResourceOutput
object
13 properties 9 required
EvaluateResourceOutput
object
2 properties 2 required
ListEvalRunsForResourceInput
object
5 properties
UpdateTestCaseForResourceInput
object
4 properties 3 required
UpdateTestSetForResourceOutput
object
CancelEvalRunForResourceOutput
object
2 properties 2 required
ListEvalRunsForResourceOutput
object
4 properties 4 required
CreateEvalRuleForResourceOutput
object
1 property 1 required
CreateTestSetForResourceOutput
object
1 property 1 required
ListTestSetsForResourceOutput
object
2 properties 2 required
MoveTestCaseForResourceOutput
object
GetObservabilityEvalScoreTimeseriesForResourceOutput
object
5 properties 5 required
DeleteTestSetForResourceInput
GetObservabilityConfigOutput
object
8 properties 7 required
CreateTestCaseOutput
object
1 property 1 required
DeletePrePublishConfigInput
DeleteObservabilityConfigInput
GetObservabilityEvalScoreTimeseriesForResourceInput
object
5 properties 3 required
CreateEvalRuleForResourceInput
object
1 property 1 required
DeletePrePublishConfigOutput
object
1 property 1 required
TriggerPrePublishChecksOutput
object
1 property 1 required
DeleteEvalRuleForResourceOutput
object
2 properties 2 required
GetTestSetForResourceOutput
object
7 properties 4 required
ListEvalBatchByResourceNewOutput
object
2 properties 2 required
EvaluateResourceInput
object
10 properties 1 required
GetPrePublishConfigOutput
object
1 property
DeleteObservabilityConfigV2Input
CreateObservabilityConfigV2Output
object
1 property 1 required
CreateObservabilityConfigInput
object
6 properties 1 required
CreateObservabilityConfigOutput
object
2 properties 1 required

Specification

The full machine-readable OpenAPI contract behind this narrative.

Source

relevance-ai-evals-api-openapi.yml Raw ↑

Other APIs Relevance AI publishes across the network.

Relevance AI MCP Server
Relevance AI Agent Debug Configs API
Relevance AI Agent Runtime Events API
Relevance AI Agents API
Relevance AI Alarms API
Relevance AI Analytics API
Relevance AI Apify API
Relevance AI Auth API
Relevance AI Billing API
Relevance AI Branding Kit API
Relevance AI Branding Kits API
Relevance AI Chat API
Where this information came from

This is an independent, third-party profile of Relevance AI Evals API, published by API Evangelist. We do not operate, host, resell, or support these APIs, and we are not affiliated with or endorsed by the company unless stated above. Everything here is built from publicly available information — the company's own site, developer portal, documentation, public repositories, and the specifications it publishes for public use. Nothing is obtained by breaching a system, defeating an access control, or using credentials.

The Kin Score and Agent Readiness rating are independently calculated assessments of a company's public API artifacts, scored against a published rubric. They are not certifications, endorsements, security assessments, or audits.

Corrections, re-scores, and removal are free — no partnership or purchase required, and you do not need to justify the request. A removed company is recorded as unrated, never scored zero for having asked. Acknowledgement within one business day; removal within two.

info@apievangelist.com · Read the full data-sourcing policy →
On a security or compliance team? Put security in the subject line and you will get a person, not a form — we will tell you exactly which public URLs this profile was built from.