Need help with your APIs? I offer API discovery, governance & evangelism services. Explore services →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

Helicone Evaluator API

The Evaluator API from Helicone — 10 operation(s) for evaluator.

Helicone Evaluator API is one of 33 APIs that Helicone publishes on the APIs.io network, described by a machine-readable OpenAPI specification.

Tagged areas include Evaluator. The published artifact set on APIs.io includes an OpenAPI specification, API documentation, a getting-started guide, pricing, an API reference, and an engineering blog.

This API exposes 13 operations across 10 paths, and defines 35 schemas. It is described by OpenAPI 3.0.0, at version 1.0.0.

Requests are made against 2 base URLs: https://api.helicone.ai/, http://localhost:8585/.

13 operations 10 paths 35 schemas 2 DELETE4 GET6 POST1 PUT

Metadata

The identity and technical contract details declared by the specification.

Specification
OpenAPI 3.0.0
API Version
1.0.0
Base URL
https://ai-gateway.helicone.ai
Authentication
API Key
License
MIT
Resource Areas
1

Authentication & Security 1

Helicone Evaluator API declares 1 security scheme for authenticating requests. An API key is passed in the header as Authorization (api_key).

  • api_key — Bearer token authentication. Format: 'Bearer YOURAPIKEY'

Paths & Operations 13

Across 10 paths, the API surfaces 13 operations — 2 DELETE, 4 GET, 6 POST, 1 PUT. Each is listed below with its method, path, parameters, and response codes.

Evaluator 13
POST
/v1/evaluator
CreateEvaluator body → 200
GET
/v1/evaluator/{evaluatorId}
GetEvaluator 1 param → 200
PUT
/v1/evaluator/{evaluatorId}
UpdateEvaluator 1 param body → 200
DELETE
/v1/evaluator/{evaluatorId}
DeleteEvaluator 1 param → 200
POST
/v1/evaluator/query
QueryEvaluators body → 200
GET
/v1/evaluator/{evaluatorId}/experiments
GetExperimentsForEvaluator 1 param → 200
GET
/v1/evaluator/{evaluatorId}/onlineEvaluators
GetOnlineEvaluators 1 param → 200
POST
/v1/evaluator/{evaluatorId}/onlineEvaluators
CreateOnlineEvaluator 1 param body → 200
DELETE
/v1/evaluator/{evaluatorId}/onlineEvaluators/{onlineEvaluatorId}
DeleteOnlineEvaluator 2 params → 200
POST
/v1/evaluator/python/test
TestPythonEvaluator body → 200
POST
/v1/evaluator/llm/test
TestLLMEvaluator body → 200
POST
/v1/evaluator/lastmile/test
TestLastMileEvaluator body → 200
GET
/v1/evaluator/{evaluatorId}/stats
GetEvaluatorStats 1 param → 200

Schemas 35

The contract defines 35 schemas that model the data the API accepts and returns. The most detailed are EvaluatorResult (9 properties), CreateEvaluatorParams (5 properties), EvaluatorStats (5 properties), UpdateEvaluatorParams (5 properties). Each schema is shown below with its type and property counts.

ResultSuccess__output-string--traces-string-Array--statusCode_63_-number__
object
2 properties 2 required
Result_EvaluatorResult.string_
ResultSuccess_EvaluatorResult_
object
2 properties 2 required
UpdateEvaluatorParams
object
5 properties
LastMileConfigForm
ResultSuccess__score-number--input-string--output-string--ground_truth_63_-string__
object
2 properties 2 required
Record_string.any_
object
Construct a type with a set of properties K of type T
ResultSuccess_EvaluatorExperiment-Array_
object
2 properties 2 required
ResultSuccess_OnlineEvaluatorByEvaluatorId-Array_
object
2 properties 2 required
EvaluatorScore
object
1 property 1 required
Result_null.string_
OnlineEvaluatorByEvaluatorId
object
2 properties 2 required
ResultSuccess_EvaluatorScore_
object
2 properties 2 required
CreateOnlineEvaluatorParams
object
1 property 1 required
ResultSuccess_EvaluatorStats_
object
2 properties 2 required
ResultSuccess_null_
object
2 properties 2 required
EvaluatorExperiment
object
3 properties 3 required
EvaluatorScoreResult
TestInput
object
4 properties 3 required
EvaluatorConfig
object
3 properties 1 required
Result_EvaluatorExperiment-Array.string_
Record_string.string_
object
Construct a type with a set of properties K of type T
Result_EvaluatorScore.string_
BaseLastMileConfigForm
object
3 properties 3 required
Result__score-number--input-string--output-string--ground_truth_63_-string_.string_
Result_OnlineEvaluatorByEvaluatorId-Array.string_
Result__output-string--traces-string-Array--statusCode_63_-number_.string_
Result_EvaluatorResult-Array.string_
EvaluatorResult
object
9 properties 9 required
Result_EvaluatorStats.string_
CreateEvaluatorParams
object
5 properties 2 required
DataEntry
EvaluatorStats
object
5 properties 5 required
ResultError_string_
object
2 properties 2 required
ResultSuccess_EvaluatorResult-Array_
object
2 properties 2 required

Specification

The full machine-readable OpenAPI contract behind this narrative.

Source

helicone-evaluator-api-openapi.yml Raw ↑

Other APIs Helicone publishes across the network.

Helicone Agent API
Helicone API Key API
Helicone Comparison API
Helicone Credits API
Helicone Customer API
Helicone Dashboard API
Helicone Dataset API
Helicone Evals API
Helicone Experiment API
Helicone HeliconeSql API
Helicone Integration API
Helicone Metrics API