Cohere Tokenize API
The Cohere Tokenize API splits input text into tokens using the tokenizer associated with a specified model. It returns both the token strings and their corresponding token IDs. This is useful for understanding how text will be processed by Cohere models, estimating token counts for billing purposes, and debugging input formatting issues.
Cohere Tokenize API is one of 9 APIs that cohere publishes on the APIs.io network, described by a machine-readable OpenAPI specification.
Tagged areas include Artificial Intelligence, Natural Language Processing, Text Processing, and Tokenization. The published artifact set on APIs.io includes API documentation and an OpenAPI specification.
This API exposes 1 operation across 1 path, and defines 3 schemas. It is described by OpenAPI 3.1.0, at version 2.0.
Requests are made against a single base URL, https://api.cohere.com.
Metadata
The identity and technical contract details declared by the specification.
Authentication & Security 1
Cohere Tokenize API declares
1 security scheme
for authenticating requests.
It accepts HTTP bearer tokens (bearerAuth).
By default, every request must be authenticated.
bearerAuth— Bearer authentication using a Cohere API key. Pass the API key in the Authorization header as Bearer .
Paths & Operations 1
Across 1 path, the API surfaces 1 operation — 1 POST. Each is listed below with its method, path, parameters, and response codes.
Endpoints for splitting text into tokens using byte-pair encoding for a specified Cohere model.
Schemas 3
The contract defines 3 schemas that model the data the API accepts and returns. The most detailed are TokenizeResponse (3 properties), TokenizeRequest (2 properties), Error (1 property). Each schema is shown below with its type and property counts.
Specification
The full machine-readable OpenAPI contract behind this narrative.
Source
More from cohere 8
Other APIs cohere publishes across the network.