How AI is applied across API Evangelist and APIs.io. Read my AI disclosure →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

Need help?

Contact us
NVIDIA Run:ai

NVIDIA Run:ai Distributed Inferences API

Distributed inference enables running inference workloads across multiple pods, typically to scale model serving beyond a single container or node. This approach is useful when a single instance cannot meet resource requirements.NVIDIA Run:ai supports this model using Leader Worker Set (LWS). Each pod plays a specific role, either as a leader or worker, and together they form a coordinated service. NVIDIA Run:ai manages the orchestration and configuration of these pods to ensure efficient and scalable inference execution

Documentation Provider Profile APIs.io OpenAPI GitHub

NVIDIA Run:ai Distributed Inferences API is one of 62 APIs that NVIDIA Run:ai publishes on the APIs.io network, described by a machine-readable OpenAPI specification.

Tagged areas include Distributed Inferences. The published artifact set on APIs.io includes an OpenAPI specification, API documentation, and authentication docs.

This API exposes 4 operations across 2 paths, and defines 116 schemas. It is described by OpenAPI 3.2.0, at version latest.

Requests are made against a single base URL, https://app.run.ai.

4 operations 2 paths 116 schemas 1 DELETE1 GET1 PATCH1 POST

Metadata

The identity and technical contract details declared by the specification.

Specification
OpenAPI 3.2.0
API Version
latest
Base URL
https://app.run.ai
Authentication
HTTP Bearer
License
NVIDIA Run:ai
Resource Areas
1

Authentication & Security 1

NVIDIA Run:ai Distributed Inferences API declares 1 security scheme for authenticating requests. It accepts HTTP bearer tokens (JWT) (bearerAuth). By default, every request must be authenticated.

  • bearerAuth — Bearer authentication

Paths & Operations 4

Across 2 paths, the API surfaces 4 operations — 1 DELETE, 1 GET, 1 PATCH, 1 POST. Each is listed below with its method, path, parameters, and response codes.

Distributed Inferences 4

Distributed inference enables running inference workloads across multiple pods, typically to scale model serving beyond a single container or node.

POST
/api/v1/workloads/distributed-inferences
Create a distributed inference. [Experimental]
create_distributed_inference body → 202400401403503
DELETE
/api/v1/workloads/distributed-inferences/{workloadId}
Delete a distributed inference
delete_distributed_inference 1 param → 202401403404500503
GET
/api/v1/workloads/distributed-inferences/{workloadId}
Get a distributed inference data
get_distributed_inference 1 param → 200401403404500503
PATCH
/api/v1/workloads/distributed-inferences/{workloadId}
Update distributed inference spec
update_distributed_inference_spec 1 param body → 202401403404500503

Schemas 116

The contract defines 116 schemas that model the data the API accepts and returns. The most detailed are DistributedInferenceLeaderWorkerSpec1 (18 properties), WorkloadMeta1 (11 properties), EnvironmentVariable (8 properties), Toleration (7 properties). Each schema is shown below with its type and property counts.

EnvironmentVariableSecret
objectnull
Details of the secret and key use to populate the environment variable
2 properties
PriorityClass
stringnull
Specifies the priority class for the workload, which determines its scheduling behavior. Valid values are: very-low, low, medium-low, medium, medium-high, high…
DefaultMode
stringnull
File permission mode in octal string format. This value must be a 4-digit octal number, representing the default file mode when mounting a Secret or ConfigMap…
Tolerations
arraynull
Set of tolerations to apply to the workload.
DistributedInferenceLeaderWorkerSpec1
object
18 properties
GpuRequestType
stringnull
Sets the unit type for GPU resources requests. Stated in terms of portion or memory. Sets the unit type for other GPU request fields. If gpuDevicesRequest 1, o…
SecretItems1
arraynull
Set of secret volumes to use in the workload
ImagePullSecrets
arraynull
A list of references to Kubernetes secrets in the same namespace used for pulling container images.
Category
stringnull
Specify the workload category assigned to the workload. Categories are used to classify and monitor different types of workloads within the NVIDIA Run:ai platf…
LargeShmRequest
booleannull
A large /dev/shm device to mount into a container running the created workload. An shm is a shared file system mounted on RAM.
Image
stringnull
Docker image name. For more information, see [Images](https://kubernetes.io/docs/concepts/containers/images). The image name is mandatory for creating a worklo…
WorkingDir
stringnull
Container's working directory. If not specified, the container runtime default will be used. This may be configured in the container image.
StorageInstanceName
objectnull
1 property
EnvironmentVariable
objectnull
Details of an environment variable which is populated into the container.
8 properties
DistributedInferenceLeaderSpecFields
object
1 property
DepartmentId2
string
The id of the department.
SecretInstance2
objectnull
HttpResponse
object
2 properties 2 required
ProbeHandler
objectnull
The action taken to determine the health of the container. (mandatory)
1 property
DistributedInferenceStartupPolicy
stringnull
Determines when the worker pods should start during workload initialization. - LeaderCreated: Workers start after the leader pod is created. - LeaderReady: Wor…
PvcAddedAttrValues
array
an optional array of key-values pairs that are written as annotations on the created PVC. the allowed attributes are determined according to the storage class…
WorkloadDesiredPhase
string
The desired phase of the workload.
EnvironmentVariablePodFieldReference
objectnull
Details of the field-reference and key use to populate the environment variable
1 property
Probe
objectnull
6 properties
DistributedInferenceWorkerSpecFields
object
1 property
Command
stringnull
A command to the server as the entry point of the container running the workload.
ImagePullSecret
objectnull
A reference to a secret in the same namespace used to pull container images.
3 properties
TolerationOperator
stringnull
A key's relationship to the value. Equal uses key and value. Exists is equivalent to wildcard for value, so that a workload can tolerate all taints of a partic…
CpuMemoryLimit
stringnull
Limitations on the CPU memory to allocate for this workload (1G, 20M, .etc). The system guarantees that this workload will not be able to consume more than thi…
EnvironmentVariableConfigMap
objectnull
Details of the configMap and key use to populate the environment variable
2 properties
NodePools
arraynull
A prioritized list of node pools for the scheduler to run the workload on. The scheduler will always try to use the first node pool before moving to the next o…
DistributedInference
CpuCoreRequest
numbernull
CPU units to allocate for the created workload (0.5, 1, .etc). The workload will receive at least this amount of CPU. Note that the workload will not be schedu…
ConfigMapItems
arraynull
Set of config map volumes to use in the workload
DistributedInferenceStartupPolicyField
objectnull
1 property
PodAffinity
objectnull
Pod affinity scheduling rules (e.g. co-locate this workload in the same node, zone, etc. as some other workloads).
2 properties
PvcFieldsUpdatable
objectnull
1 property
DistributedInferenceServingPort
objectnull
Defines the configuration for the inference serving endpoint. This determines how applications or services can send inference requests to the workload.
PvcInstance
objectnull
GpuMemoryRequest
stringnull
Required if and only if gpuRequestType is memory. States the GPU memory to allocate for the created workload, per GPU device. Note that the workload will not b…
NodeSelectorTerm
objectnull
A null or empty node selector term matches no objects. The requirements of them are ANDed.
1 property
DistributedInferenceSpecSpec
DistributedInferenceServingPortContainerAndProtocol
object
2 properties
CpuMemoryRequest
stringnull
The amount of CPU memory to allocate for this workload (1G, 20M, .etc). The workload will receive at least this amount of memory. Note that the workload will n…
SupplementalGroups
stringnull
Comma separated list of groups that the user running the container belongs to, in addition to the group indicated by runAsGid. Use only when the source uid/gid…
CreateHomeDir
booleannull
When set to true, creates a home directory for the container.
PvcClaimSize
stringnull
Requested size for the PVC. Mandatory when existingPvc is false. Recommended sizes: TB/GB/MB/TIB/GIB/MIB
PvcItems
arraynull
Set of pvc persistent volume claims to use in the workload.
EmptyDirInstance
objectnull
ProbeHandlerScheme
stringnull
Scheme to use for connecting to the host, defaults to HTTP.
PvcVolumeMode
stringnull
Default volume mode for the PVC. Choose between Filesystem (default) or Block.
ExtendedResource
objectnull
Quantity of an extended resource.
3 properties
EnvironmentVariableUserCredential
objectnull
Defines a reference to a user-created credential and a specific key within that credential whose value will populate the environment variable. User credentials…
2 properties
CpuCoreLimit
numbernull
Limitations on the number of CPUs consumed by the workload (0.5, 1, .etc). The system guarantees that this workload will not be able to consume more than this…
DistributedInferenceServingPortAccess
object
5 properties
WorkloadCreationMeta
object
4 properties 3 required
ProjectId
string
The id of the project.
Error
object
3 properties 2 required
SecretFieldsUpdatable
objectnull
2 properties
Probes
objectnull
Probes are used to determine if the container is healthy and ready to accept traffic.
1 property
RunAsNonRoot
booleannull
Force the container to run as a non-root user.
Labels
arraynull
Set of labels to populate into the container running the workload.
DistributedInferenceCommonSpec
objectnull
Label
objectnull
Label details to be populated into the container.
3 properties
DistributedInferenceServingPortProtocol
stringnull
The protocol used to access the port.
DistributedInferenceCreationRequest
PvcFieldsNonUpdatable
objectnull
6 properties
GpuPortionRequest
numbernull
Required if and only if gpuRequestType is portion. States the portion of the GPU to allocate for the created workload, per GPU device, between 0 and 1. The def…
Annotations
arraynull
Set of annotations to populate into the container running the workload.
Args
stringnull
Arguments to the command that the container running the workload executes.
EnvironmentVariables
arraynull
Set of environment variables to populate into the container running the workload.
RunAsGid
integernull
The group id to run the entrypoint of the container which executes the workspace. Default to the value specified in the environment asset runAsGid field (optio…
NodeType3
stringnull
Nodes (machines), or a group of nodes on which the workload will run. To use this feature, your Administrator will need to label nodes. For more information, s…
ConfigMapInstance
objectnull
SecretFieldsNonUpdatable
objectnull
1 property
Capability
string
UpdateSpec
object
The specifications of the inference to be updated.
1 property
ReadOnlyRootFileSystem
booleannull
If true, mounts the container's root filesystem as read-only.
ExtendedResources
arraynull
Extended resources and their quantity.
WorkloadId2
string
A unique ID of the workload.
PodAffinityType
stringnull
The affinity type, required or preferred. (mandatory)
UpdateRequest
UidGidSource
stringnull
Indicate the way to determine the user and group ids of the container. The options are a. fromTheImage - user and group ids are determined by the docker image…
GpuDevicesRequest
integernull
Requested number of GPU devices. Currently if more than one device is requested, it is not possible to provide values for gpuMemory or gpuPortion.
MatchExpression
objectnull
A selector that contains values, a key, and an operator that relates the key and values.
3 properties 2 required
GpuPortionLimit
numbernull
Limitations on the portion consumed by the workload, per GPU device. The system guarantees The gpuPotionLimit must be no less than the gpuPortionRequest.
Annotation
objectnull
Annotation details to be populated into the container.
3 properties
ImagePullPolicy
stringnull
Image pull policy. Defaults to Always if :latest tag is specified, otherwise it is IfNotPresent.
DistributedInferenceServingPortAccessAuthorizationTypeEnum
stringnull
Specifies who can send inference requests to the serving endpoint: Possible values: - public: No authorization is required. (Default) - authenticatedUsers: Any…
Toleration
objectnull
Toleration details.
7 properties
DistributedInferenceSpec
object
The specifications of the distributed inference to be created.
1 property
Secret5
PvcAddedAttrValue
object
2 properties 1 required
EmptyDirItems
arraynull
A list of emptyDir volumes to mount in the workload.
Capabilities
arraynull
Add POSIX capabilities to running containers. Defaults to the default set of capabilities granted by the container runtime.
Preemptibility
stringnull
Specifies whether the workload can be preempted by higher-priority workloads. Valid values are preemptible and non-preemptible. If explicitly set, this value t…
DistributedInferenceReplicasField
object
1 property
NodeAffinityRequired
objectnull
If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements…
1 property
RunAsUid
integernull
The user id to run the entrypoint of the container which executes the workspace. Default to the value specified in the environment asset runAsUid field (option…
ExcludeField
objectnull
1 property
WorkloadMeta1
object
11 properties 8 required
Pvc
ClusterId
string
The id of the cluster.
EmptyDir
objectnull
3 properties
ClaimInfo
objectnull
Claim information for the newly created PVC. The information should not be provided when attempting to use existing PVC.
5 properties
SubmitWithTemplateId
object
1 property
TolerationEffect
stringnull
The taint effect to match. (mandatory)
Phase
string
PvcAccessModes
objectnull
Default access mode(s) applied to newly created PVCs unless explicitly overridden.
3 properties
ConfigMap
objectnull
4 properties
DistributedInferenceRestartPolicy
stringnull
Determines the behavior when a pod fails. - RecreateGroupOnPodRestart: Restarts all pods in the group if any pod fails. - None: No automatic restart behavior i…
SeccompProfileType
stringnull
Indicates which kind of seccomp profile will be applied to the container. The options are a. RuntimeDefault - the container runtime default profile should be u…
MatchExpressionOperator
string
Represents a key's relationship to a set of values (mandatory).
WorkloadName
string
The name of the workload.
GpuMemoryLimit
stringnull
Limitation on the memory consumed by the workload, per GPU device. The system guarantees The gpuMemoryLimit must be no less than gpuMemoryRequest.
DistributedInferenceWorkersField
object
1 property

Specification

The full machine-readable OpenAPI contract behind this narrative.

Source

runai-distributed-inferences-api-openapi.yml Raw ↑

Other APIs NVIDIA Run:ai publishes across the network.

NVIDIA Run:ai Access Keys API
AE Profile APIs.io
NVIDIA Run:ai Access rules API
AE Profile APIs.io
NVIDIA Run:ai Administrator Command Line Interface API
AE Profile APIs.io
NVIDIA Run:ai AI Applications API
AE Profile APIs.io
NVIDIA Run:ai Applications API
AE Profile APIs.io
NVIDIA Run:ai Clusters API
AE Profile APIs.io
NVIDIA Run:ai Compute API
AE Profile APIs.io
NVIDIA Run:ai ConfigMap API
AE Profile APIs.io
NVIDIA Run:ai Credentials API
AE Profile APIs.io
NVIDIA Run:ai Departments API
AE Profile APIs.io
NVIDIA Run:ai Distributed API
AE Profile APIs.io
NVIDIA Run:ai Environment API
AE Profile APIs.io
#Distributed Inferences
← Back to N APIs
API Details
ID
runai:runai-distributed-inferences-api
Base URL
https://app.run.ai
Docs
https://run-ai-docs.nvidia.com/api/readme.md
Provider
NVIDIA Run:ai
APIs.io →  GitHub →
Explore more
All APIs Providers APIs.io Network
Work with this as data

Every API here is available over the API and to AI agents over MCP. APIs is not yet its own endpoint on the v1 API. Reach this content through network search and the tag graph, or the MCP server below.

Installs https://mcp.apievangelist.com/mcp in Claude, Cursor, VS Code and the rest — one button, every client.

MCP tools for apis
4 tools reach this content
  • search_api_evangelistSearch every content type across the network at once.
  • find_relatedThe shared-tag relevance graph — what else covers this.
  • get_tagEverything one tag labels, across all content types.
  • guide_topicPRO — a curated bundle for a topic: area, guidance, rules, papers, stories, services.
All tools & prompts
Call it yourself
curl for this page
Search the network for this 
curl "https://api.apievangelist.com/v1/search?q=runai-distributed-inferences-api&limit=10"
Everything under a tag 
curl "https://api.apievangelist.com/v1/tags/runai-distributed-inferences-api"

Discovery needs no key. Full bodies and the synthesis endpoints are Pro.

Get an API key

Free tier, no email required. Sign in with whichever identity you already have.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have. Your account

Where this information came from

This is an independent, third-party profile of NVIDIA Run:ai Distributed Inferences API, published by API Evangelist. We do not operate, host, resell, or support these APIs, and we are not affiliated with or endorsed by the company unless stated above. Everything here is built from publicly available information — the company's own site, developer portal, documentation, public repositories, and the specifications it publishes for public use. Nothing is obtained by breaching a system, defeating an access control, or using credentials.

The Kin Score and Agent Readiness rating are independently calculated assessments of a company's public API artifacts, scored against a published rubric. They are not certifications, endorsements, security assessments, or audits.

Corrections, re-scores, and removal are free — no partnership or purchase required, and you do not need to justify the request. A removed company is recorded as unrated, never scored zero for having asked. Acknowledgement within one business day; removal within two.

info@apievangelist.com · Read the full data-sourcing policy →
On a security or compliance team? Put security in the subject line and you will get a person, not a form — we will tell you exactly which public URLs this profile was built from.