How AI is applied across API Evangelist and APIs.io. Read my AI disclosure →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

Need help?

Contact us
NVIDIA Run:ai

NVIDIA Run:ai Distributed Inferences API

Distributed inference enables running inference workloads across multiple pods, typically to scale model serving beyond a single container or node. This approach is useful when a single instance cannot meet resource requirements.NVIDIA Run:ai supports this model using Leader Worker Set (LWS). Each pod plays a specific role, either as a leader or worker, and together they form a coordinated service. NVIDIA Run:ai manages the orchestration and configuration of these pods to ensure efficient and scalable inference execution

Documentation Provider Profile APIs.io OpenAPI GitHub

NVIDIA Run:ai Distributed Inferences API is one of 62 APIs that NVIDIA Run:ai publishes on the APIs.io network, described by a machine-readable OpenAPI specification.

Tagged areas include Distributed Inferences. The published artifact set on APIs.io includes an OpenAPI specification, API documentation, and authentication docs.

This API exposes 4 operations across 2 paths, and defines 116 schemas. It is described by OpenAPI 3.2.0, at version latest.

Requests are made against a single base URL, https://app.run.ai.

4 operations 2 paths 116 schemas 1 DELETE1 GET1 PATCH1 POST

Metadata

The identity and technical contract details declared by the specification.

Specification
OpenAPI 3.2.0
API Version
latest
Base URL
https://app.run.ai
Authentication
HTTP Bearer
License
NVIDIA Run:ai
Resource Areas
1

Authentication & Security 1

NVIDIA Run:ai Distributed Inferences API declares 1 security scheme for authenticating requests. It accepts HTTP bearer tokens (JWT) (bearerAuth). By default, every request must be authenticated.

  • bearerAuth — Bearer authentication

Paths & Operations 4

Across 2 paths, the API surfaces 4 operations — 1 DELETE, 1 GET, 1 PATCH, 1 POST. Each is listed below with its method, path, parameters, and response codes.

Distributed Inferences 4

Distributed inference enables running inference workloads across multiple pods, typically to scale model serving beyond a single container or node. This approach is useful when a…

POST
/api/v1/workloads/distributed-inferences
Create a distributed inference. [Experimental]
create_distributed_inference body → 202400401403503
DELETE
/api/v1/workloads/distributed-inferences/{workloadId}
Delete a distributed inference.
delete_distributed_inference 1 param → 202401403404500503
GET
/api/v1/workloads/distributed-inferences/{workloadId}
Get a distributed inference data.
get_distributed_inference 1 param → 200401403404500503
PATCH
/api/v1/workloads/distributed-inferences/{workloadId}
Update distributed inference spec.
update_distributed_inference_spec 1 param body → 202401403404500503

Schemas 116

The contract defines 116 schemas that model the data the API accepts and returns. The most detailed are DistributedInferenceLeaderWorkerSpec1 (18 properties), WorkloadMeta1 (11 properties), EnvironmentVariable (8 properties), Toleration (7 properties). Each schema is shown below with its type and property counts.

LargeShmRequest
booleannull
A large /dev/shm device to mount into a container running the created workload. An shm is a shared file system mounted on RAM.
SecretFieldsUpdatable
objectnull
2 properties
EnvironmentVariableConfigMap
objectnull
Details of the configMap and key use to populate the environment variable
2 properties
ConfigMapInstance
objectnull
WorkloadId2
string
A unique ID of the workload.
DefaultMode
stringnull
File permission mode in octal string format. This value must be a 4-digit octal number, representing the default file mode when mounting a Secret or ConfigMap…
CpuMemoryRequest
stringnull
The amount of CPU memory to allocate for this workload (1G, 20M, .etc). The workload will receive at least this amount of memory. Note that the workload will n…
GpuDevicesRequest
integernull
Requested number of GPU devices. Currently if more than one device is requested, it is not possible to provide values for gpuMemory or gpuPortion.
PriorityClass
stringnull
Specifies the priority class for the workload, which determines its scheduling behavior. Valid values are: very-low, low, medium-low, medium, medium-high, high…
Label
objectnull
Label details to be populated into the container.
3 properties
DepartmentId2
string
The id of the department.
DistributedInferenceStartupPolicyField
objectnull
1 property
NodeSelectorTerm
objectnull
A null or empty node selector term matches no objects. The requirements of them are ANDed.
1 property
ImagePullSecrets
arraynull
A list of references to Kubernetes secrets in the same namespace used for pulling container images.
PvcVolumeMode
stringnull
Default volume mode for the PVC. Choose between Filesystem (default) or Block.
TolerationEffect
stringnull
The taint effect to match. (mandatory)
Annotation
objectnull
Annotation details to be populated into the container.
3 properties
NodeType3
stringnull
Nodes (machines), or a group of nodes on which the workload will run. To use this feature, your Administrator will need to label nodes. For more information, s…
PvcClaimSize
stringnull
Requested size for the PVC. Mandatory when existingPvc is false. Recommended sizes: TB/GB/MB/TIB/GIB/MIB
EnvironmentVariablePodFieldReference
objectnull
Details of the field-reference and key use to populate the environment variable
1 property
Labels
arraynull
Set of labels to populate into the container running the workload.
PvcFieldsNonUpdatable
objectnull
6 properties
DistributedInferenceSpecSpec
Preemptibility
stringnull
Specifies whether the workload can be preempted by higher-priority workloads. Valid values are preemptible and non-preemptible. If explicitly set, this value t…
PvcAddedAttrValues
array
an optional array of key-values pairs that are written as annotations on the created PVC. the allowed attributes are determined according to the storage class…
RunAsUid
integernull
The user id to run the entrypoint of the container which executes the workspace. Default to the value specified in the environment asset runAsUid field (option…
StorageInstanceName
objectnull
1 property
WorkloadCreationMeta
object
4 properties 3 required
WorkloadName
string
The name of the workload.
Image
stringnull
Docker image name. For more information, see [Images](https://kubernetes.io/docs/concepts/containers/images). The image name is mandatory for creating a worklo…
SeccompProfileType
stringnull
Indicates which kind of seccomp profile will be applied to the container. The options are a. RuntimeDefault - the container runtime default profile should be u…
EnvironmentVariable
objectnull
Details of an environment variable which is populated into the container.
8 properties
DistributedInferenceStartupPolicy
stringnull
Determines when the worker pods should start during workload initialization. - LeaderCreated: Workers start after the leader pod is created. - LeaderReady: Wor…
PvcAccessModes
objectnull
Default access mode(s) applied to newly created PVCs unless explicitly overridden.
3 properties
EnvironmentVariables
arraynull
Set of environment variables to populate into the container running the workload.
Probe
objectnull
6 properties
PodAffinity
objectnull
Pod affinity scheduling rules (e.g. co-locate this workload in the same node, zone, etc. as some other workloads).
2 properties
CreateHomeDir
booleannull
When set to true, creates a home directory for the container.
UpdateSpec
object
The specifications of the inference to be updated.
1 property
ProjectId
string
The id of the project.
DistributedInferenceReplicasField
object
1 property
Tolerations
arraynull
Set of tolerations to apply to the workload.
DistributedInference
ExtendedResource
objectnull
Quantity of an extended resource.
3 properties
DistributedInferenceServingPort
objectnull
Defines the configuration for the inference serving endpoint. This determines how applications or services can send inference requests to the workload.
DistributedInferenceServingPortProtocol
stringnull
The protocol used to access the port.
PvcItems
arraynull
Set of pvc persistent volume claims to use in the workload.
DistributedInferenceLeaderWorkerSpec1
object
18 properties
Args
stringnull
Arguments to the command that the container running the workload executes.
DistributedInferenceServingPortAccess
object
5 properties
EnvironmentVariableSecret
objectnull
Details of the secret and key use to populate the environment variable
2 properties
Command
stringnull
A command to the server as the entry point of the container running the workload.
Category
stringnull
Specify the workload category assigned to the workload. Categories are used to classify and monitor different types of workloads within the NVIDIA Run:ai platf…
SecretInstance2
objectnull
GpuMemoryRequest
stringnull
Required if and only if gpuRequestType is memory. States the GPU memory to allocate for the created workload, per GPU device. Note that the workload will not b…
DistributedInferenceServingPortContainerAndProtocol
object
2 properties
ProbeHandlerScheme
stringnull
Scheme to use for connecting to the host, defaults to HTTP.
Capability
string
EmptyDir
objectnull
3 properties
UidGidSource
stringnull
Indicate the way to determine the user and group ids of the container. The options are a. fromTheImage - user and group ids are determined by the docker image…
MatchExpression
objectnull
A selector that contains values, a key, and an operator that relates the key and values.
3 properties 2 required
ExtendedResources
arraynull
Extended resources and their quantity.
EmptyDirItems
arraynull
A list of emptyDir volumes to mount in the workload.
NodePools
arraynull
A prioritized list of node pools for the scheduler to run the workload on. The scheduler will always try to use the first node pool before moving to the next o…
RunAsGid
integernull
The group id to run the entrypoint of the container which executes the workspace. Default to the value specified in the environment asset runAsGid field (optio…
SecretItems1
arraynull
Set of secret volumes to use in the workload
MatchExpressionOperator
string
Represents a key's relationship to a set of values (mandatory).
ProbeHandler
objectnull
The action taken to determine the health of the container. (mandatory)
1 property
SecretFieldsNonUpdatable
objectnull
1 property
WorkloadDesiredPhase
string
The desired phase of the workload.
ClaimInfo
objectnull
Claim information for the newly created PVC. The information should not be provided when attempting to use existing PVC.
5 properties
DistributedInferenceLeaderSpecFields
object
1 property
DistributedInferenceCreationRequest
PvcAddedAttrValue
object
2 properties 1 required
GpuRequestType
stringnull
Sets the unit type for GPU resources requests. Stated in terms of portion or memory. Sets the unit type for other GPU request fields. If gpuDevicesRequest 1, o…
TolerationOperator
stringnull
A key's relationship to the value. Equal uses key and value. Exists is equivalent to wildcard for value, so that a workload can tolerate all taints of a partic…
Secret5
DistributedInferenceServingPortAccessAuthorizationTypeEnum
stringnull
Specifies who can send inference requests to the serving endpoint: Possible values: - public: No authorization is required. (Default) - authenticatedUsers: Any…
Pvc
SubmitWithTemplateId
object
1 property
PvcInstance
objectnull
ConfigMapItems
arraynull
Set of config map volumes to use in the workload
DistributedInferenceWorkerSpecFields
object
1 property
HttpResponse
object
2 properties 2 required
EnvironmentVariableUserCredential
objectnull
Defines a reference to a user-created credential and a specific key within that credential whose value will populate the environment variable. User credentials…
2 properties
Annotations
arraynull
Set of annotations to populate into the container running the workload.
ExcludeField
objectnull
1 property
ReadOnlyRootFileSystem
booleannull
If true, mounts the container's root filesystem as read-only.
CpuCoreRequest
numbernull
CPU units to allocate for the created workload (0.5, 1, .etc). The workload will receive at least this amount of CPU. Note that the workload will not be schedu…
ClusterId
string
The id of the cluster.
CpuMemoryLimit
stringnull
Limitations on the CPU memory to allocate for this workload (1G, 20M, .etc). The system guarantees that this workload will not be able to consume more than thi…
DistributedInferenceWorkersField
object
1 property
WorkingDir
stringnull
Container's working directory. If not specified, the container runtime default will be used. This may be configured in the container image.
ImagePullPolicy
stringnull
Image pull policy. Defaults to Always if :latest tag is specified, otherwise it is IfNotPresent.
NodeAffinityRequired
objectnull
If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements…
1 property
DistributedInferenceCommonSpec
objectnull
RunAsNonRoot
booleannull
Force the container to run as a non-root user.
Toleration
objectnull
Toleration details.
7 properties
EmptyDirInstance
objectnull
GpuPortionLimit
numbernull
Limitations on the portion consumed by the workload, per GPU device. The system guarantees The gpuPotionLimit must be no less than the gpuPortionRequest.
Capabilities
arraynull
Add POSIX capabilities to running containers. Defaults to the default set of capabilities granted by the container runtime.
Probes
objectnull
Probes are used to determine if the container is healthy and ready to accept traffic.
1 property
UpdateRequest
Phase
string
PodAffinityType
stringnull
The affinity type, required or preferred. (mandatory)
CpuCoreLimit
numbernull
Limitations on the number of CPUs consumed by the workload (0.5, 1, .etc). The system guarantees that this workload will not be able to consume more than this…
WorkloadMeta1
object
11 properties 8 required
ImagePullSecret
objectnull
A reference to a secret in the same namespace used to pull container images.
3 properties
PvcFieldsUpdatable
objectnull
1 property
DistributedInferenceSpec
object
The specifications of the distributed inference to be created.
1 property
GpuPortionRequest
numbernull
Required if and only if gpuRequestType is portion. States the portion of the GPU to allocate for the created workload, per GPU device, between 0 and 1. The def…
ConfigMap
objectnull
4 properties
DistributedInferenceRestartPolicy
stringnull
Determines the behavior when a pod fails. - RecreateGroupOnPodRestart: Restarts all pods in the group if any pod fails. - None: No automatic restart behavior i…
GpuMemoryLimit
stringnull
Limitation on the memory consumed by the workload, per GPU device. The system guarantees The gpuMemoryLimit must be no less than gpuMemoryRequest.
SupplementalGroups
stringnull
Comma separated list of groups that the user running the container belongs to, in addition to the group indicated by runAsGid. Use only when the source uid/gid…
Error
object
3 properties 2 required

Specification

The full machine-readable OpenAPI contract behind this narrative.

Source

runai-distributed-inferences-api-openapi.yml Raw ↑

Other APIs NVIDIA Run:ai publishes across the network.

NVIDIA Run:ai Access Keys API
AE Profile APIs.io
NVIDIA Run:ai Access rules API
AE Profile APIs.io
NVIDIA Run:ai Administrator Command Line Interface API
AE Profile APIs.io
NVIDIA Run:ai AI Applications API
AE Profile APIs.io
NVIDIA Run:ai Applications API
AE Profile APIs.io
NVIDIA Run:ai AuditLogs API
AE Profile APIs.io
NVIDIA Run:ai Clusters API
AE Profile APIs.io
NVIDIA Run:ai Compute API
AE Profile APIs.io
NVIDIA Run:ai ConfigMap API
AE Profile APIs.io
NVIDIA Run:ai Credentials API
AE Profile APIs.io
NVIDIA Run:ai Datasources API
AE Profile APIs.io
NVIDIA Run:ai Datavolumes API
AE Profile APIs.io
#Distributed Inferences
← Back to N APIs
API Details
ID
runai:runai-distributed-inferences-api
Base URL
https://app.run.ai
Docs
https://run-ai-docs.nvidia.com/api/readme.md
Provider
NVIDIA Run:ai
APIs.io →  GitHub →
Explore more
All APIs Providers APIs.io Network
Work with this as data

Every API here is available over the API and to AI agents over MCP. APIs is not yet its own endpoint on the v1 API. Reach this content through network search and the tag graph, or the MCP server below.

Installs https://mcp.apievangelist.com/mcp in Claude, Cursor, VS Code and the rest — one button, every client.

MCP tools for apis
4 tools reach this content
  • search_api_evangelistSearch every content type across the network at once.
  • find_relatedThe shared-tag relevance graph — what else covers this.
  • get_tagEverything one tag labels, across all content types.
  • guide_topicPRO — a curated bundle for a topic: area, guidance, rules, papers, stories, services.
All tools & prompts
Call it yourself
curl for this page
Search the network for this 
curl "https://api.apievangelist.com/v1/search?q=runai-distributed-inferences-api&limit=10"
Everything under a tag 
curl "https://api.apievangelist.com/v1/tags/runai-distributed-inferences-api"

Discovery needs no key. Full bodies and the synthesis endpoints are Pro.

Get an API key

Free tier, no email required. Sign in with whichever identity you already have.

GitHub Google LinkedIn

A second provider on the same verified email joins the account you already have. Your account

Where this information came from

This is an independent, third-party profile of NVIDIA Run:ai Distributed Inferences API, published by API Evangelist. We do not operate, host, resell, or support these APIs, and we are not affiliated with or endorsed by the company unless stated above. Everything here is built from publicly available information — the company's own site, developer portal, documentation, public repositories, and the specifications it publishes for public use. Nothing is obtained by breaching a system, defeating an access control, or using credentials.

The Kin Score and Agent Readiness rating are independently calculated assessments of a company's public API artifacts, scored against a published rubric. They are not certifications, endorsements, security assessments, or audits.

Corrections, re-scores, and removal are free — no partnership or purchase required, and you do not need to justify the request. A removed company is recorded as unrated, never scored zero for having asked. Acknowledgement within one business day; removal within two.

info@apievangelist.com · Read the full data-sourcing policy →
On a security or compliance team? Put security in the subject line and you will get a person, not a form — we will tell you exactly which public URLs this profile was built from.