NVIDIA Run:ai NVIDIA NIM API
The NVIDIA NIM API provides endpoints to create and manage workloads that deploy NVIDIA Inference Microservices (NIM) through the NIM Operator. These workloads package optimized NVIDIA model servers and run as managed services on the NVIDIA Run:ai platform.Each request includes NVIDIA Run:ai scheduling metadata (for example, project, priority, and category) and a NIM service specification that defines the container image, compute resources, environment variables, storage, and networking configuration. Once submitted, NVIDIA Run:ai handles scheduling, orchestration, and lifecycle management of the NIM service to ensure reliable and efficient model serving.
NVIDIA Run:ai NVIDIA NIM API is one of 62 APIs that NVIDIA Run:ai publishes on the APIs.io network, described by a machine-readable OpenAPI specification.
Tagged areas include NVIDIA NIM. The published artifact set on APIs.io includes an OpenAPI specification, API documentation, and authentication docs.
This API exposes 3 operations across 2 paths, and defines 85 schemas. It is described by OpenAPI 3.0.3, at version latest.
Requests are made against a single base URL, https://app.run.ai.
3 operations
2 paths
85 schemas
1 GET1 PATCH1 POST
Metadata
The identity and technical contract details declared by the specification.
Authentication & Security 1
NVIDIA Run:ai NVIDIA NIM API declares
1 security scheme
for authenticating requests.
It accepts HTTP bearer tokens (JWT) (bearerAuth).
By default, every request must be authenticated.
bearerAuth— Bearer authentication
Paths & Operations 3
Across 2 paths, the API surfaces 3 operations — 1 GET, 1 PATCH, 1 POST. Each is listed below with its method, path, parameters, and response codes.
NVIDIA NIM 3
The NVIDIA NIM API provides endpoints to create and manage workloads that deploy NVIDIA Inference Microservices (NIM) through the NIM Operator. These workloads package optimized N…
POST
/api/v2/workloads/nim-services
Create a NVIDIA NIM service. [Experimental]
GET
/api/v2/workloads/nim-services/{WorkloadV2Id}
Get a NVIDIA NIM service. [Experimental]
PATCH
/api/v2/workloads/nim-services/{WorkloadV2Id}
Update NVIDIA NIM service spec. [Experimental]
Schemas 85
The contract defines 85 schemas that model the data the API accepts and returns. The most detailed are WorkloadV2MetadataAutoFill (13 properties), EnvironmentVariable (8 properties), NimServiceServingPort (7 properties), Toleration (7 properties). Each schema is shown below with its type and property counts.
Category
string
Specify the workload category assigned to the workload. Categories are used to classify and monitor different types of workloads within the NVIDIA Run:ai platf…
Probe
object
6 properties
Labels
array
Set of labels to populate into the container running the workload.
Error
object
3 properties
2 required
DepartmentName1
string
The name of the department
CpuMemoryLimit
string
Limitations on the CPU memory to allocate for this workload (1G, 20M, .etc). The system guarantees that this workload will not be able to consume more than thi…
PvcClaimSize
string
Requested size for the PVC. Mandatory when existingPvc is false. Recommended sizes: TB/GB/MB/TIB/GIB/MIB
PvcAddedAttrValues
array
an optional array of key-values pairs that are written as annotations on the created PVC. the allowed attributes are determined according to the storage class…
Toleration
object
Toleration details.
7 properties
ClusterId
string
The id of the cluster.
GpuMemoryRequest
string
Required if and only if gpuRequestType is memory. States the GPU memory to allocate for the created workload, per GPU device. Note that the workload will not b…
NimServiceUpdateRequest
object
1 property
1 required
NimServicePvcFields
object
4 properties
ServingPortServiceType
string
The type of Kubernetes service to create for the inference deployment. Options include 'ClusterIP' (default), 'NodePort', 'LoadBalancer', and 'ExternalName'.
EnvironmentVariable
object
Details of an environment variable which is populated into the container.
8 properties
RunAsGid
integer
The group id to run the entrypoint of the container which executes the workspace. Default to the value specified in the environment asset runAsGid field (optio…
EnvironmentVariableSecret
object
Details of the secret and key use to populate the environment variable
2 properties
WorkloadV2MetadataResponse
object
5 properties
2 required
ServingPortExposeExternally
boolean
Indicates whether the inference serving endpoint should be accessible outside the cluster. If set to true, the endpoint will be exposed externally. To enable e…
Annotations
array
Set of annotations to populate into the container running the workload.
ProbeHandler
object
The action taken to determine the health of the container. (mandatory)
1 property
WorkloadId3
string
A unique ID of the workload.
TolerationEffect
string
The taint effect to match. (mandatory)
ImagePullPolicy
string
Image pull policy. Defaults to Always if :latest tag is specified, otherwise it is IfNotPresent.
WorkloadName
string
The name of the workload.
ClaimInfo
object
Claim information for the newly created PVC. The information should not be provided when attempting to use existing PVC.
5 properties
AutoScalingMinReplicas
integer
The minimum number of replicas for autoscaling. Defaults to 1.
AutoScalingMetricNim
string
The metric to use for autoscaling (mandatory).
WorkloadV2Metadata
EnvironmentVariables
array
Set of environment variables to populate into the container running the workload.
CpuCoreRequest
number
CPU units to allocate for the created workload (0.5, 1, .etc). The workload will receive at least this amount of CPU. Note that the workload will not be schedu…
Preemptibility
string
Specifies whether the workload can be preempted by higher-priority workloads. Valid values are preemptible and non-preemptible. If explicitly set, this value t…
TenantId
integer
The id of the tenant.
PolicyRuleEnum
string
Indicates which validation rule (e.g., min, max, step, options, required, canEdit, canAdd) conflicted with policy restrictions, causing the asset or template t…
ImagePullSecrets
array
A list of references to Kubernetes secrets in the same namespace used for pulling container images.
Image
string
Docker image name. For more information, see [Images](https://kubernetes.io/docs/concepts/containers/images). The image name is mandatory for creating a worklo…
NimServiceResponse
object
3 properties
3 required
EnvironmentVariableUserCredential
object
Defines a reference to a user-created credential and a specific key within that credential whose value will populate the environment variable. User credentials…
2 properties
ServingPortGrpcPort
integer
The GRPC port that the container running the inference service exposes.
NimServiceWorkers
integer
Specifies the number of worker nodes to use when running the NIM service in multi-node.
Annotation
object
Annotation details to be populated into the container.
3 properties
WorkloadV2MetadataAutoFill
object
13 properties
11 required
GpuMemoryLimit
string
Limitation on the memory consumed by the workload, per GPU device. The system guarantees The gpuMemoryLimit must be no less than gpuMemoryRequest.
EnvironmentVariableConfigMap
object
Details of the configMap and key use to populate the environment variable
2 properties
DesiredPhase
string
The desired phase of the workload.
CpuCoreLimit
number
Limitations on the number of CPUs consumed by the workload (0.5, 1, .etc). The system guarantees that this workload will not be able to consume more than this…
GpuDevicesRequest
integer
Requested number of GPU devices. Currently if more than one device is requested, it is not possible to provide values for gpuMemory or gpuPortion.
NodePools
array
A prioritized list of node pools for the scheduler to run the workload on. The scheduler will always try to use the first node pool before moving to the next o…
ServingPortExposedUrl
string
The custom URL to use for the serving port. If empty (default), an autogenerated URL will be used.
PvcAddedAttrValue
object
2 properties
1 required
EnvironmentVariablePodFieldReference
object
Details of the field-reference and key use to populate the environment variable
1 property
ProjectName2
string
The name of the project
AutoScalingMaxReplicas
integer
The maximum number of replicas for autoscaling. Defaults to minReplicas. Must be no less than minReplicas.
GpuRequestType
string
Sets the unit type for GPU resources requests. Stated in terms of portion or memory. Sets the unit type for other GPU request fields. If gpuDevicesRequest 1, o…
NimServiceCreateRequest
object
2 properties
2 required
NimServiceMultiNode
object
Defines whether the NIM service runs as a multi-node deployment. If workers is set to 1 or more, the service runs in multi-node.
1 property
ProbeHandlerScheme
string
Scheme to use for connecting to the host, defaults to HTTP.
NimServiceReplicas
integer
The number of replicas to deploy.
ServingPortMetricsPort
integer
The port where metrics are exposed, required only if it's different than the main port.
AutoScalingScaleWindowSeconds
integer
The time window for autoscaling decisions, in seconds. Defaults to 300 seconds.
ComplianceIssuesV2
object
1 property
PriorityClass
string
Specifies the priority class for the workload, which determines its scheduling behavior. Valid values are: very-low, low, medium-low, medium, medium-high, high…
ServingPortExposedProtocol
string
The protocol to use for the exposed URL. If grpcPort is set, this defaults to grpc. Otherwise, it defaults to http.
SubmissionErrorV2
PvcVolumeMode
string
Default volume mode for the PVC. Choose between Filesystem (default) or Block.
Label
object
Label details to be populated into the container.
3 properties
Tolerations
array
Set of tolerations to apply to the workload.
ServingPortPort
integer
The port that the container running the inference service exposes (mandatory).
ProjectId
string
The id of the project.
NIMServiceMetadataCreateParams
object
3 properties
2 required
CpuMemoryRequest
string
The amount of CPU memory to allocate for this workload (1G, 20M, .etc). The workload will receive at least this amount of memory. Note that the workload will n…
GpuPortionRequest
number
Required if and only if gpuRequestType is portion. States the portion of the GPU to allocate for the created workload, per GPU device, between 0 and 1. The def…
NimCache
object
The specification of a NIM cache volume.
2 properties
GpuPortionLimit
number
Limitations on the portion consumed by the workload, per GPU device. The system guarantees The gpuPotionLimit must be no less than the gpuPortionRequest.
NimServiceSpec
NimServiceServingPort
object
A port for accessing the inference service
7 properties
NimServiceNgcAuthSecret
string
The name of a Kubernetes secret containing the NGC access credentials. The secret must contain a key named NGCAPIKEY with the API key as the value.
Probes
object
Probes are used to determine if the container is healthy and ready to accept traffic.
1 property
AutoScalingMetricThreshold
integer
The threshold to use with the specified metric for autoscaling (mandatory).
GVK
object
Specifies the Group, Version, and Kind (GVK) of the Kubernetes resource that defines the workload.
3 properties
3 required
ImagePullSecret
object
A reference to a secret in the same namespace used to pull container images.
3 properties
DepartmentId3
string
The id of the department.
TolerationOperator
string
A key's relationship to the value. Equal uses key and value. Exists is equivalent to wildcard for value, so that a workload can tolerate all taints of a partic…
RunAsUid
integer
The user id to run the entrypoint of the container which executes the workspace. Default to the value specified in the environment asset runAsUid field (option…
PvcAccessModes
object
Default access mode(s) applied to newly created PVCs unless explicitly overridden.
3 properties
Specification
The full machine-readable OpenAPI contract behind this narrative.
Source
More from NVIDIA Run:ai 12
Other APIs NVIDIA Run:ai publishes across the network.
NVIDIA Run:ai Access Keys API
NVIDIA Run:ai Access rules API
NVIDIA Run:ai Administrator Command Line Interface API
NVIDIA Run:ai AI Applications API
NVIDIA Run:ai Applications API
NVIDIA Run:ai AuditLogs API
NVIDIA Run:ai Clusters API
NVIDIA Run:ai Compute API
NVIDIA Run:ai ConfigMap API
NVIDIA Run:ai Credentials API
NVIDIA Run:ai Datasources API
NVIDIA Run:ai Datavolumes API
#NVIDIA NIM