Cognite Records API

Records are mutable or immutable data objects (depending on the stream template) stored in a stream. Records are created by ingesting data into a stream.Records are shaped similarly to instances in the Data Modeling service, and their schema is defined by the containers referenced as sources.Each record is associated with a space. Space-based access control applies to records.This section specifies the `records` resource and uses the following Data Modeling concepts: - [Spaces](https://docs.cognite.com/cdf/dm/dm_concepts/dm_spaces_instances#space) - [Containers](https://docs.cognite.com/cdf/dm/dm_concepts/dm_containers_views_datamodels#containers)**Note:** Every `record` has a required top-level `externalId` property, which can be usedin some queries to retrieve or aggregate records (for filtering or sorting).For mutable records, `externalId` (together with `space`) uniquely identifies a record for write operations (create, upsert, delete).For immutable records, the records API does not enforce uniqueness of the `space` + `externalId` pair, so multiple records may share the same identifier.## Rate and concurrency limitsBoth the rate of requests (denoted as requests per second, or ‘**rps**’) and the number of concurrent (parallel)requests are governed by limits for all CDF API endpoints. If a request exceeds a limit,it will be throttled with a `429: Too Many Requests` response. See [Resource throttling](https://docs.cognite.com/dev/concepts/resource_throttling) for limit types and how to avoid throttling.Limits apply to the API endpoints for this service.Some request types consume more resources (compute, storage I/O) than others. For example, ingest requestsare less resource-intensive than analytical requests (Aggregate, Retrieve).Limits for query endpoints (Sync, Retrieve, Aggregate) are hierarchical:- All query endpoints share a common **Query request budget** (shown in the tables below)- The Retrieve endpoint has an **additional** dedicated budget that is checked first- The Aggregate endpoint has an **additional** dedicated budget that is checked firstThe Sync endpoint only checks the Query request budget. The Retrieve and Aggregate endpointsmust pass both their dedicated budget check AND the Query request budget check.For example, with mutable streams you can make up to 40 rps total across all query endpoints(Query budget limit), but only up to 20 of those can be Retrieve requests (Retrieve budget limit)and only up to 15 can be Aggregate requests (Aggregate budget limit).That means you could send 20 Retrieve + 15 Aggregate + 5 Sync = 40 total RPS.Query performance and rate limits vary between mutable and immutable streams because oftheir different storage characteristics.Mutable streams provide consistent high-performance queries and higher rate limits(see "mutable streams" limits in the tables below).Immutable streams are optimized for ingesting very large amounts of data,which results in lower query performance and stricter rate limits than mutable streams(see "immutable streams" limits in the tables below).When designing data access patterns, use mutable streams for high-performance queries and higher rate limits, or immutable streams when the priorityis high-volume ingestion and long-term storage.The amount of data returned in responses from query endpoints (Sync, Retrieve, Aggregate)is also limited. Prefer reading only the data you need: use an appropriate `filter` and limit which `sources` are retrieved, rather thanretrieving large result sets that you will not use.The version limits for the records endpoints are shown in the tables below.These limits are subject to change, pending review of consumption patterns and resource availability over time: Ingest request budget Overall Per ID Requests per second 40 30 Concurrent requests 20 15 Query request budget Overall Per ID Requests per second: mutable streams 40 30 Concurrent requests: mutable streams 30 22 Requests per second: immutable streams 10 7 Concurrent requests: immutable streams 10 7 Response MB per second 4 3 Additional dedicated budgets for Retrieve and Aggregate endpoints: The Retrieve and Aggregate endpoints have dedicated budgets that are checked in addition to the Query request budget shown above. A request to these endpoints must pass both budget checks. Retrieve request budget Overall Per ID Requests per second: mutable streams 20 15 Concurrent requests: mutable streams 20 15 Requests per second: immutable streams 10 7 Concurrent requests: immutable streams 10 7 Note: Retrieve endpoint requests are checked against both this budget and the Query request budget above. Aggregate request budget Overall Per ID Requests per second: mutable streams 15 12 Concurrent requests: mutable streams 10 7 Requests per second: immutable streams 5 4 Concurrent requests: immutable streams 5 4 Note: Aggregate endpoint requests are checked against both this budget and the Query request budget above. Summary: Sync endpoint: Only limited by Query request budget Retrieve endpoint: Limited by Retrieve request budget AND Query request budget Aggregate endpoint: Limited by Aggregate request budget AND Query request budget