Cache Write

A node that stores the answer of the model under the key a lookup in the cache above it missed, so that the same request is answered from the cache next time.

3 min read

Cache Write stores an answer for later. It takes the answer of the model above it and stores it under the key that Cache Lookup looked up and did not find.

Node

Cache Write

Type

cache_write

Category

Cache

Ports

Default default must be connected

Ends the path

What it does

When an execution reaches the node, it stores the last answer of AI Model on the path for the time its setting says, and leaves through Default. The next request with the same key is answered from the cache by the lookup.

The node stores nothing, and the execution goes on all the same, when:

no lookup above it missed on this execution with a key it could use — the lookup hit, its key said nothing about the request, or there is no lookup;
the answer did not end cleanly: the model stopped for another reason than the end of its answer, or the answer is empty;
the answer is larger than the cache takes — 256 KB by default;
the cache store could not be reached.

When to use it

Put it after AI Model on the Miss branch of a Cache Lookup. One lookup and one write make the pair: the lookup asks, the write answers the question the lookup could not.

Settings

Setting	What it is for
`ttl_seconds`	How long the answer is kept, in whole seconds, at least 1. A node without a value keeps it as long as the service is set up to — 3600 seconds by default.

In the builder the setting is Keep for (seconds). A new node keeps an answer 3600 seconds, an hour; a node without a value keeps it as long as the service is set up to — an hour by default.

Ports

The node has one port, Default, and it is required. A flow in which it leads nowhere is not published.

Output

The node leaves no output. A later node reads the answer of the model from the output of AI Model.

Errors

The node raises no error. When it stores nothing, a warning event of the trace says why. These are the warnings the node writes:

JSON

[
  { "type": "warning", "node_key": "store", "payload": { "code": "cache_write_without_lookup" } },
  { "type": "warning", "node_key": "store", "payload": { "code": "cache_write_unfinished_answer" } },
  { "type": "warning", "node_key": "store", "payload": { "code": "cache_write_empty_answer" } },
  { "type": "warning", "node_key": "store", "payload": { "code": "cache_value_refused" } },
  { "type": "warning", "node_key": "store", "payload": { "code": "cache_store_unavailable" } }
]

Code	Why nothing was stored
`cache_write_without_lookup`	No lookup above the node missed on this execution
`cache_write_unfinished_answer`	The model did not finish its answer
`cache_write_empty_answer`	The answer has no text
`cache_value_refused`	The answer is larger than the cache takes
`cache_store_unavailable`	The cache store could not be reached

A node with no Cache Lookup on any way above it gets the warning cache_write_without_lookup of the publish check; the flow is published all the same. The setting is checked when the flow is validated and when it is published.

In the API an entry of issues names such a check in params.detail; its code is invalid_node_config.

Check	Message in the builder
`cache_invalid_ttl`	The lifetime of “`{node}`” must be a whole number of seconds, at least 1.

Example

On a miss the model answers, the node stores the answer, and Return Response answers the caller with it. The same request after that is a hit.

Cache Lookup — the key and who shares an answer.
How a flow is built — every node of the builder.
AI Model — the call whose answer the cache keeps.

Built from openapi b070e1f, sdk contract 5cc7055, php ae4a53b, node 84bc62d