Estimate Cost

Estimate Cost says what the model call ahead is expected to cost, before the call is made. It holds nothing and refuses nothing.

What it does

When an execution reaches the node, Slen estimates the call of the AI Model node the chain leads to, the way the hold of that call will be estimated: the same model, the messages of the input, the output ceiling of the model node and the policies that change them. The node writes the estimate into the trace as an estimated event and passes the execution on through Default.

The figure is a forecast. The model node estimates the call again when it takes the hold, a moment later, and when the call is answered that estimate is written on this node as a second estimated event. The two tell themselves apart by source: preflight for the forecast, authorize for the estimate of the hold.

When the call cannot be estimated, the event says so, and the execution passes on all the same. The model node then decides about the call on its own.

When to use it

• To see on the canvas and in the trace what a call was expected to cost before any money was held.

• To let a later node read the estimate: a Condition can compare it with a threshold and choose a branch.

• Budget Check does not need the node: it makes the same estimate itself.

• The caller gets the estimate without the node: cost.estimated of the execution adds up the estimates the holds of its calls were taken on — Executions.

Settings

The node has no settings. The safety multiplier of a hold is chosen by Slen from how far the estimates of the operation have missed so far, and a policy can replace it — Policies. A multiplier stored on the node in an older version is not read.

Ports

The node has one port, Default, and it is required.

Output

The node leaves the forecast for the nodes that follow.

{ "estimated_cost": "0.00048000", "safety_multiplier": "1.200" }

• estimated_cost is what the call is expected to cost.

• safety_multiplier is the multiplier the hold would be taken with. The hold is the estimate multiplied by it.

Both are decimal numbers written as text. A later node reads them by the path nodes.<key>.output.estimated_cost, where <key> is the key of this node in the flow. When the call could not be estimated, the output is empty.

Errors

The node raises no error of its own and has no setting to check. Its place is checked when the flow is published: the node leads to exactly one AI Model. One that leads to no model is misplaced_finance_node, and one that leads to several is ambiguous_finance_chain — The money chain.

Example

Estimate Cost writes the forecast of the call into the trace, AI Model holds the money and calls the model, and Return Response answers the caller. The trace shows both estimates of the call on the first node.

Related

• The money chain — the four nodes around a model call and the order they stand in.

• Budget Check — refuse a call before any money is held.

• AI Model — the node that takes the hold.