Refuse or limit a request before money is reserved
Write a policy — the conditions a request is checked against and what happens when they hold — put the policies in order and try a request against them.
5 min read
A policy is a rule that is applied to a request before money is reserved. It refuses the request or changes its limits. This page is about the policies page of the console.
Where the policies are
Open Settings, then Policies. The page lists the policies of every project you reach and the ones that apply to all projects, in the order they are checked.
A row shows the name of the policy, where it applies, its priority and the rule itself, read as one sentence.
The page opens for whoever has the management:read permission. Creating, changing and deleting a policy need management:write. A policy that applies to all projects is changed only by a member who reaches every project.
How a request is checked
Slen checks a request against the enabled policies of its project and the enabled policies that apply to all projects. It does so before the budgets are checked and before money is reserved.
The policy with the highest priority is checked first.
A policy applies when all of its conditions hold. A policy with no conditions applies to every request.
The first policy that refuses stops the request. The request ends with the code
policy_denied, and the message of the error names the policy.A policy that changes a limit does not stop the check. The policies after it are still checked.
Conditions
| Condition | Compared as | What is compared |
|---|---|---|
| Spent | At least, at most | What the project has spent since the first day of the month |
| Request cost | At least, at most | The estimated cost of the request |
| Operations in flight | At least, at most | The number of reservations the project holds at that moment |
| Model | Is, one of | The model code of the request |
| Operation | Is | The operation code of the request |
| Provider | Is | The provider of the model |
| End customer | Is | The id of your customer the request names |
| Time of day | Between | The time the request arrives at, in the time zone you choose. A window that ends before it starts crosses midnight |
Actions
| Action | What it does |
|---|---|
| Deny the request | Refuses every request the policy applies to |
| Allow the request | Changes nothing by itself. The policies after it are still checked, and the budgets still apply |
| Allow only these models | Refuses a request whose model is not in the list |
| Cap the cost of one call | Refuses a request whose estimated cost is above the ceiling |
| Change the safety multiplier | Sets what the estimate is multiplied by to get the reservation. With 1.5 the reservation is one and a half times the estimate. When several such policies apply, the first one checked decides |
| Change the output-token ceiling | Lowers the number of output tokens the request may ask for, before its cost is estimated |
Change the output-token ceiling is applied before the cost and the provider of the request are known. Give such a policy only the conditions Model, Operation, End customer and Time of day. With any other condition the policy is not saved, in the console or through the API. A policy of this kind that already carries another condition never applies.
Write a policy
1. Start a new policy
Press New policy.
2. Name the policy
The reason of a refusal quotes the name, so say what the policy is for. The name can be up to 120 characters long.
3. Choose where it applies
Under Applies to choose a project or All projects of the organization. The second choice is offered only to a member who reaches every project. After the policy is saved, where it applies cannot be changed.
4. Add the conditions
Under When choose what is compared, how, and the value. Press Add a condition for one more. The form does not save the policy while a condition has no value. Remove every condition to make the policy apply to every request.
5. Choose the action
Under Then choose the action and fill in its value. The form does not save the policy while the value is of the wrong kind: the models are a list of codes separated by commas, the ceiling of the cost is an amount written with a dot and up to eight decimal places, the safety multiplier is a number above 0 and below 1000 with up to three, and the output-token ceiling is a whole number of 1 or more.
6. Set the priority
Open Order of application and set Priority, a whole number from 0 to 1000. A new policy starts at 100. A refusal stops the request, so give a strict policy a higher number.
7. Create the policy
Press Create policy.
A new policy is enabled and acts on the next request. A policy that denies and has no conditions refuses every request where it applies.
Turn off, change or delete a policy
The switch in the row of a policy says Enabled. Turn it off, and requests are no longer checked against the policy, while the policy keeps its settings. The menu in the row holds Edit and Delete.
Try a request
Press Try a request, choose the Project the request comes from and fill in the request: the model, the operation, the provider, the estimated cost, the time and the end customer. The panel answers Allowed or Denied, gives the reason and says, policy by policy, what each one did.
The check is made in your browser. Nothing is sent, and nothing is spent.
The check runs the policies a real request of the chosen project meets: the ones of that project and the ones for all projects.
A policy with the condition Spent or Operations in flight depends on live counters, so the panel does not judge it and says so.
A policy that changes the output-token ceiling and carries a condition it cannot see is marked as one that never applies.
Where to go next
Glossary — what an estimate, a reservation and a budget are.
Audit log — who created, changed or deleted a policy.
Error codes — every code of the API, what it means and what to do about it.
Budgets — cap what a period may spend; the budgets are checked after the policies.