Cost management
For AI Agents: the Markdown version of this page is at https://ankole.agentbull.com/en-US/docs/cost-management/index.md. The documentation index is at https://ankole.agentbull.com/en-US/llms.txt.
Model tokens account for most of Ankole’s operational cost, so control-plane settings give you direct control over which models run, how many iterations a turn executes, and how much background fanout or retry work can occur. Every lever on this page maps to a concrete control-plane setting.
Lever 1: model-profile tiers
Model Profiles resolve through 3-tier sparse inheritance (Site -> Pod -> Agent/System) in Console → Model Profiles. Each Agent exposes seven built-in slots—four language-model slots plus web_search, web_fetch, and image_generate—while Site, Pod, and system Principals (ankole_brain and system) add the rerank, decision, speech, transcription, ocr, video, and music slots. Unset slots inherit from Pod and Site defaults, with cross-slot fallbacks heavy -> primary and light -> primary.
| Slot | When it runs | Cost impact |
|---|---|---|
primary |
Main reasoning model for most turns | Largest single cost driver |
light |
High-volume, low-complexity paths (falls back to primary) |
Bind to a low-cost model to avoid billing bulk work at primary rates |
heavy |
Complex synthesis, Background Agent Jobs, and Dreaming (falls back to primary) |
Selects the provider and model for deep reasoning and durable background work |
vision_fallback |
When primary cannot process an image |
Bind only when the Agent receives images |
web_search, web_fetch |
Web tools | See Lever 3 |
image_generate |
Image generation | High cost per call; bind only when needed |
rerank (Site / Pod / system) |
Brain recall reranking and /rerank defaults |
Keep bound to a fast, low-cost reranker |
decision (Site / Pod / system) |
/decisions answers with probabilities |
Bind a decision model; left unset, the light model answers |
speech (Site / Pod / system) |
/audio/speech turns text into audio |
Bind a text-to-speech model; left unset, speech requests fail |
transcription (Site / Pod / system) |
/audio/transcriptions turns audio into text |
Bind a speech-to-text model; left unset, transcription requests fail |
ocr (Site / Pod / system) |
/ocr reads PDFs and images as Markdown |
Bind an OCR model to read PDFs; left unset, a language model reads images only. Once bound, Pod also sends the scanned pages of newly uploaded PDFs to it, and the provider charges for every page |
video (Site / Pod / system) |
/videos generates videos |
Bind a video model; left unset, video requests fail. The provider charges for each generation |
music (Site / Pod / system) |
/audio/music generates music |
Bind a music model; left unset, music requests fail. The provider charges for each generation |
Company Brain maintenance and Pod Applet calls have different execution subjects:
ankole_brain: Executes all Company Brain model calls and records usage underankole_brain—resolvinglight(fallbackprimary) for conversation and Source extraction,heavy(fallbackprimary) for Dreaming and Skill lesson review,web_fetchfor URL Sources, andrerankfor recall reranking, plus the site-levelbrain.embedding_modelkey in AppConfigure. Override any slot forankole_brainin Console → Model Profiles whenever Brain maintenance should run on a separate budget tier from foreground Agents; Brain → Health reports profile readiness (see Brain).- Pod Applet
ctx.ai: Ordinary calls run as the current caller; the admin capability runs as the Applet owner. AIGateway resolves the applicable profiles and records the call for that subject. Applet-origin calls do not enter Agent token-quota windows.
Two profile adjustments save the most spend:
- Bind
lightto a low-cost model at the Site or Pod level. Becauselightfalls back toprimarywhen unset and handles high-volume paths (includingankole_brainextraction), binding a dedicated low-cost model immediately lowers bulk spend across every inheriting Agent andankole_brain. - Size
primaryfor normal turns. Start with a balanced model forprimaryand overrideprimaryorheavyonly on specialist Agents whose task quality demands a larger tier.
Check the effective optional profiles inherited from Site and Pod before changing Agent overrides. Vision fallback is unavailable when the effective vision_fallback profile is empty. When Use the model’s provider is on for image generation, the main provider’s native capability is used and the dedicated profile is ignored. Turn that switch off to use image_generate. Similarly, when heavy is unset, Background Agent Jobs fall back to primary on AIGateway; configure heavy whenever complex reasoning or Background Agent Jobs should use a dedicated frontier model or subscription pool.
Lever 2: reasoning effort
For providers that support Codex reasoning effort, model_reasoning_effort offers six levels: minimal | low | medium | high | xhigh | max (defaulting to high). Console lists only the levels that the selected model accepts. Lower effort reduces both latency and token spend, whereas higher effort improves difficult synthesis at higher token cost.
Before moving an Agent’s primary profile to a larger model, try tuning model_reasoning_effort first, reserving high and above for Agents that tackle complex synthesis rather than routine operational turns.
Lever 3: web tools
web_search and web_fetch are independent profiles that bill per call:
- Check the selected search path. Web search defaults to Use the model’s provider. Turn that switch off to use the dedicated
web_searchprofile; unbinding that profile does not disable native search. Limit unnecessary calls through the Agent’s task instructions. - Prefer
web_fetchoverweb_searchwhen you already know the URL. Fetching a known URL takes a single call, whereas searching incurs a search call plus follow-up fetches.
The agent_workers.rendered_fetch_idle_ttl_ms AppConfigure key sets the idle timeout before the Worker releases an idle browser-sidecar route used for rendered web_fetch calls. Adjusting this key controls how long the Worker retains browser memory rather than caching URL responses.
Lever 4: agent-loop budgets and cold-wake rotation
Tenant-scoped (scope: :tenant) AppConfigure keys cap per-turn spend and prevent cold prompt-cache re-billing on long-lived threads:
| Key | What it controls |
|---|---|
ai_agent.max_iterations |
Agent-loop model iterations per turn |
ai_agent.max_output_tokens |
Output token cap per model response |
ai_agent.inactivity_timeout_ms |
Maximum idle time before an inactive turn is reaped |
sessions.wake_recycle |
Cold-wake conversation rotation policy (min_cold_gap_seconds: 3600, min_context_tokens: 128000, im_max_idle_seconds: 259200) |
ai_agent.max_iterations bounds multi-step tool loops, while ai_agent.max_output_tokens bounds the length of each model response. Both ai_agent.* keys inherit the site default whenever a tenant has no override. Size them for normal turns in each tenant: when a complex turn reaches max_iterations, the loop prompts the Agent to synthesize a final answer from the results collected so far.
Once an upstream provider’s prompt cache expires, resuming a large stored model thread re-bills the full uncached input context on the next turn. sessions.wake_recycle eliminates that repeated cold-cache penalty: when a Session wakes after min_cold_gap_seconds (default 1 hour) with at least min_context_tokens (128,000 tokens), or after im_max_idle_seconds (72 hours) on external chat channels, Ankole rotates the underlying AIGateway model conversation onto a fresh successor branch seeded with a compact <wake-briefing> while keeping the visible room history intact.
Lever 5: Workflow fanout and task attempts
Each Workflow agent() attempt runs a complete model turn. Because a single agent() call can make up to three attempts, a Workflow run with wide fanout can multiply model and web-tool usage from one conversation request.
| AppConfigure key | Default | Maximum | What it controls |
|---|---|---|---|
workflow.max_concurrency_per_run |
8 | 32 | Tasks from one run that can execute concurrently |
workflow.max_running_per_agent |
8 | 64 | Running Workflow tasks across all runs of one Agent |
workflow.max_agent_calls_per_run |
32 | 1,024 | Total subagent calls that one run can create |
Increasing concurrency shortens wall-clock time without reducing total model calls, so size max_agent_calls from the finite input collection and request only the concurrency you need. A Workflow program can request lower concurrency and max_agent_calls values for a run, whereas workflow.max_running_per_agent is operator-only and no run can exceed an AppConfigure limit.
Because Workflow enforces per-turn iteration, output-token, and inactivity limits on each task rather than a batch-wide token budget, keep task prompts narrow, return compact structured results, handle null failures explicitly, and split large collections across runs when needed. See Workflows.
Lever 6: background-job retry and slot caps
Background Agent Jobs consume tokens across execution attempts and concurrent slots:
| Cap | Value | Effect |
|---|---|---|
max_execution_attempts |
5 | Execution failures before a job transitions to failed |
max_consecutive_turn_failures |
5 | Consecutive turn failures before a job stops |
max_running_per_agent |
3 (default) | Concurrent running jobs per Agent, configurable per tenant via background_agent_jobs.max_running_per_agent (1..64) |
| retry delay | ~30s | Minimum delay between retry attempts |
Ankole can retry transient execution failures until the Job reaches five execution failures; configuration errors fail immediately. A claim starts or resumes an attempt, and the separate total-claims cap is 25. Credential-pool backpressure does not consume the execution-failure budget. Claims do not each imply a complete model run or the same token cost. Lower background_agent_jobs.max_running_per_agent (or instruct the Agent in its persona to run one job at a time) whenever a tenant does not need three concurrent background model loops per Agent.
Lever 7: per-Agent token quota
While the six levers above bound per-turn and per-job behavior, the per-Agent token quota caps cumulative token consumption over a repeating period: once an Agent reaches its limit, AIGateway rejects further model calls for that Agent until the period ends.
Configure the period length (in days), period start time, and token limit on the Agent page. When a conversation hits that limit, Ankole replies with the tokens used, the configured limit, and the period end time; Background Agent Jobs at the limit fail immediately. During an authorized traffic spike, click Reset period to start a fresh quota period right away. See Agents.
Inspect where spend occurs
Before changing models or concurrency, check which Agent, conversation, Workflow, or Background Agent Job actually generated the calls:
GET /api/v2/ai-gateway/conversationslists recent model calls, resolved profiles, call counts, and providers so you can distinguish high-volumeprimarytraffic, expensiveheavycalls, and frequentweb_searchcalls.- Ask the main Agent to inspect its Workflow run directly; task counts show total fanout and failed attempts (Ankole does not expose a separate Workflow Console page or per-run currency total).
GET /api/v2/background-agent-jobsreports Jobattempts, the claim count. Inspect the recorded model calls to determine the actual token cost.- Structured control-plane logs emit provider call events tagged with Agent and provider fields for aggregation in your log store.
Worked example
Suppose a tenant’s weekly token spend doubles while primary call volume stays flat, and /api/v2/ai-gateway/conversations reveals a tenfold jump in web_search calls from a channel assistant configured with may_intervene. Tightening the Agent’s persona instructions (“run web_search only when a participant asks an explicit factual question”) or narrowing its Signal Binding policy eliminates the extra spend at its source without throttling normal turns.
Scope of cost controls
Ankole governs model selection, reasoning effort, tool availability, loop iterations, concurrency, and token quotas; it does not display a real-time currency billing dashboard because actual currency spend depends on your upstream provider’s pricing tiers. Inspect conversation and job logs first, then tune the specific lever driving the usage.
Next steps
- For Agent model profiles and quotas, read Agents.
- For
ai_agent.*,brain.*,pod.applet.*, andagent_computer.*runtime settings, read AppConfigure. - For conversation and Background Agent Job endpoints, read Console API reference.
- For bounded subagent fanout, read Workflows.
- For Background Agent Job execution limits, read Background Agent Jobs.