Skip to content
Ankole

Cost management

For AI Agents: the Markdown version of this page is at https://ankole.agentbull.com/en-US/docs/cost-management/index.md. The documentation index is at https://ankole.agentbull.com/en-US/llms.txt.

Model tokens account for most of Ankole’s operational cost, so control-plane settings give you direct control over which models run, how many iterations a turn executes, and how much background fanout or retry work can occur. Every lever on this page maps to a concrete control-plane setting.

Lever 1: model-profile tiers

Model Profiles resolve through 3-tier sparse inheritance (Site -> Pod -> Agent/System) in Console → Model Profiles. Each Agent exposes seven built-in slots—four language-model slots plus web_search, web_fetch, and image_generate—while Site, Pod, and system Principals (ankole_brain and system) add the rerank, decision, speech, transcription, ocr, video, and music slots. Unset slots inherit from Pod and Site defaults, with cross-slot fallbacks heavy -> primary and light -> primary.

Slot When it runs Cost impact
primary Main reasoning model for most turns Largest single cost driver
light High-volume, low-complexity paths (falls back to primary) Bind to a low-cost model to avoid billing bulk work at primary rates
heavy Complex synthesis, Background Agent Jobs, and Dreaming (falls back to primary) Selects the provider and model for deep reasoning and durable background work
vision_fallback When primary cannot process an image Bind only when the Agent receives images
web_search, web_fetch Web tools See Lever 3
image_generate Image generation High cost per call; bind only when needed
rerank (Site / Pod / system) Brain recall reranking and /rerank defaults Keep bound to a fast, low-cost reranker
decision (Site / Pod / system) /decisions answers with probabilities Bind a decision model; left unset, the light model answers
speech (Site / Pod / system) /audio/speech turns text into audio Bind a text-to-speech model; left unset, speech requests fail
transcription (Site / Pod / system) /audio/transcriptions turns audio into text Bind a speech-to-text model; left unset, transcription requests fail
ocr (Site / Pod / system) /ocr reads PDFs and images as Markdown Bind an OCR model to read PDFs; left unset, a language model reads images only. Once bound, Pod also sends the scanned pages of newly uploaded PDFs to it, and the provider charges for every page
video (Site / Pod / system) /videos generates videos Bind a video model; left unset, video requests fail. The provider charges for each generation
music (Site / Pod / system) /audio/music generates music Bind a music model; left unset, music requests fail. The provider charges for each generation

Company Brain maintenance and Pod Applet calls have different execution subjects:

  • ankole_brain: Executes all Company Brain model calls and records usage under ankole_brain—resolving light (fallback primary) for conversation and Source extraction, heavy (fallback primary) for Dreaming and Skill lesson review, web_fetch for URL Sources, and rerank for recall reranking, plus the site-level brain.embedding_model key in AppConfigure. Override any slot for ankole_brain in Console → Model Profiles whenever Brain maintenance should run on a separate budget tier from foreground Agents; Brain → Health reports profile readiness (see Brain).
  • Pod Applet ctx.ai: Ordinary calls run as the current caller; the admin capability runs as the Applet owner. AIGateway resolves the applicable profiles and records the call for that subject. Applet-origin calls do not enter Agent token-quota windows.

Two profile adjustments save the most spend:

  • Bind light to a low-cost model at the Site or Pod level. Because light falls back to primary when unset and handles high-volume paths (including ankole_brain extraction), binding a dedicated low-cost model immediately lowers bulk spend across every inheriting Agent and ankole_brain.
  • Size primary for normal turns. Start with a balanced model for primary and override primary or heavy only on specialist Agents whose task quality demands a larger tier.

Check the effective optional profiles inherited from Site and Pod before changing Agent overrides. Vision fallback is unavailable when the effective vision_fallback profile is empty. When Use the model’s provider is on for image generation, the main provider’s native capability is used and the dedicated profile is ignored. Turn that switch off to use image_generate. Similarly, when heavy is unset, Background Agent Jobs fall back to primary on AIGateway; configure heavy whenever complex reasoning or Background Agent Jobs should use a dedicated frontier model or subscription pool.

Lever 2: reasoning effort

For providers that support Codex reasoning effort, model_reasoning_effort offers six levels: minimal | low | medium | high | xhigh | max (defaulting to high). Console lists only the levels that the selected model accepts. Lower effort reduces both latency and token spend, whereas higher effort improves difficult synthesis at higher token cost.

Before moving an Agent’s primary profile to a larger model, try tuning model_reasoning_effort first, reserving high and above for Agents that tackle complex synthesis rather than routine operational turns.

Lever 3: web tools

web_search and web_fetch are independent profiles that bill per call:

  • Check the selected search path. Web search defaults to Use the model’s provider. Turn that switch off to use the dedicated web_search profile; unbinding that profile does not disable native search. Limit unnecessary calls through the Agent’s task instructions.
  • Prefer web_fetch over web_search when you already know the URL. Fetching a known URL takes a single call, whereas searching incurs a search call plus follow-up fetches.

The agent_workers.rendered_fetch_idle_ttl_ms AppConfigure key sets the idle timeout before the Worker releases an idle browser-sidecar route used for rendered web_fetch calls. Adjusting this key controls how long the Worker retains browser memory rather than caching URL responses.

Lever 4: agent-loop budgets and cold-wake rotation

Tenant-scoped (scope: :tenant) AppConfigure keys cap per-turn spend and prevent cold prompt-cache re-billing on long-lived threads:

Key What it controls
ai_agent.max_iterations Agent-loop model iterations per turn
ai_agent.max_output_tokens Output token cap per model response
ai_agent.inactivity_timeout_ms Maximum idle time before an inactive turn is reaped
sessions.wake_recycle Cold-wake conversation rotation policy (min_cold_gap_seconds: 3600, min_context_tokens: 128000, im_max_idle_seconds: 259200)

ai_agent.max_iterations bounds multi-step tool loops, while ai_agent.max_output_tokens bounds the length of each model response. Both ai_agent.* keys inherit the site default whenever a tenant has no override. Size them for normal turns in each tenant: when a complex turn reaches max_iterations, the loop prompts the Agent to synthesize a final answer from the results collected so far.

Once an upstream provider’s prompt cache expires, resuming a large stored model thread re-bills the full uncached input context on the next turn. sessions.wake_recycle eliminates that repeated cold-cache penalty: when a Session wakes after min_cold_gap_seconds (default 1 hour) with at least min_context_tokens (128,000 tokens), or after im_max_idle_seconds (72 hours) on external chat channels, Ankole rotates the underlying AIGateway model conversation onto a fresh successor branch seeded with a compact <wake-briefing> while keeping the visible room history intact.

Lever 5: Workflow fanout and task attempts

Each Workflow agent() attempt runs a complete model turn. Because a single agent() call can make up to three attempts, a Workflow run with wide fanout can multiply model and web-tool usage from one conversation request.

AppConfigure key Default Maximum What it controls
workflow.max_concurrency_per_run 8 32 Tasks from one run that can execute concurrently
workflow.max_running_per_agent 8 64 Running Workflow tasks across all runs of one Agent
workflow.max_agent_calls_per_run 32 1,024 Total subagent calls that one run can create

Increasing concurrency shortens wall-clock time without reducing total model calls, so size max_agent_calls from the finite input collection and request only the concurrency you need. A Workflow program can request lower concurrency and max_agent_calls values for a run, whereas workflow.max_running_per_agent is operator-only and no run can exceed an AppConfigure limit.

Because Workflow enforces per-turn iteration, output-token, and inactivity limits on each task rather than a batch-wide token budget, keep task prompts narrow, return compact structured results, handle null failures explicitly, and split large collections across runs when needed. See Workflows.

Lever 6: background-job retry and slot caps

Background Agent Jobs consume tokens across execution attempts and concurrent slots:

Cap Value Effect
max_execution_attempts 5 Execution failures before a job transitions to failed
max_consecutive_turn_failures 5 Consecutive turn failures before a job stops
max_running_per_agent 3 (default) Concurrent running jobs per Agent, configurable per tenant via background_agent_jobs.max_running_per_agent (1..64)
retry delay ~30s Minimum delay between retry attempts

Ankole can retry transient execution failures until the Job reaches five execution failures; configuration errors fail immediately. A claim starts or resumes an attempt, and the separate total-claims cap is 25. Credential-pool backpressure does not consume the execution-failure budget. Claims do not each imply a complete model run or the same token cost. Lower background_agent_jobs.max_running_per_agent (or instruct the Agent in its persona to run one job at a time) whenever a tenant does not need three concurrent background model loops per Agent.

Lever 7: per-Agent token quota

While the six levers above bound per-turn and per-job behavior, the per-Agent token quota caps cumulative token consumption over a repeating period: once an Agent reaches its limit, AIGateway rejects further model calls for that Agent until the period ends.

Configure the period length (in days), period start time, and token limit on the Agent page. When a conversation hits that limit, Ankole replies with the tokens used, the configured limit, and the period end time; Background Agent Jobs at the limit fail immediately. During an authorized traffic spike, click Reset period to start a fresh quota period right away. See Agents.

Inspect where spend occurs

Before changing models or concurrency, check which Agent, conversation, Workflow, or Background Agent Job actually generated the calls:

  • GET /api/v2/ai-gateway/conversations lists recent model calls, resolved profiles, call counts, and providers so you can distinguish high-volume primary traffic, expensive heavy calls, and frequent web_search calls.
  • Ask the main Agent to inspect its Workflow run directly; task counts show total fanout and failed attempts (Ankole does not expose a separate Workflow Console page or per-run currency total).
  • GET /api/v2/background-agent-jobs reports Job attempts, the claim count. Inspect the recorded model calls to determine the actual token cost.
  • Structured control-plane logs emit provider call events tagged with Agent and provider fields for aggregation in your log store.

Worked example

Suppose a tenant’s weekly token spend doubles while primary call volume stays flat, and /api/v2/ai-gateway/conversations reveals a tenfold jump in web_search calls from a channel assistant configured with may_intervene. Tightening the Agent’s persona instructions (“run web_search only when a participant asks an explicit factual question”) or narrowing its Signal Binding policy eliminates the extra spend at its source without throttling normal turns.

Scope of cost controls

Ankole governs model selection, reasoning effort, tool availability, loop iterations, concurrency, and token quotas; it does not display a real-time currency billing dashboard because actual currency spend depends on your upstream provider’s pricing tiers. Inspect conversation and job logs first, then tune the specific lever driving the usage.

Next steps

  • For Agent model profiles and quotas, read Agents.
  • For ai_agent.*, brain.*, pod.applet.*, and agent_computer.* runtime settings, read AppConfigure.
  • For conversation and Background Agent Job endpoints, read Console API reference.
  • For bounded subagent fanout, read Workflows.
  • For Background Agent Job execution limits, read Background Agent Jobs.