Adding a provider

An AIGateway provider is how Ankole talks to an upstream AI service — an LLM, an embedding model, a web-search API. This page is the contributor walkthrough: the provider DSL, the compiled ProviderDefinition it produces, the settings and capabilities a provider declares, and the prepare function that owns request construction. It builds on the AIGateway concept page; this is how to add a provider.

The decisive property, stated up front: the provider module owns request preparation in Elixir; the Rust UniversalAIClient owns the wire — transport, encoding, response normalization. The compiled definition describes only stable metadata, settings, and which prepare function owns each capability. A provider is not a full HTTP client; it is a prepare function plus a declaration.

The provider DSL

A provider module uses Ankole.AIGateway.ProviderDSL and declares itself with a small block-structured DSL. The DSL records metadata and capability ownership; it deliberately does not describe request body fields — each provider’s prepare function is normal Elixir code in the same module.

defmodule Ankole.AIGateway.Providers.MyProvider do
  use Ankole.AIGateway.ProviderDSL

  provider :my_provider do
    label(%{"default" => "My Provider"})
    base_url("https://api.example.com/v1")

    setting(:api_key, encrypted: true, scope: :credential)

    language_model do
      upstream(:sse)
      api_resolver(:openai_responses)
      prepare(:prepare_language_model)
      supports_parallel_tool_calls()
    end
  end

  def prepare_language_model(context) do
    # normal Elixir — build the prepared request from the context
  end
end

The provider block compiles into a ProviderDefinition struct, consumed by the AIGateway registry at runtime.

The compiled definition

A ProviderDefinition carries:

The registry resolves a provider by provider_kind, looks up the capability for the requested kind, calls the capability’s prepare function, and hands the result to UniversalAIClient.

Settings

A Setting declares one operator or request option:

setting(:api_key, encrypted: true, scope: :credential)
setting(:organization, advanced: true)
setting(:reasoningEffort, type: :select, default: "high",
       options: ["minimal", "low", "medium", "high", "xhigh"], scope: :request)

Every provider row has a credential pool, including the one-member case. The resolver selects one healthy member and decrypts its :credential settings before it calls the prepare function. Connection settings such as endpoint and custom headers stay shared by the full row. Request settings come from the model profile. The prepare function reads all three scopes from the resolved settings map and does not implement pool selection.

advanced? is presentation only. It hides the field behind an advanced toggle in the Console and does not change validation or behavior.

Capabilities

A Capability binds a user-facing model capability to the prepare function and the wire shape:

language_model do
  upstream(:sse)
  api_resolver(:openai_responses)
  prepare(:prepare_language_model)
  supports_parallel_tool_calls()
  supports_native_image_generation()
end

The six capability kinds map to the external names the runtime uses: language_model → "llm", embedding_model → "embedding", rerank_model → "rerank", and the three web/image kinds are unchanged. A provider that serves only LLMs declares only language_model; a provider that serves LLMs and embeddings declares both.

The prepare function

The prepare function is normal Elixir code in the provider module, named by the capability’s prepare field. It receives a PrepareContext (the resolved settings, the model request, the agent’s context) and returns a prepared request that UniversalAIClient sends:

def prepare_language_model(%PrepareContext{} = context) do
  %UniversalAIRequest{}
  |> put_url(context.settings.base_url, "/responses")
  |> put_auth("Bearer", context.settings.api_key)
  |> put_body(context.request)
end

This is where provider differences live — URL construction, auth headers, body shaping, and the rules of a specific upstream API. The prepare function is the provider’s actual work; the DSL declaration is how that work is discovered and routed. Credential retry remains outside this function: the control plane can select another member, rebuild the request, and ask the kernel to perform one new transport attempt.

Register the provider

First-party providers are compiled into the release and discovered by the registry. For a plugin-contributed provider, declare it through the ai_gateway.provider contract in a Control Plane Plugin’s adapter_declarations/0, and the registry picks it up from the plugin’s declaration — same model as a signal adapter, different contract id.

See Develop Skills and Control Plane Plugins for Plugin registration. The Provider contract is ai_gateway.provider, and the kind ID must match ~r/\A[a-z][a-z0-9_]{0,62}\z/.

What this guide is not

It is not an HTTP-client tutorial — the prepare function builds a prepared request, and the Rust client does the HTTP. It is not a way to bypass the kernel’s transport; the api_resolver and upstream wire shape are what the Rust UniversalAIClient owns, and a provider picks from the existing resolvers rather than inventing its own transport. And it is not a substitute for reading the existing providers; lib/ankole/ai_gateway/providers/ is the canonical reference, and the simplest one (OpenAI or openai_compatible) is the right starting point.

Next steps

• For the AIGateway concept (routing, resolution, the unified boundary), read AIGateway.
• For Plugin registration, read Develop Skills and Control Plane Plugins.
• For the first Provider setup, read Quick start.