LLM Providers

Mango supports four LLM providers out of the box. All implement the same LLMService interface, so switching providers is a one-line change.

Anthropic

pip install mango-ai[anthropic]

from mango.integrations.anthropic import AnthropicLlmService

llm = AnthropicLlmService(
    api_key="YOUR_KEY",          # or set ANTHROPIC_API_KEY env var
    model="claude-sonnet-4-6",   # default
    max_tokens=4096,
    temperature=None,            # optional; omit for the provider default
)

Recommended models:

• claude-sonnet-4-6 — best balance of speed and accuracy (default)
• claude-opus-4-6 — highest accuracy, higher cost

OpenAI

pip install mango-ai[openai]

from mango.integrations.openai import OpenAILlmService

llm = OpenAILlmService(
    api_key="YOUR_KEY",          # or set OPENAI_API_KEY env var
    model="gpt-5.4",             # default
    max_completion_tokens=4096,
    base_url=None,               # optional — OpenAI-compatible endpoints (Together AI, Groq, self-hosted)
    temperature=None,            # optional; omit for reasoning models
)

base_url is forwarded to the OpenAI client as-is, so leave it None (or omit it) unless you really target another endpoint — an empty string is not treated as "unset".

Google

pip install mango-ai[gemini]

from mango.integrations.google import GeminiLlmService

llm = GeminiLlmService(
    api_key="YOUR_KEY",          # or set GOOGLE_API_KEY env var
    model="gemini-3.1-flash-lite-preview",   # default
    vertexai=False,              # True to route through Vertex AI
    max_output_tokens=8192,
    temperature=1.0,
)

Ollama (local models)

Run Mango fully offline with any model available on Ollama.

pip install mango-ai[ollama]
ollama pull llama3.2   # or mistral, qwen2.5, etc.

from mango.integrations.ollama import OllamaLlmService

llm = OllamaLlmService(
    model="llama3.2",               # default
    host="http://localhost:11434",  # optional, this is the default
    max_tokens=4096,
)

Ollama is available from Python only — the mango CLI accepts anthropic, openai and gemini.

Requirements: The model must support tool/function calling. Not all Ollama models do — check the model's documentation before using it with Mango.

Recommended models with tool support: llama3.2, mistral, qwen2.5, command-r.

Custom LLM provider

Implement LLMService to use any LLM with tool/function calling support:

from mango.llm import LLMService, LLMResponse, Message, ToolDef, SystemPromptPart

class MyCustomLlm(LLMService):
    def chat(
        self,
        messages: list[Message],
        tools: list[ToolDef],
        system_prompt: str = "",
        system_prompt_parts: list[SystemPromptPart] | None = None,
    ) -> LLMResponse:
        # The agent calls chat(messages=..., tools=..., system_prompt_parts=[...]).
        # Join the parts (part.text) into your provider's system prompt; honour
        # part.cacheable if the provider supports explicit prompt caching.
        # Return LLMResponse with text and/or tool_calls
        ...

    def get_model_name(self) -> str:
        return "my-custom-model"

LLMResponse fields: