LLM Providers

Mango supports three LLM providers out of the box. All implement the same LLMService interface, so switching providers is a one-line change.

Anthropic

pip install mango-ai[anthropic]

from mango.integrations.anthropic import AnthropicLlmService

llm = AnthropicLlmService(
    api_key="YOUR_KEY",          # or set ANTHROPIC_API_KEY env var
    model="claude-sonnet-4-6",   # default
    max_tokens=4096,
)

Recommended models:

• claude-sonnet-4-6 — best balance of speed and accuracy (default)
• claude-opus-4-6 — highest accuracy, higher cost

OpenAI

pip install mango-ai[openai]

from mango.integrations.openai import OpenAILlmService

llm = OpenAILlmService(
    api_key="YOUR_KEY",          # or set OPENAI_API_KEY env var
    model="gpt-5.4",             # default
    max_completion_tokens=4096,
    base_url="",                 # optional - useful for providers that mimic OpenAI API (e.g. Together AI, Groq, or self-hosted)
)

Google

pip install mango-ai[gemini]

from mango.integrations.google import GeminiLlmService

llm = GeminiLlmService(
    api_key="YOUR_KEY",          # or set GOOGLE_API_KEY env var
    model="gemini-3.1-pro-preview",
)

Ollama (local models)

Run Mango fully offline with any model available on Ollama.

pip install mango-ai[ollama]
ollama pull qwen3.5:9b   # or mistral, gemma, etc.

from mango.integrations.ollama import OllamaLlmService

llm = OllamaLlmService(
    model="qwen3.5:9b",               # default
    host="http://localhost:11434",  # optional, this is the default
)

Requirements: The model must support tool/function calling. Not all Ollama models do — check the model's documentation before using it with Mango.

Recommended models with tool support: llama3.2, mistral, qwen2.5, command-r.

Custom LLM provider

Implement LLMService to use any LLM with tool/function calling support:

from mango.llm import LLMService, LLMResponse, Message, ToolDef

class MyCustomLlm(LLMService):
    def chat(
        self,
        messages: list[Message],
        tools: list[ToolDef],
        system_prompt: str = "",
    ) -> LLMResponse:
        # Call your LLM API here
        # Return LLMResponse with text and/or tool_calls
        ...

    def get_model_name(self) -> str:
        return "my-custom-model"

LLMResponse fields: