FastAPI Server

Mango's HTTP layer is a thin adapter over the agent: it streams a turn as Server-Sent Events, keeps one conversation per session, and hands your user object to the access-control middlewares. It comes in two shapes.

Standalone server

from mango.servers.fastapi import MangoFastAPIServer

server = MangoFastAPIServer(agent, user_for=user_for)
server.run()                        # http://0.0.0.0:8000
server.run(host="127.0.0.1", port=9000, workers=1)   # extra kwargs go to uvicorn.run()

MangoFastAPIServer(
    agent,
    *,
    user_for=None,               # (request) -> user, sync or async; raise HTTPException(401) to reject
    cors_origins=None,           # str "https://a.com,https://b.com" or list; default: $MANGO_CORS_ORIGINS or "*"
    session_ttl_seconds=1800.0,  # idle timeout per conversation
    max_sessions=1000,           # LRU cap on live conversations
)

server.app is the underlying FastAPI instance if you need to add middleware or routes. Without user_for every request is anonymous and the server logs a warning at startup.

Router mounted in your app

from fastapi import FastAPI
from mango.servers.fastapi import mango_router

app = FastAPI()
app.include_router(mango_router(agent, user_for=user_for), prefix="/mango")

mango_router(
    agent,
    *,
    user_for=None,
    session_ttl_seconds=1800.0,
    max_sessions=1000,
    memory_endpoints=True,       # expose /memory/train|export|import
)

The full recipe — auth shapes, permissions, frontend, checklist — is in Backend Integration.

Endpoints

Paths below are relative to the prefix (/api/v1 for the standalone server).

GET /health

Always public. Returns {"status": "healthy", "service": "mango"}.

POST /ask/stream

Request

{ "question": "How many users signed up last month?", "session_id": "optional" }

Response — text/event-stream, one JSON object per data: line:

data: {"type": "session", "session_id": "3b5af53a…"}
data: {"type": "tool_call", "tool_name": "describe_collection", "tool_args": {"collection": "users"}}
data: {"type": "tool_result", "tool_name": "describe_collection", "success": true, "preview": "{…}"}
data: {"type": "tool_call", "tool_name": "run_mql", "tool_args": {"operation": "count", "collection": "users", "filter": {…}}}
data: {"type": "tool_result", "tool_name": "run_mql", "success": true, "preview": "{\"rows\":[{\"count\":1247}],\"row_count\":1}"}
data: {"type": "answer", "text": "1,247 users signed up last month."}
data: {"type": "done", "iterations": 3, "input_tokens": 4120, "output_tokens": 94, "memory_hits": 1, "tool_calls_made": ["describe_collection", "run_mql"], "retries_made": 0}

Status codes

Requests on the same session are serialised; different sessions never share history.

POST /memory/train

Adds a verified training example. Body: {"question", "tool_name", "tool_args", "result_summary"}. Gated through the agent's before_tool hooks as memory_train — deny it per role with DenyTools.

GET /memory/export · POST /memory/import

Export all memory entries / import a list in the export format (max 1 000 per request → 413 above that). Gated as memory_export / memory_import. Both answer 503 when the agent has no memory service.

Configuration by environment

Consuming SSE in JavaScript

async function* sse(response) {
  const reader = response.body.getReader()
  const decoder = new TextDecoder()
  let buffer = ''
  while (true) {
    const { done, value } = await reader.read()
    if (done) return
    buffer += decoder.decode(value, { stream: true })
    const lines = buffer.split('\n')
    buffer = lines.pop() ?? ''
    for (const line of lines) if (line.startsWith('data: ')) yield JSON.parse(line.slice(6))
  }
}

const res = await fetch('/api/v1/ask/stream', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${token}` },
  body: JSON.stringify({ question: 'How many orders last week?', session_id }),
})
if (!res.ok) throw new Error((await res.json()).detail)
for await (const event of sse(res)) {
  if (event.type === 'session') session_id = event.session_id
  if (event.type === 'answer') console.log(event.text)
}

Known limitation

LLM calls are synchronous inside the async server: while one turn is waiting for the model, other requests on the same worker wait too. Run several uvicorn workers for concurrent users; an async path is planned.