Skip to content

replicate-mcp-agents v0.8.0

Orchestrate 50,000+ Replicate models like one system

replicate-mcp-agents is a Python framework that exposes Replicate-backed agents as native MCP tools and wraps every call in a production layer: learning bandit routers, per-model circuit breakers, QoS tiers, distributed workers, and OpenTelemetry observability.

Start in 10 minutes How it works →
  • 1,150 tests · ~90% coverage
  • 3 MCP transports
  • 15 stable subsystems
  • Python 3.10+ · Apache-2.0

Find your path

Quickstart

10 min

pip install, set your Replicate token, register an agent with one decorator, and make a routed, circuit-protected call — with replicate-agent doctor verifying every step.

Serve over MCP

stdio · SSE · HTTP

Register the server with Claude Desktop or Cursor and your agents appear as tools — zero extra wiring.

Routing & QoS

UCB1 + Thompson

How the router learns cost, latency, and quality per model — and enforces hard SLA tiers before it picks.

Agents & workflows

DAG fan-out

The @agent decorator, fluent builders, and YAML pipelines with parallel steps and safe conditions.

Distributed execution

port 7999

HTTP worker nodes with health checks, least-loaded dispatch, and worker-level circuit breakers.

Troubleshooting

runbook

Symptom → cause → fix for the top failure modes, and the published SLOs they protect.

How it works

Every call moves through three stages. The framework's value is that each stage is independently tuned, observed, and isolated — so one slow or failing model never takes your pipeline down with it.

  1. Register

    Decorate a function with @agent(model="meta/meta-llama-3-8b-instruct") or use AgentBuilder. The agent's metadata — cost, latency, tags, schema — becomes an MCP tool definition.

  2. Route

    A QoS policy filters out models that violate hard caps, then the bandit router (UCB1 for the first 20 calls, Thompson Sampling after) picks the best cost/latency/quality trade-off.

  3. Execute & learn

    The call runs inside a per-model circuit breaker with jittered retries, results stream back (and into the cache), and the recorded outcome updates the router's posterior for next time.