Reference

Python API reference

The public surface of replicate_mcp: every exported class and function, the constructor defaults that govern runtime behaviour, and the full exception hierarchy.

Everything documented here is re-exported from the top-level package, so a single import style covers the whole framework: from replicate_mcp import agent, CostAwareRouter, ResultCache. The package's __all__ is the contract — symbols outside it (such as AgentExecutor, TokenBucket, and the resilience helpers) are stable but imported from their submodules, and each is flagged below. Sections follow the request lifecycle: define agents, compose workflows, route, protect, cache, distribute, extend.

Agents and SDK

An agent is a plain function that builds a Replicate payload, plus metadata used for routing and MCP tool listings. The @agent decorator registers metadata as a side effect and returns the function unchanged; AgentBuilder does the same thing fluently for programmatic setup.

AgentBuilder(safe_name) chains .model(), .description(), .tag(*tags), .streaming(enabled=True), .estimated_cost(usd), .avg_latency(ms), and .input_schema(schema), then terminates with .build() (returns metadata without registering) or .register(registry=None) (builds and registers). Both produce an AgentMetadata dataclass with fields safe_name, description, input_schema, supports_streaming, model, estimated_cost, avg_latency_ms, and tags; replicate_model() falls back to safe_name when model is unset.

AgentRegistry (from replicate_mcp.agents.registry) is the thread-safe store behind the decorator: register() (raises DuplicateAgentError on conflict), register_or_update(), remove(), get(), has(), list_agents() (returns a copy), filter_by_tag(), count, and clear(). AgentContext is a context manager that swaps in an isolated registry and restores the previous one on exit — use it in tests so decorator registrations do not leak.

AgentExecutor (from replicate_mcp.agents.execution) runs registered agents with streaming, concurrency capping, per-model circuit breakers, and retries. Its run(agent_id, payload) method is an async generator of chunk dicts; resolve_model(agent_id) maps a short name to a model path, and circuit_breaker(model_id) returns the breaker guarding a model.

Workflows

A workflow is an ordered list of agent steps where each step's output feeds the next. WorkflowBuilder(name) chains .description(text) and .then(agent_name, input_map=None, condition=None), then .build() returns an immutable WorkflowSpec (fields name, description, steps; properties step_count and agent_names). Building with zero steps raises ReplicateMCPError.

Routing and QoS

The router learns per-model cost, latency, and quality through exponential moving averages and picks the best candidate per call. QoS policies act as a pre-filter: models that violate the SLA are removed before the selection strategy runs. See Routing and QoS for the full mental model.

CostAwareRouter(weights=None, strategy="thompson", ema_alpha=0.3) supports three strategies: "score" (deterministic weighted score, lower wins), "thompson" (the default — Beta Thompson Sampling on success/failure), and "thompson_multi" (Gaussian Thompson Sampling on a cost/latency/quality utility). Key methods: register_model(), select_model(candidates), select_model_explain(candidates) (returns a RoutingDecision with selected_model, strategy, and per-candidate scores), record_outcome(model, latency_ms=..., cost_usd=..., success=True, quality=1.0) (call this after every invocation so the router learns), stats(), leaderboard(), and dump_state()/load_state() for persistence via RouterStateManager.

QoSLevel defines three tiers and QoSPolicy.for_level(level) returns the default SLA caps for each. A policy's filter_candidates() never returns an empty set — if every model fails the SLA, the full candidate list is used (graceful degradation).

QoSPolicy fields: max_latency_ms, max_cost_usd, min_quality, min_success_rate (all optional; None means unconstrained) and level (default BALANCED). UCB1Router(exploration_c=1.0, weights=None) is a deterministic bandit that tries unvisited models first and adds select_model_with_policy(candidates, policy=...). AdaptiveRouter(explore_threshold=20, exploration_c=1.0, weights=None) uses UCB1 for the first 20 total invocations, then switches to Thompson Sampling; its active_strategy property reports which phase is live.

Resilience

Every Replicate call inside AgentExecutor runs behind a per-model circuit breaker with retry and jittered back-off. These classes live in replicate_mcp.resilience (only is_retryable_error is re-exported at the top level). Full tuning guidance is in Configure resilience and caching.

CircuitState has three values: CLOSED (calls flow), OPEN (calls raise CircuitOpenError immediately), and HALF_OPEN (limited probes allowed). RetryConfig defaults: max_retries=3, base_delay=0.5, max_delay=30.0, jitter_factor=0.25, and a retryable_exceptions tuple of (Exception,). Note that AgentExecutor builds its own RetryConfig from its max_retries=2 and retry_base=0.5 arguments, so the executor retries twice by default even though the standalone config defaults to three.

is_retryable_error(exc, config=None) classifies errors in strict order: NonRetryableError subclasses are never retried, RetryableError subclasses are always retried, and anything else falls through to the configured tuple. TokenBucket(rate, capacity) (from replicate_mcp.ratelimit) is the async rate limiter accepted by AgentExecutor: await acquire(tokens=1.0) blocks until capacity is available, try_acquire() is the non-blocking variant, and available_tokens reports the current level.

Cache and discovery

ResultCache is an in-memory, content-addressed cache keyed on a SHA-256 hash of (model, sorted-JSON payload). It is opt-in: pass it to AgentExecutor(cache=...) and identical requests within the TTL replay their chunk lists instantly. Constructor defaults: ttl_s=300.0, max_entries=500, policy=EvictionPolicy.LRU, background_eviction=False, background_interval_s=60.0. EvictionPolicy values: LRU (default), TTL (only evicts expired entries), FIFO, and LFU (reserved; currently falls back to FIFO). Introspection properties include size, hits, misses, hit_rate, evictions, and a combined stats dict.

ModelDiscovery(registry=..., config=...) pulls models from the Replicate catalog and merges them into a registry with register_or_update, so manual customisations survive refreshes. discover_and_register(...) is the one-shot wrapper, and start_background_refresh() runs the refresh loop on an interval.

Distributed execution

The distributed layer dispatches agent calls across worker nodes — in-process queues for a single machine, or HTTP workers across machines. DistributedExecutor(nodes=None, max_retries=2) is the coordinator: it is an async context manager, routes each task to the least-loaded healthy node (skipping remote nodes whose circuit breaker is OPEN), and fails over up to max_retries times. submit(agent_name, payload) returns an awaitable TaskHandle that resolves to a TaskResult (task_id, agent_name, node_id, chunks, status, error, elapsed_ms; status is one of PENDING, RUNNING, DONE, FAILED).

serve_worker launches a worker with uvicorn (requires the [http] extra) and is what replicate-agent workers start calls under the hood:

async def serve_worker(
    *,
    host: str = "0.0.0.0",
    port: int = 7999,
    api_token: str | None = None,      # falls back to REPLICATE_API_TOKEN
    node_id: str | None = None,
    log_level: str = "info",
    max_concurrency: int = 8,
    enable_circuit_breaker: bool = True,
    circuit_config: CircuitBreakerConfig | None = None,
) -> None

Plugins and integrations

Plugins are middleware around every agent invocation. Subclass BasePlugin, implement the abstract metadata property plus setup() and teardown(), and override any of three hooks: on_agent_run(agent_name, payload) (return a dict to replace the payload, or None to pass through), on_agent_result(agent_name, chunks, latency_ms) (return a list to replace the output chunks), and on_error(agent_name, error) (observational only — the exception always propagates). Hooks must not raise. PluginMetadata fields: name, version (default "0.0.1"), description, author, requires.

PluginRegistry manages lifecycle and dispatch: load(), load_many(), unload(name), unload_all(), get(), has(), names(), count, and the executor-facing dispatch_run() / dispatch_result() / dispatch_error(). load_plugins(extra_classes=None, skip_names=None) discovers plugins from the replicate_mcp.plugins entry-point group. Three guardrail plugins ship built in: PIIMaskPlugin (redacts SSNs, emails, phone numbers, and card-like sequences), ContentFilterPlugin(deny_list=None), and CostCapPlugin(per_invocation_cap=1.0, session_cap=10.0).

The Latitude integration (install pip install "replicate-mcp-agents[latitude]") exports LatitudeClient, LatitudeConfig, LatitudePrompt, LatitudeTrace, LatitudeEvalResult, LatitudePaymentRequiredError, and LatitudePlugin. The module-level boolean HAS_LATITUDE is True when the optional import succeeded — check it before touching the Latitude symbols. Configuration falls back to the LATITUDE_API_KEY and LATITUDE_PROJECT_SLUG environment variables.

Exceptions

Every framework error derives from ReplicateMCPError, so one except ReplicateMCPError clause catches anything the library raises. The classification pair RetryableError / NonRetryableError drives is_retryable_error. The hierarchy (modules other than replicate_mcp.exceptions noted in parentheses):

ReplicateMCPError
├── CycleDetectedError
├── NodeNotFoundError
├── WorkflowValidationError
├── ModelNotFoundError
├── ExecutionError
│   └── ExecutionTimeoutError
├── TokenNotSetError
├── DuplicateAgentError
├── AgentNotFoundError
├── TransformNotFoundError
├── ConditionNotFoundError
├── CheckpointCorruptedError
├── RetryableError
│   ├── RateLimitError
│   └── ServerError
├── NonRetryableError
│   ├── AuthenticationError
│   └── ClientError
├── CircuitOpenError          (resilience)
├── MaxRetriesExceededError   (resilience)
├── NodeOverloadError         (distributed)
├── NoHealthyNodesError       (distributed)
└── WorkerCircuitOpenError    (distributed)

Next steps

• CLI reference — the commands that drive these classes from the shell.
• Architecture — how the subsystems on this page fit together.
• Agents & workflows — task-oriented usage of the SDK surface.