Concepts

How replicate-mcp-agents is built

A tour of the framework's layers, the path a single request takes through them, and the design decisions that keep 15 subsystems from collapsing into one tangle — so you can predict how the system behaves before you depend on it.

This page is for engineers evaluating or adopting the framework. You will learn how the codebase is layered, what happens between calling an agent and getting a result, and how the exception hierarchy doubles as the retry contract. No code is required to follow along; every class named here is covered with exact signatures in the API reference.

The hexagonal layers

The codebase follows a hexagonal layering — sometimes called ports-and-adapters — meaning the domain model sits at the centre with no knowledge of transports, telemetry, or the network, and each outer layer depends only on the layers beneath it. Dependencies point strictly inward:

sdk.py                                          SDK facade
  @agent decorator, AgentBuilder, AgentContext
        │ depends on
server.py · distributed.py · observability.py   Infrastructure
  MCP transports, worker nodes, OTEL export
        │ depends on
routing.py · resilience.py · qos.py             Application
  bandit routers, circuit breakers, QoS tiers
        │ depends on
registry.py · exceptions.py                     Core domain
  AgentRegistry, AgentMetadata, error types — zero external deps

The core domain (registry.py, exceptions.py) has zero external dependencies — it is plain dataclasses and exception types. The application layer (routing.py, resilience.py, qos.py) implements the decision-making logic and depends only on core. The infrastructure layer (server.py, distributed.py, observability.py) wires those decisions to MCP transports, HTTP workers, and OpenTelemetry. The SDK facade (sdk.py) is the surface you import: the @agent decorator and builders, ergonomic and free of implementation leakage.

Dependency-graph analysis of the repository confirms no circular imports between layers. Optional dependencies — OpenTelemetry, YAML — are lazy-loaded inside functions, so the core install stays light and an agent process never pays for an integration it does not use.

The life of a request

If you remember one thing from this page, make it this walkthrough. Think of the framework as an assembly line for a single model call: each station does one job, and each station can be tuned or swapped without touching the others.

It starts at registration. Decorating a function with @agent (or using AgentBuilder) produces an AgentMetadata record — name, model, schema, cost and latency estimates, tags — stored in the AgentRegistry with O(1) lookup and duplicate detection. When a call arrives, a QoS policy first removes candidate models that violate hard SLA caps (latency, cost, quality floors — see QoS tiers). The bandit router then picks one model from the survivors based on learned statistics. Execution is wrapped in the resilience stack: the call acquires a rate-limit token, passes a per-model circuit breaker check, and runs on Replicate inside a retry loop with jittered back-off. On success, results stream back to the caller; the chunks are stored in the opt-in result cache, the audit logger records the invocation, the OpenTelemetry span closes with latency metrics, and record_outcome() feeds the actual latency, cost, and success back into the router — so the next call is routed a little better than this one.

Exceptions as the retry contract

The exception hierarchy in src/replicate_mcp/exceptions.py is not just error reporting — it is the machine-readable contract that tells the resilience layer what is safe to retry. Every framework error subclasses ReplicateMCPError, so one except clause catches anything framework-originated. Beneath the base sit two classifying branches:

The classifier is_retryable_error() (in resilience.py) applies three rules in order: a NonRetryableError is never retried, even if its type appears in the configured retryable tuple; a RetryableError is always retried, even if it does not; anything else falls through to the RetryConfig.retryable_exceptions tuple. CircuitOpenError also lives in resilience.py: when a model's breaker is open, the executor does not retry at all — it records the trip and fails fast with an error chunk, protecting the failing model from further load. Domain errors such as AgentNotFoundError, DuplicateAgentError, and CycleDetectedError sit directly under the base and signal programming mistakes rather than runtime conditions.

Key design decisions

Four decisions, recorded in the README and the architecture decision records under docs/adr/ in the repository, explain most of what you will notice using the framework:

• MCP-first, not HTTP-first. The framework speaks MCP natively; the HTTP and SSE servers are adapters around that core abstraction, not the other way round. This is why agents appear as tools in Claude Desktop with no extra wiring.
• Decorator and builder converge on AgentMetadata. The @agent decorator is a convenience side-effect; AgentBuilder gives programmatic control. Both produce the same data structure, so nothing downstream cares which you used.
• One circuit breaker per model. A flaky model trips its own breaker without affecting siblings, and the half-open probe detects recovery quickly (ADR-004).
• Plugin hooks over inheritance. Extension points are lifecycle hooks (on_agent_run, on_agent_result, on_error) rather than subclassing, keeping the core execution path free of user code (ADR-007).

Next steps

• Quickstart — if you have not run a call yet, do that first; the lifecycle above is what your call traversed.
• Routing and QoS — go one layer deeper into how the bandit router and SLA tiers choose a model.
• API reference — exact signatures and defaults for every class named on this page.