How we build agentic AI that survives production

01

Knowledge and ingestion

Turn the source material into something retrievable.

An agent is only as good as what it can look up. Content is chunked with overlap and embedded once at seed time, not at request time. The critical discipline is having a single source: the same files that render these pages are the files that seed the knowledge base, so the site and the agent cannot drift into telling a visitor two different stories.

• →One source of truth for content, consumed by both the site and the index
• →Chunking with overlap, so a retrieved fragment still carries its context
• →Re-seeding is a scripted step, not a manual copy-paste

02

Retrieval

Find the few passages that are actually relevant.

We run vectors in the same Postgres that holds the application data, via pgvector. That is a deliberate choice over a dedicated vector database: one datastore to operate, one backup, one connection pool, and joins between an embedding and the row it describes. A separate vector service earns its place at a scale most products never reach. Retrieval is best-effort by design — if the index is empty or embeddings fail, the agent still answers from its grounded persona instead of erroring at the visitor.

• →pgvector inside the application database — one system to run and back up
• →Provider-swappable: pgvector, Qdrant, Pinecone or Chroma behind one interface
• →Failure degrades to a grounded answer, never to a stack trace

03

Model gateway

One call site for every model, so the provider stays a config value.

Every model call in the platform goes through a single gateway interface. Nothing in the agent code names a vendor. That is what makes a provider change a configuration change rather than a refactor — and it is worth doing on day one, because the model you start on is rarely the model you finish on. The same seam makes it practical to run different models for different jobs, or to fail over.

• →OpenAI, Anthropic, Gemini, Bedrock, Vertex, Azure or LiteLLM behind one interface
• →The agent code names no vendor — the provider is an environment variable
• →Token usage is captured per call for cost attribution

04

Orchestration

Sequence the steps, and keep every one of them observable.

Agents inherit a base class that opens a trace, records the request, times each step and closes the span — including on the failure paths, which is where you actually need the data. Complexity is added only when the work needs it: a single-pass retrieve-and-answer flow stays a pipeline, and the graph engine and checkpointer come out when a workflow genuinely branches, loops or has to resume mid-run. Reaching for a graph framework to draw a straight line is cost with no return.

• →A shared base class — subclass it and observability is automatic
• →Streaming and non-streaming share one instrumentation path
• →Graph orchestration and state checkpointing available, used when the work branches

05

State and memory

Remember what happened between turns, and between runs.

Conversation history, run checkpoints and cached work all need somewhere to live, and the right somewhere differs by project: a shared Postgres for a modest deployment, Redis when the read path is hot, DynamoDB or MongoDB when that is what the client already operates. It sits behind one interface for the same reason the model does — this is a decision that gets revisited the moment the traffic profile changes, and revisiting it should cost a config change. Checkpointing is also what turns a long workflow into something that can resume, rather than a request that has to succeed in a single pass.

• →Postgres, Redis, MongoDB, DynamoDB or in-process, behind one interface
• →Conversation history and run state are separable — they need not share a store
• →Checkpointing lets a long or interrupted workflow resume instead of restarting

06

Tools and actions

Where the agent stops describing the work and does it.

This is the line between a chatbot and an agent, and the one worth engineering hardest. Every action is a typed tool with an explicit contract, and every external system sits behind a provider interface with a no-op implementation. That means the integration can be developed, demoed and tested without touching the customer's real calendar, ledger or system of record — and switching to the real one is configuration, not a rewrite.

• →Typed tools with explicit contracts, not free-form model output
• →Every external system behind a provider interface, with a no-op for development
• →Actions are enumerated — the agent cannot reach a system nobody wired up

07

Guardrails and the human boundary

Decide what the agent may assert, and where a person takes over.

The useful question is not whether a model can do something, but whether it should be the one doing it unsupervised. We draw that line explicitly and in code. Our own estimator is the example: it produces scope, phases and a timeline, and is prohibited from producing a price — pricing comes from a senior engineer who has read the brief. Answers stream token by token, so a constraint that is not in the prompt is a constraint that arrives on the visitor's screen before anything could strip it.

• →Constraints live in the prompt, because streamed output cannot be filtered after the fact
• →Judgement calls hand off to a person with the full context attached
• →The agent is instructed to refuse claims it cannot ground, and to say what it does not know

08

Observability

Know what the agent did, on the runs that went wrong.

An agent that cannot be inspected cannot be improved, and 'it gave a weird answer last Tuesday' is not a bug report. Tracing is built into the base class rather than sprinkled through the agents, so a new agent is instrumented the day it is written — spans per step, retrieval counts, token usage, latency and the failure path. The tracer is pluggable across Langfuse, LangSmith and OpenTelemetry, so traces land in whatever the client already runs.

• →Instrumentation in the base class — new agents are traced by default
• →Langfuse, LangSmith or OpenTelemetry behind one tracer interface
• →Retrieval counts, token usage and latency captured per run

09

Services and the edge

Authenticate, rate limit and persist — before anything reaches a model.

The agent service is never exposed to the internet. A backend-for-frontend sits in front of it: it verifies the caller's token, applies rate limits, writes the conversation down, and only then calls the agent over private networking with a service key. Cost control and abuse control belong here, in front of the expensive part — and this is also the seam where data the agent has no business seeing is stripped from the payload. Past that boundary the split is ordinary microservices: independently deployable services that own their data and scale on their own curve. Because everything the browser touches is HTTP and server-sent events, the frontend is your choice — Next.js, Vite and React, or a panel inside the app you already run.

• →The agent service has no public route — only the BFF can reach it
• →Token verification, rate limiting and persistence happen before the model call
• →Independently deployable services, each owning its data and scaling separately
• →An HTTP and SSE contract, so the frontend framework is your choice, not ours

10

Deployment and portability

Run the same architecture on whatever cloud you already pay for.

The services are stateless containers: configuration comes from the environment, health is an endpoint, and nothing reaches for a host-specific SDK. That is what makes the target a choice rather than a commitment — a cloud-native Kubernetes rollout on AWS, Azure or GCP, a managed container platform, or a single VM for a pilot, all from the same image. It is also why the parts an enterprise usually has to fix are the parts we keep behind seams: which cloud, which model, which identity provider, which observability stack. Right-sizing is part of the discipline — we run a cluster when the workload earns one, not by default.

• →Stateless containers, configuration from the environment, health checks as endpoints
• →Cloud-native Kubernetes on AWS, Azure or GCP — or managed hosting, from the same image
• →No host-specific SDK in application code, so the cloud stays a deployment decision
• →Migrations run as a pre-deploy step — a bad schema change stops the rollout

Agent runtime

Python + FastAPI

async services, streaming over SSE

Shared platform library

gateway, embeddings, vector store, state and tracing behind one set of interfaces

Pydantic

typed request, response and settings schemas

Models

OpenAI

the provider this site runs on — swappable at the gateway

Anthropic, Gemini, Bedrock, Vertex, Azure

first-class through the same gateway interface

LiteLLM

a bridge to everything else, when a client has an unusual requirement

text-embedding-3-small

1536-dimension embeddings, called as a hosted API

Data and state

Postgres

application data and vectors in one database

pgvector

similarity search next to the rows it describes

Qdrant, Pinecone, Chroma

alternative vector stores behind the same interface

Redis, MongoDB, DynamoDB

alternative state backends, chosen per workload

SQL migrations

forward-only, applied on deploy, one source of truth for schema

Frontend

Next.js (App Router)

what this site runs on — server-rendered, static where it can be

Vite and React

when a single-page app is the better fit

Your existing app

the agent API is HTTP and SSE — a panel is enough to start

TypeScript

end to end, including the content schemas

Identity

Supabase Auth

sign-in, sessions and email verification

Asymmetric JWTs

verified at the edge against a published key set

Service keys

how internal services trust each other on the private network

Your identity provider

the verification seam takes OIDC or SAML equally well

Operations

Docker

one image, the same locally and in production

Kubernetes

cloud-native rollout on AWS, Azure or GCP

Managed container hosting

the same image, when a cluster is not yet warranted

Langfuse / LangSmith / OpenTelemetry

tracing, wherever the client already collects it