Adding AI to an existing system without a rewrite

In short:A practical shape for introducing LLM features into a product that already works.

Most teams do not need a new architecture to use AI. They need one well-chosen seam in the system they already run. Here is the approach we use, and why it keeps risk low.

Start at the seam, not the core

Look for a place where a person currently reads, writes, sorts or summarizes something: support triage, drafting replies, extracting fields from documents, answering questions over internal data. These are natural seams. The existing system keeps doing what it does; the AI feature sits beside it.

Put the model behind your own interface

Wrap the model call in a small module with a fixed input and output contract. Then the provider, the prompt and the model version can change without touching the rest of the product. It also gives you one place to add logging, caching, retries and cost caps.

async function classifyTicket(ticket: Ticket): Promise<{ label: Label; confidence: number }>
// callers never know which model or prompt is behind this

Keep the first version read-only

Let the feature suggest, draft or answer, and let a person or existing code make the change. Write access comes later, once you have evidence about how often it is right.

Build the evaluation set before the prompt

Collect a few dozen real examples with the answer you would accept. Every change to the prompt, model or data is scored against them. This one habit prevents most of the quiet regressions that make AI features feel unreliable.

Ship behind a flag to a small group

Turn it on for a handful of users, watch real usage, and fix what surprises you. Add guardrails and spend caps before widening access, not after.

One shape this can take over a month

Week 1: pick one workflow, gather real examples, agree what 'good' means.
Week 2: prototype behind an interface and a feature flag.
Week 3: evaluation set, guardrails, fallbacks and cost caps.
Week 4: limited rollout, review real cases, decide what to widen.

That is a template, not a promise: the right timeline depends on your system and data. A short audit is a cheap way to find the best seam before committing to anything.