Retrofitting AI into legacy enterprise systems without a rewrite

The pitch decks make it sound like a choice between two futures: keep the twelve-year-old platform and fall behind, or fund a rewrite and emerge modern. In practice almost every enterprise we work with takes a third path, because the rewrite is unaffordable and the platform is load-bearing. The good news is that the third path is usually the better engineering decision anyway.

The pattern is simple to state and hard to resist over-engineering: leave the system of record alone, and put the intelligence beside it.

Start with the decision, not the data

Every worthwhile machine learning project inside an enterprise begins with a decision someone is already making by hand — which claims to route for review, which shipments will miss their window, which invoices will be disputed. Find that decision, find the person making it, and find out what they look at. That conversation tells you more about feature engineering than a month in the data warehouse.

It also tells you whether the project is viable at all. If nobody currently records the outcome of the decision, you have no labels, and no labels means no supervised model. That is a data-capture project first and a machine learning project second — an unglamorous finding that saves six-figure disappointments.

The integration surface is the real work

Once the model exists, everything expensive happens at the boundary. The legacy system needs to emit an event or expose a read model; the prediction needs somewhere to live that is not a column bolted onto a table five other systems write to; and the human needs a way to disagree with the model that gets captured as training data rather than lost in a support ticket.

• Change data capture or an outbox pattern to get events out without touching write paths
• A separate inference store keyed to the source record — never a new column in the legacy schema
• An override path in the UI that logs the human decision and the reason
• A shadow-mode period where predictions are recorded but not acted on

Shadow mode is not optional

Run the model against live traffic for a full business cycle before it influences anything. Compare its output to what the humans actually did, then sit with the disagreements. On one logistics engagement, the model and the planners disagreed on 11% of route decisions — and in about two-thirds of those cases the model was right for reasons the planners could immediately articulate once they saw them. The remaining third exposed a constraint nobody had written down anywhere.

That third is the whole value of shadow mode. It converts institutional knowledge into features before the model ever gets to embarrass you in production.

What this costs

A wrapped-model engagement on an existing platform typically runs three to five months and needs one data engineer, one machine learning engineer and part of a backend engineer who knows the legacy system. That is materially less than a rewrite, and — the part nobody says out loud — it leaves you free to do the rewrite later on your own timetable, with a working model already decoupled from whatever you replace.