Engineering · February 11, 2026 · 8 min read
The AI features that survived contact with our clients' users
We shipped LLM features into six client products last year. Half got removed within a quarter. What separated the keepers from the demos.
Last year we integrated LLM-powered features into six client products. Three are now load-bearing; three were quietly removed within a quarter. The difference was never model quality — it was where the feature sat in the workflow.
What survived
- Drafting inside an existing task: summaries, first-pass responses, extraction from documents users already had to read
- Search that answers instead of listing — grounded in the client's own data, with citations
- Classification that saves a human decision per row, thousands of rows a day
What got removed
Chatbots bolted onto products whose users had no question to ask; 'AI insights' panels that restated dashboards in longer sentences; anything where a wrong answer cost more than the typing it saved. The pattern: features that added a conversation where users wanted a button.
The engineering that mattered
Evals before launch, fallbacks for every model call, cost ceilings per tenant, and honest latency budgets. LLM calls are the new third-party script: they need the same discipline as a payment gateway, not the same optimism as a landing page.
Our rule now: AI earns a place in the product when removing it would make a daily task slower. Demos don't count.