In early 2025 we made a call that most startups never make voluntarily: we stopped shipping features for four months and rebuilt the entire product from scratch. This is the story of why we did it, what we changed, and what it cost us.
The original AIVA was built fast. We had a working voice agent in six weeks and customers in production in three months. The code reflected that.
We'd glued together four third-party services:
- a transcription API
- a language model
- a TTS engine
- Twilio
With a thin orchestration layer held together mostly by optimism.
It worked. Until it didn't.
The performance ceiling
By mid-2025 we had 80 customers and were hitting walls we'd built ourselves. Voice latency was stubbornly stuck at 380ms average. Adding a new language meant touching five configuration files and praying nothing broke.
Our infrastructure bill was growing faster than our revenue, because we were paying three vendors to do what one well-designed system could do. And on-call was a nightmare — when something failed at 2am it was never obvious which of four vendors was the culprit.
The performance ceiling was the most visible problem. 380ms sounds fast, but in voice it isn't. Human conversation flows at under 200ms of response lag. Anything above 250ms starts to feel like the call is dropping.
We were losing customers not because AIVA gave wrong answers, but because it felt slow — and in voice, feeling slow and being slow are the same thing.
How we knew it was architecture, not code
This is the part worth being precise about, because "rewrite it" is usually the wrong answer. The standard advice — most memorably Joel Spolsky's argument that rewrites are the single worst strategic mistake a software company can make — is right far more often than it's wrong.
Rewrites throw away years of accumulated bug fixes in exchange for the fantasy of a clean slate.
What made our case different was that we could show the ceiling arithmetically rather than argue about it aesthetically.
We profiled every stage and added up the best possible case: if every vendor responded at their documented p50, and every network hop ran at the theoretical minimum for the distance involved, the chain still landed around 300ms. Our requirement was 200ms.
That's the test. Not "the code is ugly" or "we'd design it differently now" — those are always true and never sufficient. The test is whether the best-case version of what you have still misses the requirement. If it does, you're not optimising, you're negotiating with a constraint that won't move.
What we rebuilt
The new stack collapses everything into a single inference pipeline we own end to end. Speech recognition, language understanding, response generation, and synthesis all run in one unified process instead of four chained API calls.
The round trip that used to cross four network boundaries now crosses zero. The engineering detail of that collapse is in the voice pipeline rewrite post; the subsequent millisecond-level tuning is in how we cut latency from 240ms to 198ms.
We also moved to regional deployment. The original AIVA ran from a single region in Mumbai. The rebuilt version runs in Mumbai, Frankfurt, and Virginia, with automatic routing to the closest region based on the caller's network path — the reasoning behind those three specific locations is in our regional inference write-up.
European and North American deployments now see latencies under 180ms; Mumbai customers see under 160ms.
The language pipeline is the part I'm most proud of. In AIVA 1.0, adding a language was a project. In AIVA 2.0, it's a config file and a model checkpoint. We've shipped six new languages since the rebuild launched — each in under two weeks, against a six-week minimum in the old system.
That capability is what made twelve Indian languages practical rather than aspirational; the current list lives on the languages page.
What we got right about the process
Two decisions mattered more than any technical choice.
We ran in parallel for six weeks before cutover. This felt slower — we wanted to build the new thing and switch. The parallel run found eleven bugs that would have been production incidents on day one.
Every one of them was a case where real caller behaviour differed from our test fixtures in a way no amount of unit testing would have surfaced.
We didn't touch the API contract. External callers — the webhook system, the dashboard, customer integrations — saw no change at all. The rebuild was internal infrastructure.
Nobody outside the engineering team had to do anything when it shipped, which meant we could cut over on a Tuesday morning rather than coordinating a migration with 80 customers.
The lesson we'd pass on: if you're hitting performance ceilings in month 12, the ceiling is probably architectural, not algorithmic. You can't optimise your way out of a bad architecture. But prove it with arithmetic before you commit — "we'd build it differently now" is not the same finding as "the best case still misses."
What we deliberately didn't rebuild
A rewrite's real risk isn't the part you replace. It's the scope creep that arrives once everyone accepts that rewriting is allowed — at which point every piece of code anyone has ever disliked becomes a candidate, and four months becomes fourteen.
We drew the line using the same test that justified the project: rebuild only what the latency arithmetic implicated. That left three sizeable systems untouched, all of which someone wanted to rewrite.
The dashboard. Dated, inconsistent in places, and completely irrelevant to call latency. It stayed exactly as it was and got its own redesign a year later, on its own schedule, with nobody's phone calls at stake.
The billing system. Nobody enjoys working in it. It also handles money correctly, which is a property that takes years to earn and an afternoon to lose. Rewriting a correct billing system during an infrastructure rewrite is how you turn one risky project into two.
The webhook delivery layer. This was the closest call, because it genuinely had problems. But it sat outside the media path, so its problems cost reliability rather than milliseconds — a different project, correctly scheduled later.
The discipline was unpopular in the moment and is the main reason the four months stayed four months. If a rewrite's scope is defined by "things that are bad," it has no boundary. If it's defined by "things the measurement implicates," it has one.
What it cost us
Four months of no new features. Two customers who couldn't wait and churned. A team that was exhausted by the end. And a difficult conversation with early investors who wanted to see the metrics moving.
That last one deserves a footnote: being bootstrapped is what made the decision possible at all. A four-month feature freeze is a hard sell to a board optimising for quarterly growth. It's a much easier conversation when the only people you have to convince are your customers and yourselves.
We don't regret any of it. Every meaningful thing we've shipped since — analytics, the new language pipeline, three regions — was possible because of the 2.0 infrastructure. The rebuild was a multiplier on every feature that came after it.
The voice latency numbers today: 198ms average, 260ms p95. We shipped our first customer over 1 million calls last month with zero outages. That's what the rebuild bought us.
You can see how the current stack behaves on the voice platform page, watch live call quality in analytics, or start free with ₹500 of credit and judge the latency yourself.