Back to all posts

Every feature in our changelog we almost didn't ship.

Escalation rules, twelve languages at once, even the changelog page itself — features that almost didn't happen. A running account of the close calls.

PS
Priya Sharma
Co-founder

Our changelog looks, from the outside, like a tidy list of things we decided to build and then built. It wasn't tidy. At least five things on it almost didn't happen — killed in a planning meeting, resurrected a week later, or shipped in a form nobody originally proposed. I run product decisions at AIVA, and I've started keeping a private note of the ones that were genuinely close calls, because the official changelog only records what we shipped, not what we almost didn't, and not how differently some of it could have turned out.

The changelog itself

The most meta example: we almost didn't build a public changelog at all. The argument against it was reasonable — publishing a running list of what's new is also publishing a running list of what wasn't there last month, which is an odd thing to hand competitors and skeptical prospects for free. A prospect doing diligence could read our changelog from six months ago and see, in black and white, exactly what AIVA couldn't do yet at the time they were evaluating it.

We shipped it anyway because the alternative — customers finding out about improvements by accident, or not at all — was worse. A bootstrapped company without a marketing department needs a place to say "here's what changed and why," even if it doubles as an admission that things weren't finished before. It also turned out to do something we hadn't fully anticipated: existing customers started checking it before opening a support ticket, because half the "is this possible" questions we used to get were already answered by something we'd shipped the previous month. Two years in, it's one of the most-read pages on the site, and the transparency cost we worried about never really showed up in the way we expected it to.

Configurable escalation rules

When we first built human handoff, the plan was a single fixed set of escalation rules baked into the product — the same triggers for every customer, no configuration screen, because a configuration screen is more surface area to build, document, and support. The three core triggers — genuine emotional distress, anything legal or compliance-adjacent, and an explicit request for a human — were never going to be optional. Those aren't negotiable regardless of industry. What we assumed would also be fixed was the sensitivity underneath them: exactly how upset is "upset enough," exactly which phrases and patterns count.

We reversed that after our first healthcare pilot, where "escalate on emotional distress" needed a materially different threshold than it would for a retail business — a patient describing symptoms in a worried tone isn't the same signal as a customer annoyed about a delivery delay, and treating them identically meant either escalating too eagerly in healthcare or not eagerly enough in retail. Making that threshold configurable, rather than a single number everyone lived with, added real engineering time to a feature that was two weeks from done and already felt finished. We shipped it anyway, later, because a fixed rule set that's wrong for half your customers isn't actually simpler. It's just broken in a way you find out about after launch instead of before, from a customer who noticed before you did.

Twelve languages on day one, not day ninety

The plan, for a long time, was English and Hindi first, with the other ten languages added over the following year as demand justified each one — the conventional, lower-risk way to sequence a hard localization problem. We changed course a few months before launch and shipped all twelve at once.

This wasn't confidence — it was risk we argued about for weeks. Shipping twelve languages simultaneously meant twelve chances for something to sound wrong, instead of two. We decided the reputational cost of "AIVA doesn't really speak Marathi yet" was worse than the engineering cost of getting all twelve right before anyone could compare them.

What actually tipped the decision wasn't a single argument winning — it was realizing that a staggered rollout would mean fielding the same disappointing conversation over and over, once for every language not yet live: a prospect in Kochi asking about Malayalam, being told "not yet," and either waiting or going elsewhere. Twelve at once meant twelve times the testing burden right before launch, and a launch date that slipped by several weeks while we got the last few languages past the same bar as the first two. It also meant that from day one, nobody could point at AIVA and say it only really worked in the two biggest languages while treating the rest as an afterthought.

Letting customers interrupt AIVA mid-sentence

This one almost got cut for being "a nice-to-have, not a requirement." Early voice demos had AIVA finish its sentence no matter what — fine in a demo, maddening in a real call, when a customer just wants to correct themselves or jump ahead. Handling interruption cleanly — telling a correction apart from a topic change, and either apart from someone just pausing mid-sentence to think — turned out to be fiddly enough that it was proposed for "v2" more than once in planning meetings, usually by whoever was under the most pressure to hit a launch date.

It stayed in scope because a handful of us tested the alternative on ourselves — calling in, trying to interrupt, getting talked over — and it was unpleasant enough in ten seconds that shipping without it felt indefensible once we'd felt it firsthand. That's become something of a pattern for us since: when a feature's value is hard to argue in a planning meeting, the fastest way to settle the argument is to make everyone in the room experience the alternative themselves rather than describe it.

Smart triggers on the widget almost weren't smart at all

The fifth close call is smaller but still real. The original web widget spec called for the simplest possible trigger: show the chat bubble the instant a visitor's page finishes loading. It was the cheapest version to build, and for a few weeks it was the plan we were building toward.

We tested it on our own site before shipping it anywhere else, and it felt exactly like what it was — a chat window jumping in front of a visitor before they'd had a chance to read a single word of the page. We rebuilt it around actual visitor signals instead — time on page, how far someone's scrolled, whether they're a return visitor — so the widget shows up when someone's plausibly ready for it, not the instant they arrive. It was a smaller re-scope than the escalation-rules or twelve-languages decisions, but it's the one that taught the team fastest that "technically working" and "actually good to be on the receiving end of" are two different bars, and we should default to testing against the second one earlier next time.

Why I keep this list

None of these features look contested now. They're just in the product, described in two lines on the changelog page, looking inevitable. They weren't. Almost everything we've built well was, at some point, one meeting away from being cut, delayed, or shipped worse than it ended up. The changelog records the decisions. It doesn't record how close some of them came to going the other way, or how many times the same argument had to happen before it stuck.

I keep the private list mostly for myself, as a check against my own instinct to trust the tidy version of our own history — the same instinct behind why confident-sounding claims are so tempting in AI marketing generally, and why we try to resist them about our own product specifically. It's also useful whenever we're debating a new feature and someone says "let's just ship the simple version for now" — that's exactly the sentence that preceded four of the five features on this list, right before we decided the simple version wasn't actually good enough to ship.

Share
PS
Written by
Priya Sharma
Co-founder

FAQ

Common questions.

No — several launched in a different form than first proposed, or almost got cut entirely. Configurable escalation rules, launching twelve languages simultaneously, mid-sentence interruption handling, and the public changelog itself were all genuinely close calls.

We debated it, because a running list of what's new also advertises what wasn't there last month. We shipped it anyway because customers finding out about improvements by accident, or not at all, felt worse than the risk of showing our work.

The original plan was a single fixed rule set for everyone. A healthcare pilot needed a materially different threshold for what counts as emotional distress than a retail business would, so we added configuration — late, and at real engineering cost — rather than ship a rule set that was wrong for half our customers.

The original plan was English and Hindi first, with the rest added over the following year. We reversed course months before launch because we decided the reputational risk of any language sounding half-finished was worse than the engineering risk of getting all twelve right before anyone could compare them.

Yes — AIVA listens continuously, including while it's speaking, and handles a correction, a topic change, or a mid-sentence pause differently depending on which one it actually is. This was nearly deferred to a later release as a nice-to-have before we tested the alternative on ourselves.

It waits for signals that someone actually wants help — time on page, scroll depth, a return visit — rather than popping up the instant a page loads. The simpler, instant-popup version was the original default and almost shipped that way.

Roughly: if the harder version measurably changes how a real customer experiences AIVA — not just how the feature looks in a demo — it's worth the extra engineering time. If it doesn't, we let it stay cut and spend the time elsewhere.

Like this? Get more.

One email a month. Engineering deep-dives, product launches, customer stories. No fluff.

4,200+ subscribers. Unsubscribe anytime.