Inside Our Low-Latency Pipeline

When people watch live captions, they almost never think about the path the words traveled to reach the screen. That is the point. A caption that lands while the sentence is still hanging in the air feels effortless. A caption that lands two seconds late feels broken, even when every word is correct.

Latency is the gap between speaking and seeing. It is the part of live transcription you feel before you read a single word. This is a plain-language look at why that gap matters, and at the general principles we use to keep it small. We keep the proprietary parts under the hood to ourselves.

Why a half-second changes everything

A meeting moves at the speed of conversation. People interrupt, agree, and pivot in real time. When captions lag behind that rhythm, they stop being live and start being a delayed transcript that happens to scroll.

The cost shows up in small ways that add up. A participant reading captions to follow a fast speaker falls a beat behind and loses the thread. Someone reading a live translation answers a question that already moved on. A presenter glancing at captions to check they were understood sees the wrong moment on screen. None of these are dramatic on their own, but together they break the feeling that the words are keeping pace with the room.

That is the bar we hold ourselves to. Captions should feel like they belong to the current sentence, not the one before it. Getting there is less about one clever trick and more about refusing to add delay at every step where adding it would be easy.

Stream the audio, do not wait for the file

The first principle is simple to say and easy to get wrong: never wait for the whole recording before you start transcribing.

A natural way to build transcription is to capture audio, save a file, and send it off to be processed. That works well for files you upload after the fact. It is the wrong shape for live captions, because the person watching has to wait for speech to finish before anything appears. The longer someone talks, the longer the wait.

Live transcription flips that around. Audio is streamed in a continuous flow, broken into small slices, and sent onward the moment each slice is ready. The transcription engine starts working on the opening words while the speaker is still mid-thought. There is no big pause at the end because there is no big file to wait on. The work has been happening the whole time.

Picture the difference between filling a bucket and then pouring it out, and running a steady trickle that never stops. The trickle is what makes captions feel live, and keeping it smooth and uninterrupted is most of the work.

Show your best guess, then refine it

The second principle is that a good live caption is allowed to change its mind.

Human speech is full of ambiguity that only resolves a few words later. The start of a sentence can point in several directions until the end arrives. If a transcription system waited for full certainty before showing anything, it would always be late, because certainty often comes after the speaker has moved on.

So instead of waiting, we show an early best guess and then refine it as more of the sentence arrives. Early words appear quickly as a tentative reading. As context fills in, those words firm up or get gently corrected into their final form. You may have seen a caption shift slightly a moment after it appeared. That is not a glitch. That is the system choosing to be fast first and exact a half-second later, which is almost always the right trade for a live audience.

The skill is in making those refinements feel calm rather than jumpy. Done well, the early text is close enough to be useful immediately, and the corrections are small enough that they never pull your attention away from the conversation.

Never lose a word when the network blinks

The last principle is about what happens when something goes wrong, because over a long session something always does. A laptop switches networks. A coffee-shop connection drops for a few seconds. A tunnel swallows a phone signal.

The easy failure here is the worst one: a gap in the connection becomes a gap in the transcript, and a stretch of conversation simply vanishes. For a live record people rely on, missing words are not a minor annoyance. They are a hole in the meeting.

Our approach treats interruptions as expected, not exceptional. When a connection wobbles, the audio captured during the rough patch is held safely rather than thrown away. When the link recovers, that audio is reconciled back into the right place in the timeline so the words land in order, and the live view catches back up instead of skipping ahead. From the reader's side, the captions briefly pause and then resume. Underneath, the system worked to make sure the pause did not cost anyone a sentence.

Low latency and reliability are usually framed as opposites, where speed means risk and safety means slowness. In live transcription they have to live together. The whole pipeline is built so that the fast path stays fast in the common case, and the careful path quietly protects the words when the network has a bad moment. When both hold, captions feel instant and complete at the same time, which is exactly how live should feel.