Beyond What You Said: Building Giantwave

Speech-to-text has gotten remarkably good at one thing: turning sound into words. What it still almost entirely ignores is how those words were said.

Two people can say the exact same sentence and mean something completely different — one flat and exhausted, one sharp and certain, one hesitant and unsure. A transcript that only captures the words throws all of that away. For a lot of applications — coaching, accessibility, communication training, customer experience — the words were never the whole story. The delivery was carrying real information the entire time, and nothing downstream of a standard transcript ever sees it.

That gap is what led me to start building Giantwave: a speech pipeline that doesn't just transcribe what was said, but extracts a structured signal of how it was said — in real time, aligned word by word with the transcript itself.

The core idea

Giantwave listens to speech the way a transcription engine does, but in parallel, it's also reading a second layer of signal — one rooted in the acoustic and delivery properties of the voice itself, not in guessing at "emotion" the way a black-box classifier might. The output isn't a single mood label slapped on a sentence; it's a small set of continuous, interpretable dimensions attached to every individual word, each with its own confidence score.

That per-word, per-dimension granularity is the part I care most about. A single "sentiment score" for a whole utterance flattens exactly the kind of nuance that matters — a moment of hesitation on one word, a burst of intensity on another. Giantwave is built to preserve that texture instead of averaging it away.

Just as important: none of this comes at the cost of speed. The transcript still appears in real time, at the latency people already expect from live captioning. The delivery signal rides alongside it rather than making it wait.

Why this matters

A few beliefs shaped the direction of this project:

• Delivery is information, not noise. Tone, pacing, and vocal effort carry meaning that word choice alone doesn't capture — and that meaning is currently thrown away by nearly every transcription tool.

• Interpretability matters more than a single confident-sounding score. A system that can say "I'm not sure" on a hard case is more trustworthy than one that always outputs a number with false confidence.

• Real-time or it doesn't count. A lot of the use cases I'm most interested in — live coaching feedback, accessibility tools, in-the-moment communication support — fall apart if the signal arrives seconds late.

Where this is headed

The current build is a working, end-to-end demo: live capture, live transcription, and a live delivery-signal readout, all in one interface. From here, the plan is to validate the signal against more real-world speech, and explore which applications benefit most from having this layer available — coaching and training tools, accessibility applications, and communication-focused products are the early front-runners.

I'm always glad to talk through the thinking in more depth — just reach out.