What we measured
A scripted line was read naturally over a real phone call and carried through the entire live path — the public phone network, compressed 8 kHz telephone audio over Twilio Media Streams, into our production speech-to-text. The transcript we scored is the exact one the AI acted on, not a separate clean recording.
Accuracy is reported as word error rate (WER): the word-level edit distance between what was said and what was transcribed, case-insensitive and with punctuation removed. Word accuracy is 100% minus the WER.
The call, word for word
“Hello Felix, the kitchen faucet in room two fourteen is dripping all night long. And the hallway light keeps flickering, and the closet door squeaks something awful.”
27 reference words
“Hello, Felix. The kitchen faucet in the Room 214 is dripping all night long. And the hallway light keeps flickering, and the closet door squeaks something awful.”
Only difference: “two fourteen” written as “214”
The numbers, both ways
The 11.1% strict word error rate is made up entirely of the transcriber writing the spoken room number as a numeral (and adding one article) — semantically faithful, just a different spelling. We quote the normalized figure as primary because a work order is built from meaning, not orthography.
The honest scope
We would rather under-claim than oversell a safety product. So, plainly:
- This is one call, one speaker, 27 reference words — a real, end-to-end measurement, not a large corpus study.
- The speaker was an adult tester, not an 80-to-95-year-old resident. So this is accuracy on a real call — not a claim about senior-speech accuracy.
- Senior-speech accuracy is the number that truly matters here, and it is the one we are working to measure and publish with design-partner communities, with consent.
Why this matters
An accurate work order starts with an accurate transcript: the right unit, the right problem, the right priority. In senior living the stakes are higher than a typo — a misheard room number on an urgent call is a real failure. Measuring on the actual phone path, and being candid about what we have and have not tested, is how we earn the trust a safety line requires.
Is this senior-speech accuracy?
No, and we will not claim it is. The speaker on this call was an adult tester, not an 80-to-95-year-old resident. Senior-speech accuracy is the measurement that actually matters for this product, and it is the number we are working to publish with design-partner communities, with consent. This figure is accuracy on a real call, stated honestly as exactly that.
How is word accuracy calculated?
Word error rate (WER) is the standard speech-recognition metric: the word-level edit distance between the reference (what was said) and the hypothesis (what was transcribed), case-insensitive and with punctuation stripped. Word accuracy is 100% minus the WER. We computed it with our own benchmark code on the exact transcript the AI acted on.
Why are there two numbers?
The strict verbatim WER is 11.1%, but every single edit was the transcriber rendering the spoken room number 'two fourteen' as the numeral '214' (plus one extra article) — semantically perfect, just a different spelling. Normalizing that numeral gives 3.8% WER (96.2% accuracy). We report both and quote the normalized figure as primary, because the work order is built from meaning, not spelling.
Why measure on a phone call instead of clean audio?
Because that is what residents use. The measurement runs through the full live path — a real phone call over the public network, compressed telephone audio, and our production transcription — so the number reflects what actually happens on the line, not a best case from a studio recording.