Benchmarking Accuracy

Try to use the principle that It’s all about the data and the volumes of data that you are analyzing/assessing when assessing conversation analytics accuracy. For most people who have not been exposed to conversation analytics before, they expect to see the platform transcribing everything accurately and consistently and accurately.

Customers can achieve good outcomes utilizing conversation analytics with even 50% accuracy. According to Maintrax, leading independent thought leaders globally on speech analytics, the accuracy rate of a very, very good transcription technology platform today would be between 80 to 85 percent transcription accuracy. This means of a single call, 80 to 85 percent of the words are transcribed accurately

What are you really measuring?

Look to understand what you are measuring when observing and assessing transcripts.

Transcription is a huge step forward in your data discovery journey. The more important aspect is knowing what’s in the calls, what’s detectable, and the level of confidence you need to make business changes.

Instead of thinking you need to be achieving one hundred percent accuracy (which is just not realistic), reflect on what you really want to uncover in the recordings (e.g. angry customers, cancellations, non complying agents etc.) and how your conversation technology can accurately identify these triggers and events.

Focus focus focus?

We recommend to not “get caught up in the weeds” in that if you are down at an individual transcription level you will always see mis-transcription (remember very, very good engines will still see you with 1 in 5 words not quite right). It’s all about the data and the volumes of data not whether the platform got every word right. Think statistically valid data.

Audio quality will always create a variable in transcription accuracy

With poor quality audio you need to utilize a higher sampling rate and assess against those – with our data output, we will give you confidence scores for each words and the overall call which is easy to then filter.

We can also improve the accuracy for things like product names, industry terms or competitors names with further field tuning. What you see at first pass is usually the baseline. We can improve “accuracy” with “tuning” for environment and words/terms/phrases.

Best Practices to Maximizing Transcription Accuracy

The better quality the audio, the more information can be retrieved, and the higher accuracy of the resulting data output. Read our Audio FAQ to help you get started.

  • Volume of audio – minimum, 20 hours to give you a decent assessment
  • Quality of audio - different audio, accents etc. and audio source quality will always have a major impact on transcription accuracy (e.g. background noise, audio compression).
  • Stereo Recordings - being able to record and present the audio in stereo will help with transcription accuracy. For example, you may have one side of the conversation being had by an employee based in the Philippines with an American bias language and the other side of the conversation with an Australian based client with an Australian dialect. We can separate both channels and apply different language models to improve the level of accuracy.

Putting it in perspective

The way to think of accuracy is to ask what error rate is tolerable for your needs. For example:

  • You can have good success improving CSAT with solutions that were on the low side of accuracy.
  • For measuring script adherence and scoring agents, you will need to seek higher accuracy
  • When you want to uncover high value phrases ( compliance violations) then lower accuracy is OK.

If your end goals is to identify root cause of churn, a transcription that captures every pause, repetition, stutter, and background noise may be of limited value.

Call Journey transcription accuracy

  • 80 – 85 percent transcription accuracy "untuned" and out of the box
  • World class speech environment. We are Global Speech Analytics partner of Microsoft, Genesys and SAS
  • With Call Journey data output, you will get confidence scores for each words and the overall call. We can also improve the accuracy for things like product names, industry terms or competitors names with field tuning.
  • Diarization – we can split mono recordings to ascertain caller and agent
  • The platform is very malleable, post implementation tuning can be delivered at reasonable cost.

“We at Maintrax have been involved in over 300 different speech analytics engagements over the last decade and have worked with many key speech technologies.

We are big fans of Call Journey’s platform and have been watching them with interest over the last couple of years.

From our perspective, in comparison to various platforms , Call Journey’s VoiceAI Ecosystem is very accurate and their architecture drives both speed and adaptability for integration.”

Scott Bakken, Co-Founder, Maintrax (Speech Analytics Specialists)