Testing Voice Apps – Emerson Sklar, Bespoken – Voice Tech Podcast ep.062

Emerson Sklar Bespoken

Episode description

Emerson Sklar is the Chief Evangelist at Bespoken, a company with a popular toolkit for testing and monitoring voice apps. We start by unpacking the fundamentals of testing and the various types, from unit and end-to-end testing to continuous and usability performance testing.

Emerson talks us through the product features of Bespoken and how it works. We discover the surprising fact that their system uses text-to-speech to generate audio, which is then fed to a virtual Alexa on the servers to generate a response. They are thus able to modify the input in order to measure what effect that has on the output.

Tuning in, listeners will also learn from case studies involving The Mars Agency and Mercedes, we cover the vital elements of a testing strategy and how it should be approached, and then we look at ways to better understand voice app users through monitoring and analytics.

We have included a bonus section at the end comprising a run-down of all the tools that we can recommend for the various stages of the voice development process, including prototyping, skill building, coding frameworks, testing, and analytics.


  • Emerson’s background doing test automation for data analytics applications for the US army.
  • Joining Bespoken to help companies around the globe to implement automation tools.
  • The value proposition of Bespoken as the only recognized voice automation company.
  • A look at the virtual devices that enable them to optimize the voice experiences developed by their clients.
  • How using cloud implementations of voice assistants instead of physical devices saves time and money.
  • Simulating the acoustic interference that one might expect in the context of device usage.
  • A breakdown of the testing process: from sending the audio to the voice platform to analyzing the response.
  • How the free component of the Bespoken suite also allows access to independent developers.
  • The three main challenges in the world of voice and how they address each.
  • The nature of the free and open-source unit testing provided by Bespoken.
  • End-to-end testing: functional or regression testing for the voice app itself and its ecosystem.
  • The accuracy of text-to-speech engines in terms of how closely it resembles human speech.
  • Ensuring that voice apps continue to perform correctly by prioritizing continuous testing.
  • The need for usability performance testing – ensuring that users have a good experience.
  • Helping Mercedes to transition from manual to automated testing.
  • How the characteristics of a person’s voice determine their experience of a voice app.
  • The benefits of testing from the earliest stages of development rather than afterward.
  • A list of Emerson’s preferred prototyping tools, including using pen and paper.
  • Top skill builders to check out e.g. Voiceflow
  • Why Jovo’s approach to writing code fits so well into the Bespoken framework.
  • What analytics tools like Dashbot allow you to track: user numbers, retention, engagement.
  • Thoughts on whether automated testing can help with the problem of discoverability.

Quotes from the show

Add quotes here in the following format:

[05:53] At a high level we are the leaders of voice automation

[32:58] What you experience is going to be dependent on the characteristics of your voice

[49:08] Peru has a number of fantastic engineering schools

Links from the show

About the author

Carl Robinson
Carl Robinson

Carl is the host of the Voice Tech Podcast. Since launching in April 2018, Carl has conducted scores of in-depth interviews with voice industry experts, building one of the most well known media brands in the voice AI space.

He also publishes Voice Chops Tuesday, the number 1 voice technology newsletter, enjoyed by thousands of voice tech fans each week.

Carl is a startup founder, product manager and data scientist, and recently presented a model for voice emotion conversion at ICASSP 2019.

To contact Carl, use the links in this box to send an email, tweet or message.

Share this article

If you found value in this article, please consider helping others by sharing to your network. Just click one of the links below.

What do you think?

Related Posts

Pexels Alex Green 5699456 Petr Marek
Muddu Sudhakar Aisera
Marco Liuni Alta Voce

Get notified about new articles

[yikes-mailchimp form="2"]

Upcoming Events

Featured Products