Use custom vocabulary in Amazon Lex to enhance speech recognition

by
Amazon AWS

In our daily conversations, we come across new words or terms that we may not know. Perhaps these are related to a new domain that we’re just getting familiar with, and we pick these up as we understand more about the domain. For example, home loan terminology (“curtailment”), shortened words, (“refi”, “comps”), and acronyms (“HELOC”) are likely new to first-time home buyers. Similarly, brand names can be tricky when you first hear of them. For example, with the sentence “I’d like to get a Cyetus 7601”, unless you’re a coffee aficionado, you may not realize the speaker is talking about a coffee machine brand. Some of these words may not be even present in the dictionary. Once we understand their usage, meaning, and spelling, they become part of our vocabulary so we can have effective conversations.

Similarly, bots have to understand and add such words to their vocabulary. Starting today, you can give Amazon Lex additional information about how to process speech input by creating a custom vocabulary. You can create a list with domain-specific words or brand names for improved speech recognition and end-user experience.

In this post, we review the custom vocabulary capability and use it for home loan customer service transactions and queries.

Overview of the custom vocabulary capability

You define the custom vocabulary for a language in the bot. In this case, we add brand names and domain-specific words and phrases in English (GB). The bot uses these phrases for transcription of speech input when it’s trying to understand the user’s request (or intent classification). You can also augment the custom vocabulary with information you have to collect (or slot elicitation) by configuring a setting on the slot type. You can specify the degree of boosting for each word or phrase by configuring a weight of 1, 2, or 3. We recommend you try out different weights. Words that are usually difficult to recognize or pronounce should have a higher weight.

The following is an example conversation with our bot:

Virtual Agent: Welcome to ACME bank. How can I help you?

Caller: Do you offer CapitaRise loans?

Virtual Agent: We offer CapitaRise loans for personal purposes. Do you want to get an application started?

Caller: No, thanks.

Virtual Agent: Is there anything else I can help you with today?

Caller: How about your refi rates?

Virtual Agent: For refinancing, we offer 4.5% with no closing costs for a 30 years fixed. Other information you might be interested in?

Caller: Yes, how about SBA?

Virtual Agent: We offer a variety of small business admin loans. You can find more information on our website. Anything else I can do for you?

Caller: I would like to do a payment on my HELOC.

Virtual Agent: To proceed with a payment, what is the account ID?

Caller: 1234

Virtual Agent: Thanks. What’s the payment amount?

Caller: 120

Virtual Agent: Thank you. The payment should be processed within 1 to 2 business days. Anything else I can help you with?

Caller: No, that’s it.

Virtual Agent: Thank you. Have a great day.

Solution overview

Let’s review the overall architecture for the solution (see the following diagram):

  • We use an Amazon Lex bot integrated with an Amazon Connect contact flow to deliver the conversational experience
  • We define the custom vocabulary for the English (GB) language by adding words such as “CapitaRise,” “HELOC,” and “refi”, along with weights
  • After the caller is authenticated, the control is passed to the bot to perform transactions (for example, to process payment)

The custom vocabulary file is a tab-separated list of values that contain the phrase to recognize and a weight to give the boost. Phrases with a higher boost value are more likely to be used when they appear in the audio input.

phrase	weight
CapitaRise	3
HELOC	2
Refi	2
S. B. A.	1

Deploy the sample Amazon Lex bot

To create the sample bot and configure the custom vocabulary, perform the following steps. This creates an Amazon Lex bot calledFinanceBot, with intents PersonalLoan, BusinessLoan, InterestRateRefinancing, InterestRateCredit, Payment, Welcome, and Goodbye, as well as two slot types (accountNumber and confirmationSlot).

  1. Download the Amazon Lex bot.
  2. On the Amazon Lex console, choose Actions, Import.
  3. Choose the file FinanceBot.zip file that you downloaded, and choose Import.
  4. In the IAM Permissions section, for Runtime role, choose Create a new role with basic Amazon Lex permissions.
  5. On the Amazon Lex console, navigate to the bot FinanceBot.
  6. Download the .zip file with the phrases that you want to add to the custom vocabulary.
  7. On the bot detail page, in the Add languages section, choose View languages.
  8. From the list of languages, choose English (GB).
  9. In the Custom vocabulary section, choose Import.
  10. Browse to the file to import, enter a password if necessary, and then choose Import.
  11. Choose Build.
  12. Download the supporting AWS Lambda code.
  13. On the Lambda console, create a new function and select Author from scratch.
  14. For Function name¸ enter FinanceBotEnglish.
  15. For Runtime, choose Python 3.8.
  16. Choose Create function.
  17. In the Code source section, open lambda_function.py and delete the existing code.
  18. Download the code and open it in a text editor.
  19. Copy and paste the code into the empty lambda_function.py tab.
  20. Choose Deploy.
  21. On the Amazon Lex console, and open FinanceBot.
  22. Choose Deployment and then Aliases, followed by TestBotAlias.
  23. On the Aliases page, in the Languages section, navigate to English (GB).
  24. For Source, select FinanceBotEnglish.
  25. For Lambda version or alias, enter $LATEST.
  26. On the Amazon Connect console, choose Contact flows.
  27. Download the contact flow to integrate with the Amazon Lex bot.
  28. In the Amazon Lex section, select your Amazon Lex bot and make it available for use in the Amazon Connect contact flows.
  29. Select the contact flow to load it into the application.
  30. Make sure the right bot is configured in the “Get Customer Input” block.
  31. Choose a queue in the “Set working queue” block.
  32. Add a phone number to the contact flow.
  33. Test the IVR flow by calling in to the phone number.

Test the solution

You can call in to the Amazon Connect phone number and interact with the bot.

Conclusion

Custom vocabulary enables improved recognition of domain-specific words and brand names for speech modality. You can easily define the custom vocabulary for your Amazon Lex bot and augment it to the bot definition. With improved recognition, you can enable more effective conversations across a broader set of use cases. You can configure custom vocabulary using the Amazon Lex V2 console or via the API. The capability is available for English (US) and English (GB) in all AWS Regions where Amazon Lex operates. To learn more, refer to custom vocabulary documentation.

About the Authors

Kai Loreck is a professional services Amazon Connect consultant. He works on designing and implementing scalable customer experience solutions. In his spare time, he can be found playing sports, snowboarding, or hiking in the mountains.

Anubhav Mishra is a Product Manager with AWS. He spends his time understanding customers and designing product experiences to address their business challenges.

Mebz Qazi is a Senior Consultant working on global projects for AWS. He very much enjoys working on technological innovation in natural language and AI/ML.

Sravan Bodapati is an Applied Science Manager at AWS Lex. He focuses on building cutting edge Artificial Intelligence and Machine Learning solutions for AWS customers in ASR and NLP space. In his spare time, he enjoys hiking, learning economics, watching TV shows and spending time with his family.

Read More