Watch the recorded panel discussion that aired the week of NeurIPS 2020.Read More
Your guide to artificial intelligence and machine learning at re:Invent 2020
With less than a week to re:Invent 2020, we are feeling the excitement and thrill, and looking forward to seeing you all at the world’s premier cloud learning event. As always, artificial intelligence (AI) and machine learning (ML) continue to be on the list of top topics with our customers and partners. We’re making it bigger and better this year with the first ever machine learning keynote, over 50 technical breakout sessions, live racing with the AWS DeepRacer League, and more. You’ll hear from AWS experts as well as many of our customers including NASCAR, Intuit, McDonalds, Mobileye, NFL, Siemens Energy, and many others, across industries such as sports, finance, retail, autonomous vehicles, manufacturing, and more.
To help you plan your agenda for the extravaganza, here are a few highlights from the artificial intelligence and machine learning track at re:Invent 2020. So fasten your seat belts, register now, and let’s get started.
Machine Learning Keynote
This year, we’re hosting the first ever machine learning keynote, delivered by Swami Sivasubramanian, Vice President, Machine Learning, AWS. Join Swami live and hear how AWS is freeing builders to innovate on machine learning with the latest developments in AI and ML. We are excited to bring you new product launches, demos, customer stories, and more.
Getting started with AI and ML
If you’re new to AI and ML, we have some sessions that’ll help you get started with the basics on how AI and ML can bring value to your business. These sessions cover the foundational elements of AI and ML, and make it fun for you with the popular AWS DeepRacer and AWS DeepComposer devices.
From PoC to Production (Session AIM203)
If you’re planning to implement ML in your organization and wondering where to start, this session is for you. Learn how to achieve success using ML at scale, including best practices to get from concept to production.
Discover insights to deliver customer delight (AIM202)
Sheer volume of customer responses and sentiments should not deter you from delivering excellent customer service. Learn how you can use Amazon Comprehend to extract valuable information and convert it into business insights.
Get rolling with AWS DeepRacer (AIM204)
Developers, start your engines. Get hands-on with AWS DeepRacer and learn about the basics of machine learning and reinforcement learning in a fun and exciting way. Build a model and submit it to the AWS DeepRacer League for a chance to win prizes and glory.
Get started with Generative AI with AWS DeepComposer (AIM207)
Generative AI has been used to solve unique problems across many industries. Learn about the latest concepts in generative AI techniques such as GANs and autoregressive models, while having fun using music with AWS DeepComposer.
Let’s get practical with use cases across industries
The broad applicability of machine learning means it can help customers for a wide range of use cases across many industries. In these sessions, we dive deep into the practical aspects of AI and ML for use cases in industries including finance, healthcare, retail, media and entertainment, and more.
Intelligent document processing for the insurance industry (AIM310)
The insurance industry spends millions of dollars on manual data extraction efforts that are error-prone, expensive, and tedious. Learn how you can use Amazon Textract and Amazon Augmented AI (Amazon A2I) to get your automated document processing workflow into production, and turn tedious processes into streamlined operations.
Deliver a superior personal experience in M&E (AIM211)
Learn how to add ML-powered recommendation engines to increase subscription and consumption with a personalized approach for your media and entertainment requirements. In this session, we talk about how Amazon Personalize can deliver excellent results for all your business requirements.
Learn how NASCAR is making content searchable and accessible with Amazon Transcribe (AIM205)
Media and entertainment content are growing exponentially. In this session, NASCAR will share how they use Amazon Transcribe to enhance user engagement with automated video subtitles. We will also discuss how other customers can use Amazon Transcribe to cost effectively create accurate transcriptions of videos and podcasts, generate metadata for content cataloging and search, and moderate content.
Build computer vision training datasets with the National Football League (NFL) (AIM307)
Tracking players on a sports field is challenging, and training high-quality ML models to predict player movement and injury can be daunting. In this session, learn how the NFL uses Amazon SageMaker Ground Truth to build training datasets to track all 22 players on the field for player safety and tracking game statistics, with a fun and insightful demo.
Use AI to automate clinical workflows (AIM303)
Learn how healthcare organizations can harness the power of AI to automate clinical workflows, digitize medical information, extract and summarize information, and protect patient data. In this session, we demonstrate how the wide range of AI services from AWS, including Amazon Textract, Amazon Comprehend Medical, Amazon Transcribe Medical, and Amazon Kendra, work together to extract, analyze, and summarize data within medical records.
Catch online fraud and act to achieve successful outcomes (AIM208)
Fraud is a problem across industries, and organizations can lose billions of dollars each year. In this session, learn how Amazon Fraud Detector can help you catch online fraud with the rich experience and expertise of over 20 years at Amazon using machine learning. Learn how customers are applying Amazon Fraud Detector to their businesses with a detailed discussion and demo.
Secure and compliant machine learning for regulated industries (AIM309)
As more and more workloads move to the cloud, having a secure environment is mandatory. At AWS, security is top priority for us, and this session dives deep into provisioning a secure ML environment using Amazon SageMaker. We deep dive into customer architectures for regulated industries such as finance with detailed concepts and a technical demo.
AI and DevOps – an unbeatable combination
Find your most expensive lines of code with Amazon CodeGuru (AIM306)
Improving application response times, resolving service issues faster and more accurately, and understanding your code better is top of mind with most developers. In this session, you learn how Amazon CodeGuru can help you with intelligent recommendations to automate code reviews during development, and how you can improve performance of your applications with a more robust code.
Advanced topics for the experienced ML practitioner
We have a number of sessions that dive deep into the technical details of machine learning across our service portfolio as well as deep learning frameworks including TensorFlow, PyTorch, and Apache MXNet. These sessions dive deep into code with demos and enable the advanced developer or the data scientist in you to solve many challenges with deep technical solutions.
Get started with Amazon SageMaker (AIM209)
Amazon SageMaker is a fully managed service enabling all developers and data scientists with every aspect of the ML lifecycle. Join us in this session as we show you how you can get started with SageMaker for a variety of use cases, including predictive maintenance, churn prediction, credit risk, and more, with a simple and intuitive experience.
Use fully managed Jupyter notebooks in Amazon SageMaker Studio (AIM305)
Managing compute instances to view, run, or share notebooks can be tedious. With Amazon SageMaker Studio Notebooks, it’s easy to use notebooks for your ML projects. The underlying compute resources are fully elastic and the notebooks can be shared with others, making collaboration easy and scalable. This session demonstrates how you can use Jupyter notebooks with SageMaker Studio, the first fully integrated development environment for machine learning.
Train ML models at scale with billions of parameters in TensorFlow 2.0 (AIM405)
Training large models with many inputs is often restricted due to limiting factors such as GPU memory. In this session, we dive deep into a technical solution with model parallelism, placing layers of models on multiple GPU devices to utilize the aggregate memory. We demonstrate how you can train models at scale using Amazon SageMaker and TensorFlow 2.0.
Workplace safety and identity verification (AIM407)
Today, there have been new challenges to the workplace introduced by COVID-19 across businesses. Join us in this session to learn how Amazon Rekognition helps enhance workplace safety and help ensure safe identity verification with an automated, scalable solution. See how employees and customers can be safe in both physical and online environments in the changing workplace environment.
Scaling ML environments on Kubernetes (AIM313)
Setting up ML environments on Kubernetes for training and inference is becoming increasingly common. In this session, we demonstrate how you can run training at scale easily using Amazon SageMaker Operators for Kubernetes with secure, high-availability endpoints using autoscaling policies. You can also hear from Intuit on how they built a Kubernetes controller using SageMaker Operators, Apache Spark, and Argo for their high-availability systems.
Maintain highly accurate ML models by detecting model drift (AIM304)
Accuracy of models is often affected due to the difference in data between the one used for training and the one in production to generate predictions. In this session, you learn how Amazon SageMaker Model Monitor automatically detects drift in deployed models, provides detailed alerts to help identify the problem, and helps maintain highly accurate ML models.
Interpretability and Explainability in Machine Learning (AIM408)
As ML becomes increasingly ubiquitous across industries, use cases, and applications, it becomes critical to understand the results of ML models and their predictions. In this session, we discuss the science of interpretability with Amazon SageMaker and look under the hood to understand how ML models work. We also introduce model explainability, an area of increasing interest in the field of machine learning.
AWS DeepRacer – The fastest and most fun way to get rolling with machine learning
Machine learning can be a lot of fun with AWS DeepRacer. Developers of all skill levels from beginners to experts can get hands-on with AWS DeepRacer by learning to train models in a cloud-based 3D racing simulator. Whether you are novice to machine learning or an expert, AWS DeepRacer is for you. Start your engines here and test your skills in the AWS DeepRacer League in an exciting autonomous car racing experience throughout re:Invent to compete for prizes and glory.
Don’t miss out on the action. Register now for free and see you soon on the artificial intelligence and machine learning track at re:Invent 2020.
About the Author
Shyam Srinivasan is on the AWS Machine Learning marketing team. He cares about making the world a better place through technology and loves being part of this journey. In his spare time, Shyam likes to run long distances, travel around the world, and experience new cultures with family and friends.
Amazon Forecast now supports accuracy measurements for individual items
We’re excited to announce that you can now measure the accuracy of forecasts for individual items in Amazon Forecast, allowing you to better understand your forecasting model’s performance for the items that most impact your business. Improving forecast accuracy for specific items—such as those with higher prices or higher costs—is often more important than optimizing for all items. With this launch, you can now view accuracy for individual items and export forecasts generated during training. This information allows you to better interpret results by easily comparing performance against observed historical demand, aggregating accuracy metrics across custom sets of SKUs or time periods, or visualizing results without needing to hold out a separate validation dataset. From there, you can tailor your experiments to further optimize accuracy for items significant for your needs.
If a smaller set of items is more important for your business, achieving a high forecasting accuracy for those items is imperative. For retailers specifically, not all SKUs are treated equally. Usually 80% of revenue is driven by 20% of SKUs, and retailers look to optimize forecasting accuracy for those top 20% SKUs. Although you can create a separate forecasting model for the top 20% SKUs, the model’s ability to learn from relevant items outside of the top 20% is limited and accuracy may suffer. For example, a bookstore company looking to increase forecasting accuracy of best sellers can create a separate model for best sellers, but without the ability to learn from other books in the same genre, the accuracy for new best sellers might be poor. Evaluating how the model, which is trained on all the SKUs, performs against those top 20% SKUs provides more meaningful insights on how a better forecasting model can have a direct impact on business objectives.
You may instead look to optimize your forecasting models for specific departments. For example, for an electronic manufacturer, the departments selling the primary products may be more important than the departments selling accessory products, encouraging the manufacturer to optimize accuracy for those departments. Furthermore, the risk tolerance for certain SKUs might be higher than others. For long shelf life items, you may prefer to overstock because you can easily store excess inventory. For items with a short shelf life, you may prefer a lower stocking level to reduce waste. It’s ideal to train one model but assess forecasting accuracy for different SKUs at different stocking levels.
To evaluate forecasting accuracy at an item level or department level, you usually hold a validation dataset outside of Forecast and feed your training dataset to Forecast to create an optimized model. After the model is trained, you can generate multiple forecasts and compare those to the validation dataset, incurring costs during this experimentation phase, and reducing the amount of data that Forecast has to learn from.
Shivaprasad KT, Founder and CEO of Ganit, an analytics solution provider, says, “We work with customers across various domains of consumer goods, retail, hospitality, and finance on their forecasting needs. Across these industries, we see that for most customers, a small segment of SKUs drive most of their business, and optimizing the model for those SKUs is more critical than overall model accuracy. With Amazon Forecast launching the capability to measure forecast accuracy at each item, we are able to quickly evaluate the different models and provide a forecasting solution to our customers faster. This helps us focus more on helping customers with their business operation analysis and less on the manual and more cost-prohibitive tasks of generating forecasts and calculating item accuracy by ourselves. With this launch, our customers are able to experiment faster incurring low costs with Amazon Forecast.”
With today’s launch, you can now access the forecasted values from Forecast’s internal testing of splitting the data into training and backtest data groups to compare forecasts versus observed data and item-level accuracy metrics. This eliminates the need to maintain a holdout test dataset outside of Forecast. During the step of training a model, Forecast automatically splits the historical demand datasets into a training and backtesting dataset group. Forecast trains a model on the training dataset and forecasts at different specified stocking levels for the backtesting period, comparing to the observed values in the backtesting dataset group.
You can also now export the forecasts from the backtesting for each item and the accuracy metrics for each item. To evaluate the strength of your forecasting model for specific items or a custom set of items based on category, you can calculate the accuracy metrics by aggregating the backtest forecast results for those items.
You may group your items by department, sales velocity, or time periods. If you select different stocking levels, you can choose to assess the accuracy of certain items at certain stocking levels, while measuring accuracy of other items at different stocking levels.
Lastly, now you can easily visualize the forecasts compared to your historical demand by exporting the backtest forecasts to Amazon QuickSight or any other visualization tool of your preference.
Forecast provides different model accuracy metrics for you to assess the strength of your forecasting models. We provide the weighted quantile loss (wQL) metric for each selected distribution point, and weighted absolute percentage error (WAPE) and root mean square error (RMSE), calculated at the mean forecast. For more information about how each metric is calculated and recommendations for the best use case for each metric, see Measuring forecast model accuracy to optimize your business objectives with Amazon Forecast.
Although Forecast provides these three industry-leading forecast accuracy measures, you might prefer to calculate accuracy using different metrics. With the launch of this feature, you can use the export of forecasts from backtesting to calculate the model accuracy using your own formula, without the need to generate forecasts and incur additional cost during experimentation.
After you experiment and finalize a forecasting model that works for you, you can continue to generate forecasts on a regular basis using the CreateForecast API.
Exporting forecasts from backtesting and accuracy metrics for each item
To use this new capability, use the newly launched CreatePredictorBacktestExportJob API after training a predictor. In this section, we walk through the steps on the Forecast console using the Bike Sharing dataset example in our GitHub repo. You can also refer to this notebook in our GitHub repo to follow through these steps using the Forecast APIs.
The bike sharing dataset forecasts the number of bike rides expected in a location. There are more than 400 locations in the dataset.
- On the Forecast console, create a dataset group.
- Upload the target time series file from the bike dataset.
- In the navigation pane, choose Predictors.
- Choose Train predictor.
- For training a predictor, we use the following configuration:
-
- For Forecast horizon, choose 24.
- For Forecast frequency, set to hourly.
- For Number of backtest windows, choose 5.
- For Backtest window offset, choose 24.
- For Forecast types, enter mean, 65, and 0.90.
- For Algorithm, select Automatic (AutoML).
- After your predictor is trained, choose your predictor on the Predictors page to view details of the accuracy metrics.
- On the predictor’s details page, choose Export backtest results in the Predictor metrics
- For S3 predictor backtest export location, enter the details of your Amazon Simple Storage Service (Amazon S3) location for exporting the CSV files.
Two type of files are exported to the Amazon S3 location in two different folders:
- forecasted-values – Contains the forecasts from each backtest window. The file name convention is
Forecasts_PredictorBacktestExportJobName_CurrentTimestamp_PartNumber.csv. For this post, the file name isForecasts_bike_demo_auto_export_2020-11-19-00Z_part0.csv. - accuracy-metrics-values – Contains the accuracy metrics for each item per backtest window. The file name convention is
Accuracy_ PredictorBacktestExportJobName_CurrentTimestamp_PartNumber.csv. For this post, the file name isAccuracy_bike_demo_auto_export_2020-11-19-00Z_part0.csv.
The wQL, WAPE, and RMSE metrics are provided for the accuracy metric file. Sometimes, output files are split into multiple parts based on the size of the output and are given numbering like part0, part1, and so on.
The following screenshot shows part of the Forecasts_bike_demo_auto_export_2020-11-19-00Z_part0.csv file from the bike dataset backtest exports.
The following screenshot shows part of the Accuracy_bike_demo_auto_export_2020-11-19-00Z_part0.csv file from the bike dataset backtest exports.
- After you finalize your predictor, choose Forecasts in the navigation pane.
- Choose Create a forecast.
- Select your trained predictor to create a forecast.
Visualizing backtest forecasts and item accuracy metrics
With the backtest forecasts, you can use a visualization tool like Amazon QuickSight to create graphs that help you visualize and compare the forecasts against actuals and graphically assess accuracy. In our notebook, we walk through some visualization examples for your reference. The graph below visualizes the backtest forecasts at different distributions points and the actual observed demand for different items.
Calculating custom metrics
We provide the following model accuracy metrics: weighted quantile loss (wQL) metric, weighted absolute percentage error (WAPE) and root mean square error (RMSE). Now with the export of the backtest forecasts, you can also calculate custom model accuracy metrics such as MAPE using the following formula:
In our notebook, we discuss how you can calculate MAPE and metrics for slow- and fast-moving items.
Tips and best practices
In this section, we share a few tips and best practices when using Forecast:
- Before experimenting with Forecast, define your business problem related to costs of under-forecasting or over-forecasting. Evaluate the trade-offs and prioritize if you would rather over-forecast than under. This helps you determine the forecasting quantile to choose.
- Experiment with multiple distribution points to optimize your forecast model to balance the costs associated with under-forecasting and over-forecasting. Choose a higher quantile if you want to over-forecast to meet demand. The backtest forecasts and accuracy metric files help you assess and optimize the model’s performance against under-forecasting and over-forecasting.
- If you’re comparing different models, use the weighted quantile loss metric at the same quantile for comparison. The lower the value, the more accurate the forecasting model.
- Forecast allows you to select up to five backtest windows. Forecast uses backtesting to tune predictors and produce accuracy metrics. To perform backtesting, Forecast automatically splits your time series datasets into two sets: training and testing. The training set is used to train your model, and the testing set to evaluate the model’s predictive accuracy. We recommend choosing more than one backtest window to minimize selection bias that may make one window more or less accurate by chance. Assessing the overall model accuracy from multiple backtest windows provides a better measure of the strength of the model.
- To create mean forecasts, specify mean as a forecast type. A forecast type of 0.5 refers to the median quantile.
- The wQL metric is not defined for the mean forecast type. However, the WAPE and RMSE metrics are calculated at mean. To view the wQL metric, specify a forecast type of 0.5.
- If you want to calculate a custom metric, such as MAPE, specify mean as a forecast type, and use the backtest forecasts corresponding to mean for this calculation.
Conclusion
Some items may be more important than others in a dataset, and optimizing accuracy for those important items becomes critical. Forecast now supports forecast accuracy measurement for each item separately, enabling you to make better forecasting decisions for items that drive your business metrics. To get started with this capability, see the CreatePredictorBacktestExportJob API. We also have a notebook in our GitHub repo that walks you through how to use the Forecast APIs to export accuracy measurements of each item and calculate accuracy metrics for a custom set of items. You can use this capability in all Regions where Forecast is publicly available. For more information about Region availability, see Region Table.
About the Authors
Namita Das is a Sr. Product Manager for Amazon Forecast. Her current focus is to democratize machine learning by building no-code/low-code ML services. On the side, she frequently advises startups and is raising a puppy named Imli.
Punit Jain is working as SDE on the Amazon Forecast team. His current work includes building large scale distributed systems to solve complex machine learning problems with high availability and low latency as a major focus. In his spare time, he enjoys hiking and cycling.
Christy Bergman is working as an AI/ML Specialist Solutions Architect at AWS. Her work involves helping AWS customers be successful using AI/ML services to solve real world business problems. Prior to joining AWS, Christy worked as a Data Scientist in banking and software industries. In her spare time, she enjoys hiking and bird watching.
Amazon Lex launches support for Latin American Spanish and German
¡Amazon Lex lanza soporte para español para América Latina! Amazon Lex startet auf Deutsch!
Amazon Lex is a service for building conversational interfaces into any application using voice and text. Starting today, Amazon Lex supports Latin American Spanish and German. Now you can easily create virtual agents, conversational IVR systems, self-service chatbots, or application bots to answer and resolve questions for your Latin American Spanish and German speakers.
Customer stories
To increase employee productivity and provide a better customer service experience, companies are looking to create more ways for their customers to get their questions answered and tasks completed. See how some of our customers are using Amazon Lex to create virtual contact center agents, chat interfaces, and knowledge management bots.
Xpertal
Fomento Económico Mexicano, S.A.B. de C.V., or FEMSA, is a Mexican multinational beverage and retail company. Xpertal Global Services is FEMSA’s Service Unit, offering consulting, IT, back-office transactional, and consumable procurement services to the rest of FEMSA’s business units. One of Xpertal’s services operates an internal help desk comprised of 150-agent Contact Center that handles approximately 4 million calls per year.
As Xpertal shares in the post How Xpertal is creating the Contact Center of the future with Amazon Lex, they first started to build Amazon Lex bots with US Spanish for multiple internal web portals and integrated it with Amazon Connect to their Contact Center. With today’s launch of Latin American Spanish, they are excited to migrate and create an even more localized experience for their customers.
Xpertal’s Contact Center Manager, Chester Perez, shares, “Our goal is to keep evolving as an organization and find better ways to deliver our products and improve customer satisfaction. Our talented internal team developed various initiatives focused on bringing more intelligence and automation into our contact center to provide self-service capabilities, improve call deflection rates, reduce call wait times, and increase agent productivity. Amazon Lex is simple to use and the Contact Center team was already creating bots after just a 1-hour enablement session.
“Thanks to AWS AI services, we can finally focus on how to apply the technology for our users’ benefit and not on what’s behind it.”
Decadia
Amazon Lex is also opening opportunities for Decadia, a provider of company pension plans in Germany. Joerg Passmann, an Executive Director at Decadia, says, “Providing quality corporate pension plans is our passion, and the concerns of our pension members are close to our hearts at Decadia. We’d love to address them around the clock. Amazon Lex gives us the opportunity to process inquiries outside of our regular service hours and to be available with adequate assistance at any time. We look forward to exploring the diverse fields of application as part of a pilot phase.”
E.ON
E.ON is the largest energy company in Europe. Dr. Juan Bernabé-Moreno, Chief Data Officer, says, “The energy world is changing at an unprecedented speed, and E.ON is actively shaping it. Handling the ever-increasing complexity requires a new approach to how we organize and manage our knowledge to better understand our customers, our competitors, our regulatory and political environment, and also the emerging trends and the new players. For that, we created an AI-powered knowledge management engine called Sophia. With Amazon Lex, we want to bring Sophia closer to each and every employee in the company, so that our decisions are always taken considering all the facts and knowledge available… in other words, we want to make Sophia part of each and every conversation, and Lex capabilities are quite promising to humanize Sophia.”
How to get started
Start exploring how you can apply Amazon Lex to your business processes. This post shows you how to Expand Amazon Lex conversational experiences with Spanish. To use the new Amazon Lex languages, simply choose the language when creating a new bot via the Amazon Lex console or SDK.
For more information, see the Amazon Lex Developer Guide.
Conclusion
Amazon Lex is a powerful service for building conversational interfaces into your applications. Try using it to help increase call deflect rates, increase first call resolution rates, and reduce call times in your contact center. Or add it to the front of your knowledge bases to help your employees and customers find the answers they need faster. See all the ways in which other customers are using Amazon Lex.
About the Author
Esther Lee is a Product Manager for AWS Language AI Services. She is passionate about the intersection of technology and education. Out of the office, Esther enjoys long walks along the beach, dinners with friends and friendly rounds of Mahjong.
How Xpertal is creating the Contact Center of the future with Amazon Lex
This is a joint blog post with AWS Solutions Architects, Jorge Alfaro Hidalgo and Mauricio Zajbert, and Chester Perez, the Contact Center Manager at Xpertal. Fomento Económico Mexicano, S.A.B. de C.V. (FEMSA) is a Mexican multinational beverage and retail company headquartered in Monterrey, Mexico.
Fomento Económico Mexicano, S.A.B. de C.V., or FEMSA, is a Mexican multinational beverage and retail company headquartered in Monterrey, Mexico. Xpertal Global Services is FEMSA’s service unit that offers consulting, IT, back-office transactional, and consumable procurement services to the rest of FEMSA’s business units. Xpertal operates a Contact Center which serves as an internal help desk for employees and has 150 agents that handles 4 million calls per year. Their goal is to automate the majority calls by 2023 with a chatbot and only escalate complex queries requiring human intervention to live agents.
The contact center started this transformation 2 years ago with Robotic Process Automation (RPA) solutions and it has already been a big success. They’ve removed repetitive tasks such as password resets and doubled the number of requests serviced with the same number of agents.
This technology was helpful, but to achieve the next level of automation, they needed to start looking at systems that could naturally emulate human interactions. This is where Amazon AI has been helpful. As part of the journey, Xpertal started exploring Amazon Lex, a service to create self-service virtual agents to improve call response times. In addition, they used other AI services such as Amazon Comprehend, Amazon Polly, and Amazon Connect to automate other parts of the contact center.
In the first stage, Xpertal used Amazon Comprehend, a natural language processing service to classify support request emails automatically and route them to the proper resolution teams. This process used to take 4 hours to perform manually, and was reduced to 15 minutes with Amazon Comprehend.
Next, Xpertal started to build bots supporting US Spanish with Amazon Lex for multiple internal websites. They’ve been able to optimize each bot to fit each business units’ need and integrate it with Amazon Connect, an omni-channel cloud contact center. Then the Lex bot can also help to resolve employee calls coming into the Contact Center. With today’s launch of Latin American Spanish, they are excited to migrate and create an even more localized experience for their employees.
It was easy to integrate Amazon Lex with Amazon Connect and other third-party collaboration tools used within FEMSA to achieve an omni-channel system for support requests. Some of the implemented channels include email, phone, collaboration tools, and internal corporate websites.
In addition, Amazon Lex has been integrated with a diverse set of information sources within FEMSA to create a virtual help desk that enables their employees to find answers faster. These information sources include their CRM and internal ticketing systems. It’s now possible for users to easily chat with the help desk system to create support tickets or get a status update. They’ve also been able to build these types of conversational interactions with other systems and databases to provide more natural responses to users.
The following diagram shows the solution architecture for Xpertal’s Contact Center.
This architecture allows calls coming into the Contact Center to be routed to Amazon Connect. Amazon Connect then invokes Amazon Lex to identify the caller’s need. Subsequently, Amazon Lex uses AWS Lambda to interact with applications’ databases to either fulfill the user’s need by retrieving the information needed or creating a ticket to escalate the user’s request to the appropriate support team.
In addition, all customer calls are recorded and transcribed with Amazon Transcribe for post-call analytics to identify improvement areas and usage trends. The Xpertal team is effectively able to track user interactions. By analyzing the user utterances the bot didn’t understand, the team is able to monitor the solution’s effectiveness and continuously improve containment rates.
Xpertal’s Contact Center Manager, Chester Perez, shares, “Our goal is to keep evolving as an organization and find better ways to deliver our products and improve customer satisfaction. Our talented internal team developed various initiatives focused on bringing more intelligence and automation into our internal contact center to provide self-service capabilities, improve call deflection rates, reduce call wait times, and increase agent productivity. With Amazon Lex’s easy to use interface, our Contact Center team was able to create bots after a 1-hour training session. Thanks to AWS AI services, we can finally focus on how to apply the technology for our users’ benefit and not on what’s behind it.”
Summary
AWS has been working with a variety of customers such as Xpertal to find ways for AI services like Amazon Lex to boost self-service capabilities that lead to call containment and improve the overall contact center productivity and customer experience in Spanish.
Get started with this “How to create a virtual call center agent with Amazon Lex” tutorial with any of our localized Spanish language choices. Amazon Lex now offers Spanish, US Spanish, and LATAM Spanish. Depending on your contact center goals, learn more about Amazon Connect’s omni-channel, cloud-based contact center or bring your own telephony (BYOT) with AWS Contact Center Intelligence.
About the Authors
Chester Perez has 17 years working with FEMSA group, has contributed in areas of development, infrastructure and architecture. He has also designed and implemented teams that provide specialized support and center of excellence in data center. He is currently Manager of the Contact Center at Xpertal and his main challenge is to improve the quality and efficiency of the service it provides, by transforming the area into a technological and internal talent sense.
Jorge Alfaro Hidalgo is an Enterprise Solutions Architect in AWS Mexico with more than 20 years of experience in IT industry, he is passionate about helping enterprises to AWS cloud journey building innovative solutions to achieve their business objectives.
Mauricio Zajbert has more than 30 years of experience in the IT industry and a fully recovered infrastructure professional; he’s currently Solutions Architecture Manager for Enterprise accounts in AWS Mexico leading a team that helps customers in their cloud journey. He’s lived through several technology waves and deeply believes none has offered the benefits of the cloud.
Announcing the launch of Amazon Comprehend Events
Every day, financial organizations need to analyze news articles, SEC filings, and press releases, as well as track financial events such as bankruptcy announcements, changes in executive leadership at companies, and announcements of mergers and acquisitions. They want to accurately extract the key data points and associations among various people and organizations mentioned within an announcement to update their investment models in a timely manner. Traditional natural language processing services can extract entities such as people, organizations and locations from text, but financial analysts need more. They need to understand how these entities relate to each other in the text.
Today, Amazon Comprehend is launching Comprehend Events, a new API for event extraction from natural language text documents. With this launch, you can use Comprehend Events to extract granular details about real-world events and associated entities expressed in unstructured text. This new API allows you to answer who-what-when-where questions over large document sets, at scale and without prior NLP experience.
This post gives an overview of the NLP capabilities that Comprehend Events supports, along with suggestions for processing and analyzing documents with this feature. We’ll close with a discussion of several solutions that use Comprehend Events, such as knowledge base population, semantic search, and document triage, all of which can be developed with companion AWS services for storing, visualizing, and analyzing the predictions made by Comprehend Events.
Comprehend Events overview
The Comprehend Events API, under the hood, converts unstructured text into structured data that answers who-what-when-where-how questions. Comprehend Events lets you extract the event structure from a document, distilling pages of text down to easily processed data for consumption by your AI applications or graph visualization tools. In the following figure, an Amazon press release announcing the 2017 acquisition of Whole Foods Market, Inc. is rendered as a graph showing the core semantics of the acquisition event, as well as the status of Whole Foods’ CEO post merger.
Amazon (AMZN) today announced that they will acquire Whole Foods Market (WFM) for $42 per share in an all-cash transaction valued at approximately $13.7 billion, including Whole Foods Market’s net debt. Whole Foods Market will continue to operate stores under the Whole Foods Market brand and source from trusted vendors and partners around the world. John Mackey will remain as CEO of Whole Foods Market and Whole Foods Market’s headquarters will stay in Austin, Texas.
Extracted event triggers – Which events took place. In our example, CORPORATE_ACQUISITION and EMPLOYMENT events were detected. Not shown in the preceding figure, the API also returns which words in the text indicate the occurrence of the event, for example the words “acquire” and “transaction” in the context of the document indicate that a CORPORATE_ACQUISITION took place. The Comprehend Events API returns a variety of insights into the event semantics of a document:
- Extracted entity mentions – Which words in the text indicate which entities are involved in the event, including named entities such as “Whole Foods Market” and common nouns such as “today.” The API also returns the type of the entity detected, for example
ORGANIZATIONfor “Whole Foods Market.” - Event argument role (also known as slot filling) – Which entities play which roles in which events; for example Amazon is an
INVESTORin the acquisition event. - Groups of coreferential event triggers – Which triggers in the document refer to the same event. The API also groups triggers such as “transaction” and “acquire” around the
CORPORATE_ACQUISITIONevent (not shown above). - Groups of coreferential entity mentions – Which mentions in the document refer to the same entity. For example, the API returns the grouping of “Amazon” with “they” as a single entity (not shown above).
At the time of launch, Comprehend Events is available as an asynchronous API supporting extraction of a fixed set of event types in the finance domain. This domain includes a variety of event types (such as CORPORATE_ACQUISITION and IPO), both standard and novel entity types (such as PER and ORG vs. STOCK_CODE and MONETARY_VALUE), and the argument roles that can connect them (such as INVESTOR, OFFERING_DATE, or EMPLOYER). For the complete ontology, see the Detect Events API documentation.
To demonstrate the functionality of the feature, we’ll show you how to process a small set of sample documents, using both the Amazon Comprehend console and the Python SDK.
Formatting documents for processing
The first step is to transform raw documents into a suitable format for processing. Comprehend Events imposes a few requirements on document size and composition:
- Individual documents must be UTF-8 encoded and no more than 10 KB in length. As a best practice, we recommend segmenting larger documents at logical boundaries (section headers) or performing sentence segmentation with existing open-source tools.
- For best performance, markup (such as HTML), tabular material, and other non-prose spans of text should be removed from documents. The service is intended to process paragraphs of unstructured text.
- A single job must not contain more than 50 MB of data. Larger datasets must be divided into smaller sets of documents for parallel processing. The different document format modes also impose size restrictions:
- One document per file (ODPF) – A maximum of 5,000 files in a single Amazon Simple Storage Service (Amazon S3) location.
- One document per line (ODPL) – A maximum of 5,000 lines in a single text file. Newline characters (
n,r,rn) should be replaced with other whitespace characters within a given document.
For this post, we use a set of 117 documents sampled from Amazon’s Press Center: sample_finance_dataset.txt. The documents are formatted as a single ODPL text file and already conform to the preceding requirements. To implement this solution on your own, just upload the text file to an S3 bucket in your account before continuing with the following steps.
Job creation option 1: Using the Amazon Comprehend console
Creating a new Events labeling job takes only a few minutes.
- On the Amazon Comprehend console, choose Analysis jobs.
- Chose Create job.
- For Name, enter a name (for this post, we use
events-test-job). - For Analysis type¸ choose Events.
- For Language, choose English.
- For Target event types, choose your types of events (for example, Corporate acquisition).
- In the Input data section, for S3 location, enter the location of the sample ODPL file you downloaded earlier.
- In the Output data section, for S3 location, enter a location for the event output.
- For IAM role, choose to use an existing AWS Identity and Access (IAM) role or create a new one.
- Choose Create job.
A new job appears in the Analysis jobs queue.
Job creation option 2: Using the SDK
Alternatively, you can perform these same steps with the Python SDK. First, we specify Comprehend Events job parameters, just as we would with any other Amazon Comprehend feature. See the following code:
# Client and session information
session = boto3.Session()
comprehend_client = session.client(service_name="comprehend")
# Constants for S3 bucket and input data file.
bucket = "comprehend-events-blogpost-us-east-1"
filename = 'sample_finance_dataset.txt'
input_data_s3_path = f's3://{bucket}/' + filename
output_data_s3_path = f's3://{bucket}/'
# IAM role with access to Comprehend and specified S3 buckets
job_data_access_role = 'arn:aws:iam::xxxxxxxxxxxxx:role/service-role/AmazonComprehendServiceRole-test-events-role'
# Other job parameters
input_data_format = 'ONE_DOC_PER_LINE'
job_uuid = uuid.uuid1()
job_name = f"events-job-{job_uuid}"
event_types = ["BANKRUPTCY", "EMPLOYMENT", "CORPORATE_ACQUISITION",
"INVESTMENT_GENERAL", "CORPORATE_MERGER", "IPO",
"RIGHTS_ISSUE", "SECONDARY_OFFERING", "SHELF_OFFERING",
"TENDER_OFFERING", "STOCK_SPLIT"]
Next, we use the start_events_detection_job API endpoint to start the analysis of the input data file and capture the job ID, which we use later to poll and retrieve results:
# Begin the inference job
response = comprehend_client.start_events_detection_job(
InputDataConfig={'S3Uri': input_data_s3_path,
'InputFormat': input_data_format},
OutputDataConfig={'S3Uri': output_data_s3_path},
DataAccessRoleArn=job_data_access_role,
JobName=job_name,
LanguageCode='en',
TargetEventTypes=event_types
)
# Get the job ID
events_job_id = response['JobId']
An asynchronous Comprehend Events job typically takes a few minutes for a small number of documents and up to several hours for lengthier inference tasks. For our sample dataset, inference should take approximately 20 minutes. It’s helpful to poll the API using the describe_events_detection_job endpoint. When the job is complete, the API returns a JobStatus of COMPLETED. See the following code:
# Get current job status
job = comprehend_client.describe_events_detection_job(JobId=events_job_id)
# Loop until job is completed
waited = 0
timeout_minutes = 30
while job['EventsDetectionJobProperties']['JobStatus'] != 'COMPLETED':
sleep(60)
waited += 60
assert waited//60 < timeout_minutes, "Job timed out after %d seconds." % waited
job = comprehend_client.describe_events_detection_job(JobId=events_job_id)
Finally, we collect the Events inference output from Amazon S3 and convert to a list of dictionaries, each of which contains the predictions for a given document:
# The output filename is the input filename + ".out"
output_data_s3_file = job['EventsDetectionJobProperties']['OutputDataConfig']['S3Uri'] + filename + '.out'
# Load the output into a result dictionary # Get the files.
results = []
with smart_open.open(output_data_s3_file) as fi:
results.extend([json.loads(line) for line in fi.readlines() if line])The Comprehend Events API output schema
When complete, the output is written to Amazon S3 in JSON lines format, with each line encoding all the event extraction predictions for a single document. Our output schema includes the following information:
- Comprehend Events system output contains separate objects for entities and events, each organized into groups of coreferential objects.
- The API output includes the text, character offset, and type of each entity mention and trigger.
- Event argument roles are linked to entity groups by an
EntityIndex. - Confidence scores for classification tasks are given as Score. Confidence of entity and trigger group membership is given with
GroupScore. - Two additional fields,
FileandLine, are present as well, allowing you to track document provenance.
The following Comprehend Events API output schema represents entities as lists of mentions and events as lists of triggers and arguments:
{
"Entities": [
{
"Mentions": [
{
"BeginOffset": number,
"EndOffset": number,
"Score": number,
"GroupScore": number,
"Text": "string",
"Type": "string"
}, ...
]
}, ...
],
"Events": [
{
"Type": "string",
"Arguments": [
{
"EntityIndex": number,
"Role": "string",
"Score": number
}, ...
],
"Triggers": [
{
"BeginOffset": number,
"EndOffset": number,
"Score": number,
"Text": "string",
"GroupScore": number,
"Type": "string"
}, ...
]
}, ...
]
"File": "string",
"Line": "string
}
Analyzing Events output
The API output encodes all the semantic relationships necessary to immediately produce several useful visualizations of any given document. We walk through a few such depictions of the data in this section, referring you to the Amazon SageMaker Jupyter notebook accompanying this post for the working Python code necessary to produce them. We use the press release about Amazon’s acquisition of Whole Foods mentioned earlier in this post as an example.
Visualizing entity and trigger spans
As with any sequence labeling task, one of the simplest visualizations for Comprehend Events output is highlighting triggers and entity mentions, along with their respective tags. For this post, we use displaCy‘s ability to render custom tags. In the following visualization, we see some of the the usual range of entity types detected by NER systems (PERSON, ORGANIZATION), as well as finance-specific ones, such as STOCK_CODE and MONETARY_VALUE. Comprehend Events detects non-named entities (common nouns and pronouns) as well as named ones. In addition to entities, we also see tagged event triggers, such as “merger” (CORPORATE_MERGER) and “acquire” (CORPORATE_ACQUISITION).
Graphing event structures
Highlighting tagged spans is informative because it localizes system predictions about entity and event types in the text. However, it doesn’t show the most informative thing about the output: the predicted argument role associations among events and entities. The following plot depicts the event structure of the document as a semantic graph. In the graph, vertices are entity mentions and triggers; edges are the argument roles held by the entities in relation to the triggers. For simple renderings of a small number of events, we recommend common open-source tools such as networkx and pyvis, which we used to produce this visualization. For larger graphs, and graphs of large numbers of documents, we recommend a more robust solution for graph storage, such as Amazon Neptune.
Tabulating event structures
Lastly, you can always render the event structure produced by the API as a flat table, indicating, for example, the argument roles of the various participants in each event, as in the following table. The table demonstrates how Comprehend Events groups entity mentions and triggers into coreferential groups. You can use these textual mention groups to verify and analyze system predictions.
Setting up the Comprehend Events AWS CloudFormation stack
You can quickly try out this example for yourself by deploying our sample code into your own account from the provided AWS CloudFormation template. We’ve included all the necessary steps in a Jupyter notebook, so you can easily walk through creating the preceding visualizations and see how it all works. From there, you can easily modify it to run over other custom datasets, modify the results, ingest them into other systems, and build upon the solution. Complete the following steps:
- Choose Launch Stack:
- After the template loads in the AWS CloudFormation console, choose Next.
- For Stack name, enter a name for your deployment.
- Choose Next.
- Choose Next on the following page.
- Select the check box acknowledging this template will create IAM resources.
This allows the SageMaker notebook instance to talk with Amazon S3 and Amazon Comprehend.
- Choose Create stack.
- When stack creation is complete, browse to your notebook instances on the SageMaker console.
A new instance is already loaded with the example data and Jupyter notebook.
- Choose Open Jupyter for the
comprehend-events-blognotebook.
The data and notebook are already loaded on the instance. This was done through a SageMaker lifecycle configuration.
- Choose the
notebooksfolder. - Choose the
comprehend_events_finance_tutorial.ipynbnotebook.
- Step through the notebook to try Comprehend Events out yourself.
Applications using Comprehend Events
We have demonstrated applying Comprehend Events to a small set of documents and demonstrated visualizing the event structures found in a sample document. The power of Comprehend Events, however, lies in its ability to extract and structure business-relevant facts from large collections of unstructured documents. In this section, we discuss a few potential solutions that you could build on top of the foundation provided by Comprehend Events.
Knowledge graph construction
Business and financial services analysts need to visually explore event-based relationships among corporate entities, identifying potential patterns over large collections of data. Without a tool like Comprehend Events, you have to manually identify entities of interests and events in documents and manually enter them in network visualization tools for tracking. Comprehend Events allows you to populate knowledge graphs over large collections of data. You can store these graphs and search, for example, in Neptune and explore using network visualization tools without expensive manual extraction.
Semantic search
Analysts also need to find documents in which actors of interest participate in events of interest (at places, at times). The most common approach to this task involves enterprise search: using complex Boolean queries to find co-occurring strings that typically match your desired search patterns. Natural language is rich and highly variable, however, and even the best searches often miss key details in unstructured text. Comprehend Events allows you to populate a search index with event-argument associations, enriching free text search with extracted event data. You can process collections of documents with Comprehend Events, index the documents in Amazon Elasticsearch Service (Amazon ES) with the extracted event data, and enable field-based search over event-argument tuples in downstream applications.
Document triage
An additional application of Comprehend Events is simple filtration of large text collections for events of interest. This task is typically performed with a tool such as Amazon Comprehend customer classification, but requires hundreds or thousands of annotated training documents to produce a custom model. Comprehend Events allows developers without such training data to process a large collection of documents and detect financial events found in the event taxonomy. You can simply process batches of documents with the asynchronous API and route documents matching pre-defined event patterns to downstream applications.
Conclusion
This post has demonstrated the application and utility of Comprehend Events for information processing in the finance domain. This new feature gives you the ability to enrich your applications with close semantic analysis of financial events from unstructured text, all without any NLP model training or tuning. For more information, just check out our documentation or try out the above walkthrough for yourself in the Console or in our Jupyter notebook through CloudFormation or on Github. We’re exciting to hear your comments and questions in the comments section!
About the Authors
Graham Horwood is a data scientist at Amazon AI. His work focuses on natural language processing technologies for customers in the public and commercial sectors.
Ben Snively is an AWS Public Sector Specialist Solutions Architect. He works with government, non-profit, and education customers on big data/analytical and AI/ML projects, helping them build solutions using AWS.
Sameer Karnik is a Sr. Product Manager leading product for Amazon Comprehend, AWS’s natural language processing service.
Bringing your own R environment to Amazon SageMaker Studio
Amazon SageMaker Studio is the first fully integrated development environment (IDE) for machine learning (ML). With a single click, data scientists and developers can quickly spin up SageMaker Studio notebooks to explore datasets and build models. On October 27, 2020, Amazon released a custom images feature that allows you to launch SageMaker Studio notebooks with your own images.
SageMaker Studio notebooks provide a set of built-in images for popular data science and ML frameworks and compute options to run notebooks. The built-in SageMaker images contain the Amazon SageMaker Python SDK and the latest version of the backend runtime process, also called kernel. With the custom images feature, you can register custom built images and kernels, and make them available to all users sharing a SageMaker Studio domain. You can start by cloning and extending one of the example Docker files provided by SageMaker, or build your own images from scratch.
This post focuses on adding a custom R image to SageMaker Studio so you can build and train your R models with SageMaker. After attaching the custom R image, you can select the image in Studio and use R to access the SDKs using the RStudio reticulate package. For more information about R on SageMaker, see Coding with R on Amazon SageMaker notebook instances and R User Guide to Amazon SageMaker.
You can create images and image versions and attach image versions to your domain using the SageMaker Studio Control Panel, the AWS SDK for Python (Boto3), and the AWS Command Line Interface (AWS CLI)—for more information about CLI commands, see AWS CLI Command Reference. This post explains both AWS CLI and SageMaker console UI methods to attach and detach images to a SageMaker Studio domain.
Prerequisites
Before getting started, you need to meet the following prerequisites:
- Install the AWS CLI on your local machine. This post uses AWS CLI version 2. You should make necessary adjustments if you use a different AWS CLI version.
- Permissions to access the Amazon Elastic Container Registry (Amazon ECR). For more information, see Amazon ECR Managed Policies.
- Install Docker on your local machine. For more information, see Orientation and setup. This is necessary for building Docker images and pushing them to Amazon ECR.
- For instructions on building a container from a Studio development environment, see Using the Amazon SageMaker Studio Image Build CLI to build container images from your Studio notebooks and the SageMaker Docker Build GitHub repo.
- Dockerfile for the R image. A sample Dockerfile is provided in this post that you can customize for your own specific case, but you can also use your own Dockerfile.
- Building the R image from the Dockerfile installs dependencies that may be licensed under copyleft licenses such as GPLv3. You should review the license terms and make sure they are acceptable for your use case before proceeding to build this image.
- An AWS Identity and Access Management (IAM) role that has the
AmazonSageMakerFullAccesspolicy attached. If you have onboarded to SageMaker Studio, you can get the role from the Studio Summary section on the SageMaker Studio Control Panel. - A SageMaker Studio domain. For instructions, see Onboard to Amazon SageMaker Studio.
Creating your Dockerfile
Before attaching your image to Studio, you need to build a Docker image using a Dockerfile. You can build a customized Dockerfile using base images or other Docker image repositories, such as Jupyter Docker-stacks repository, and use or revise the ones that fit your specific need.
SageMaker maintains a repository of sample Docker images that you can use for common use cases (including R, Julia, Scala, and TensorFlow). This repository contains examples of Docker images that are valid custom images for Jupyter KernelGateway Apps in SageMaker Studio. These custom images enable you to bring your own packages, files, and kernels for use within SageMaker Studio.
For more information about the specifications that apply to the container image that is represented by a SageMaker image version, see Custom SageMaker image specifications.
For this post, we use the sample R Dockerfile. This Dockerfile takes the base Python 3.6 image and installs R system library prerequisites, conda via Miniconda, and R packages and Python packages that are usable via reticulate. You can create a file named Dockerfile using the following script and copy it to your installation folder. You can customize this Dockerfile for your specific use case and install additional packages.
# This project is licensed under the terms of the Modified BSD License
# (also known as New or Revised or 3-Clause BSD), as follows:
# Copyright (c) 2001-2015, IPython Development Team
# Copyright (c) 2015-, Jupyter Development Team
# All rights reserved.
FROM python:3.6
ARG NB_USER="sagemaker-user"
ARG NB_UID="1000"
ARG NB_GID="100"
# Setup the "sagemaker-user" user with root privileges.
RUN
apt-get update &&
apt-get install -y sudo &&
useradd -m -s /bin/bash -N -u $NB_UID $NB_USER &&
chmod g+w /etc/passwd &&
echo "${NB_USER} ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers &&
# Prevent apt-get cache from being persisted to this layer.
rm -rf /var/lib/apt/lists/*
USER $NB_UID
# Make the default shell bash (vs "sh") for a better Jupyter terminal UX
ENV SHELL=/bin/bash
NB_USER=$NB_USER
NB_UID=$NB_UID
NB_GID=$NB_GID
HOME=/home/$NB_USER
MINICONDA_VERSION=4.6.14
CONDA_VERSION=4.6.14
MINICONDA_MD5=718259965f234088d785cad1fbd7de03
CONDA_DIR=/opt/conda
PATH=$CONDA_DIR/bin:${PATH}
# Heavily inspired from https://github.com/jupyter/docker-stacks/blob/master/r-notebook/Dockerfile
USER root
# R system library pre-requisites
RUN apt-get update &&
apt-get install -y --no-install-recommends
fonts-dejavu
unixodbc
unixodbc-dev
r-cran-rodbc
gfortran
gcc &&
rm -rf /var/lib/apt/lists/* &&
mkdir -p $CONDA_DIR &&
chown -R $NB_USER:$NB_GID $CONDA_DIR &&
# Fix for devtools https://github.com/conda-forge/r-devtools-feedstock/issues/4
ln -s /bin/tar /bin/gtar
USER $NB_UID
ENV PATH=$CONDA_DIR/bin:${PATH}
# Install conda via Miniconda
RUN cd /tmp &&
curl --silent --show-error --output miniconda-installer.sh https://repo.anaconda.com/miniconda/Miniconda3-${MINICONDA_VERSION}-Linux-x86_64.sh &&
echo "${MINICONDA_MD5} *miniconda-installer.sh" | md5sum -c - &&
/bin/bash miniconda-installer.sh -f -b -p $CONDA_DIR &&
rm miniconda-installer.sh &&
conda config --system --prepend channels conda-forge &&
conda config --system --set auto_update_conda false &&
conda config --system --set show_channel_urls true &&
conda install --quiet --yes conda="${CONDA_VERSION%.*}.*" &&
conda update --all --quiet --yes &&
conda clean --all -f -y &&
rm -rf /home/$NB_USER/.cache/yarn
# R packages and Python packages that are usable via "reticulate".
RUN conda install --quiet --yes
'r-base=4.0.0'
'r-caret=6.*'
'r-crayon=1.3*'
'r-devtools=2.3*'
'r-forecast=8.12*'
'r-hexbin=1.28*'
'r-htmltools=0.4*'
'r-htmlwidgets=1.5*'
'r-irkernel=1.1*'
'r-rmarkdown=2.2*'
'r-rodbc=1.3*'
'r-rsqlite=2.2*'
'r-shiny=1.4*'
'r-tidyverse=1.3*'
'unixodbc=2.3.*'
'r-tidymodels=0.1*'
'r-reticulate=1.*'
&&
pip install --quiet --no-cache-dir
'boto3>1.0<2.0'
'sagemaker>2.0<3.0' &&
conda clean --all -f -y
WORKDIR $HOME
USER $NB_UIDSetting up your installation folder
You need to create a folder on your local machine and add the following files in that folder:
.
├── Dockerfile
├── app-image-config-input.json
├── create-and-attach-image.sh
├── create-domain-input.json
└── default-user-settings.jsonIn the following scripts, the Amazon Resource Names (ARNs) should have a format similar to:
arn:partition:service:region:account-id:resource-id
arn:partition:service:region:account-id:resource-type/resource-id
arn:partition:service:region:account-id:resource-type:resource-idDockerfileis the Dockerfile that you created in the previous step.
- Create a file named
app-image-config-input.jsonwith the following content:{ "AppImageConfigName": "custom-r-image-config", "KernelGatewayImageConfig": { "KernelSpecs": [ { "Name": "ir", "DisplayName": "R (Custom R Image)" } ], "FileSystemConfig": { "MountPath": "/home/sagemaker-user", "DefaultUid": 1000, "DefaultGid": 100 } }
- Create a file named
default-user-settings.jsonwith the following content. If you’re adding multiple custom images, add to the list ofCustomImages.{ "DefaultUserSettings": { "KernelGatewayAppSettings": { "CustomImages": [ { "ImageName": "custom-r", "AppImageConfigName": "custom-r-image-config" } ] } } }
- Create one last file in your installation folder named
create-and-attach-image.shusing the following bash script. The script runs the following in order:- Creates a repository named smstudio-custom in Amazon ECR and logs into that repository
- Builds an image using the Dockerfile and attaches a tag to the image r
- Pushes the image to Amazon ECR
- Creates an image for SageMaker Studio and attaches the Amazon ECR image to that image
- Creates an
AppImageConfigforthis image usingapp-image-config-input.json# Replace with your AWS account ID and your Region, e.g. us-east-1, us-west-2 ACCOUNT_ID = <AWS ACCOUNT ID> REGION = <STUDIO DOMAIN REGION> # create a repository in ECR, and then login to ECR repository aws --region ${REGION} ecr create-repository --repository-name smstudio-custom aws ecr --region ${REGION} get-login-password | docker login --username AWS --password-stdin ${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/smstudio-custom # Build the docker image and push to Amazon ECR (modify image tags and name as required) $(aws ecr get-login --region ${REGION} --no-include-email) docker build . -t smstudio-r -t ${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/smstudio-custom:r docker push ${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/smstudio-custom:r # Using with SageMaker Studio ## Create SageMaker Image with the image in ECR (modify image name as required) ROLE_ARN = "<YOUR EXECUTION ROLE ARN>" aws sagemaker create-image --region ${REGION} --image-name custom-r --role-arn ${ROLE_ARN} aws sagemaker create-image-version --region ${REGION} --image-name custom-r --base-image ${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/smstudio-custom:r ## Create AppImageConfig for this image (modify AppImageConfigName and ## KernelSpecs in app-image-config-input.json as needed) ## note that 'file://' is required in the file path aws sagemaker create-app-image-config --region ${REGION} --cli-input-json file://app-image-config-input.json
Updating an existing SageMaker Studio domain with a custom image
If you already have a Studio domain, you don’t need to create a new domain, and can easily update your existing domain by attaching the custom image. You can do this either using the AWS CLI for Amazon SageMaker or the SageMaker Studio Control Panel (which we discuss in the following sections). Before going to the next steps, make sure your domain is in Ready status, and get your Studio domain ID from the Studio Control Panel. The domain ID should be in d-xxxxxxxx format.
Using the AWS CLI for SageMaker
In the terminal, navigate to your installation folder and run the following commands. This makes the bash scrip executable:
chmod +x create-and-attach-image.shThen execute the following command in terminal:
./create-and-attach-image.shAfter you successfully run the bash script, you need update your existing domain by executing the following command in the terminal. Make sure you provide your domain ID and Region.
aws sagemaker update-domain --domain-id <DOMAIN_ID>
--region <REGION_ID>
--cli-input-json file://default-user-settings.jsonAfter executing this command, your domain status shows as Updating for a few seconds and then shows as Ready again. You can now open Studio.
When in the Studio environment, you can use the Launcher to launch a new activity, and should see the custom-r (latest) image listed in the dropdown menu under Select a SageMaker image to launch your activity.
Using the SageMaker console
Alternatively, you can update your domain by attaching the image via the SageMaker console. The image that you created is listed on the Images page on the console.
- To attach this image to your domain, on the SageMaker Studio Control Panel, under Custom images attached to domain, choose Attach image.
- For Image source, choose Existing image.
- Choose an existing image from the list.
- Choose a version of the image from the list.
- Choose Next.
- Choose the IAM role. For more information, see Create a custom SageMaker image (Console).
- Choose Next.
- Under Studio configuration, enter or change the following settings. For information about getting the kernel information from the image, see DEVELOPMENT in the SageMaker Studio Custom Image Samples GitHub repo.
- For EFS mount path, enter the path within the image to mount the user’s Amazon Elastic File System (Amazon EFS) home directory.
- For Kernel name, enter the name of an existing kernel in the image.
- (Optional) For Kernel display name, enter the display name for the kernel.
- Choose Add kernel.
- (Optional) For Configuration tags, choose Add new tag and add a configuration tag.
For more information, see the Kernel discovery and User data sections of Custom SageMaker image specifications.
- Choose Submit.
- Wait for the image version to be attached to the domain.
While attaching, your domain status is in Updating. When attached, the version is displayed in the Custom images list and briefly highlighted, and your domain status shows as Ready.
The SageMaker image store automatically versions your images. You can select a pre-attached image and choose Detach to detach the image and all versions, or choose Attach image to attach a new version. There is no limit to the number of versions per image or the ability to detach images.
Using a custom image to create notebooks
When you’re done updating your Studio domain with the custom image, you can use that image to create new notebooks. To do so, choose your custom image from the list of images in the Launcher. In this example, we use custom-r. This shows the list of kernels that you can use to create notebooks. Create a new notebook with the R kernel.
If this is the first time you’re using this kernel to create a notebook, it may take about a minute to start the kernel, and the Kernel Starting message appears on the lower left corner of your Studio. You can write R scripts while the kernel is starting but can only run your script after your kernel is ready. The notebook is created with a default ml.t3.medium instance attached to it. You can see R (Custom R Image) kernel and the instance type on the upper right corner of the notebook. You can change ML instances on the fly in SageMaker Studio. You can also right-size your instances for different workloads. For more information, see Right-sizing resources and avoiding unnecessary costs in Amazon SageMaker.
To test the kernel, enter the following sample R script in the first cell and run the script. This script tests multiple aspects, including importing libraries, creating a SageMaker session, getting the IAM role, and importing data from public repositories.
The abalone dataset in this post is from Dua, D. and Graff, C. (2019). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science (http://archive.ics.uci.edu/ml/datasets/Abalone).
# Simple script to test R Kernel in SageMaker Studio
# Import reticulate, readr and sagemaker libraries
library(reticulate)
library(readr)
sagemaker <- import('sagemaker')
# Create a sagemaker session
session <- sagemaker$Session()
# Get execution role
role_arn <- sagemaker$get_execution_role()
# Read a csv file from UCI public repository
# Dua, D. and Graff, C. (2019). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml].
# Irvine, CA: University of California, School of Information and Computer Science
data_file <- 'http://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.data'
# Copy data to a dataframe, rename columns, and show dataframe head
abalone <- read_csv(file = data_file, col_names = FALSE, col_types = cols())
names(abalone) <- c('sex', 'length', 'diameter', 'height', 'whole_weight', 'shucked_weight', 'viscera_weight', 'shell_weight', 'rings')
head(abalone)If the image is set up properly and the kernel is running, the output should look like the following screenshot.
Listing, detaching, and deleting custom images
If you want to see the list of custom images attached to your Studio, you can either use the AWS CLI or go to SageMaker console to view the attached image in the Studio Control Panel.
Using the AWS CLI for SageMaker
To view your list of custom images via the AWS CLI, enter the following command in the terminal (provide the Region in which you created your domain):
aws sagemaker list-images --region <region-id> The response includes the details for the attached custom images:
{
"Images": [
{
"CreationTime": "xxxxxxxxxxxx",
"ImageArn": "arn:aws:sagemaker:us-east-2:XXXXXXX:image/custom-r",
"ImageName": "custom-r",
"ImageStatus": "CREATED",
"LastModifiedTime": "xxxxxxxxxxxxxx"
},
....
]
}If you want to detach or delete an attached image, you can do it on the SageMaker Studio Control Panel (see Detach a custom SageMaker image). Alternatively, use the custom image name from your default-user-settings.json file and rerun the following command to update the domain by detaching the image:
aws sagemaker update-domain --domain-id <YOUR DOMAIN ID>
--cli-input-json file://default-user-settings.jsonThen, delete the app image config:
aws sagemaker delete-app-image-config
--app-image-config-name custom-r-image-configDelete the SageMaker image, which also deletes all image versions. The container images in Amazon ECR that are represented by the image versions are not deleted.
aws sagemaker delete-image
--region <region-id>
--image-name custom-rAfter deleting the image, it will not be listed under custom images in SageMaker Studio. For more information, see Clean up resources.
Using the SageMaker console
You can also detach (and delete) images from your domain via the Studio Control Panel UI. To do so, under Custom images attached to domain, select the image and choose Detach. You have the option to also delete all versions of the image from your domain. This detaches the image from the domain.
Getting logs in Amazon CloudWatch
You can also get access to SageMaker Studio logs in Amazon CloudWatch, which you can use for troubleshooting your environment. The metrics are captured under the /aws/sagemaker/studio namespace.
To access the logs, on the CloudWatch console, choose CloudWatch Logs. On the Log groups page, enter the namespace to see logs associated with the Jupyter server and the kernel gateway.
For more information, see Log Amazon SageMaker Events with Amazon CloudWatch.
Conclusion
This post outlined the process of attaching a custom Docker image to your Studio domain to extend Studio’s built-in images. We discussed how you can update an existing domain with a custom image using either the AWS CLI for SageMaker or the SageMaker console. We also explained how you can use the custom image to create notebooks with custom kernels.
For more information, see the following resources:
- Bringing your own custom container image to Amazon SageMaker Studio notebooks
- R User Guide to Amazon SageMaker
- Coding with R on Amazon SageMaker notebook instances
- Amazon SageMaker Examples GitHub repo
About the Authors
Nick Minaie is an Artificial Intelligence and Machine Learning (AI/ML) Specialist Solution Architect, helping customers on their journey to well-architected machine learning solutions at scale. In his spare time, Nick enjoys family time, abstract painting, and exploring nature.
Sam Liu is a product manager at Amazon Web Services (AWS). His current focus is the infrastructure and tooling of machine learning and artificial intelligence. Beyond that, he has 10 years of experience building machine learning applications in various industries. In his spare time, he enjoys making short videos for technical education or animal protection.
Alexa & Friends features Ruhi Sarikaya, Alexa AI director of applied science
Newly named IEEE Fellow discusses his experience in the field of conversational AI, and the ways he and his team are working to make Alexa more intelligent.Read More
Automatically assessing conversations with Alexa
Model for estimating customer satisfaction with interactions that span multiple domains improves on predecessors by 27%.Read More
Building natural conversation flows using context management in Amazon Lex
Understanding the direction and context of an ever-evolving conversation is beneficial to building natural, human-like conversational interfaces. Being able to classify utterances as the conversation develops requires managing context across multiple turns. Consider a caller who asks their financial planner for insights regarding their monthly expenses: “What were my expenses this year?” They may also ask for more granular information, such as “How about for last month?” As the conversation progresses, the bot needs to understand if the context is changing and adjust its responses accordingly.
Amazon Lex is a service for building conversational interface in voice and text. Previously, you had to write code to manage context via session attributes. Depending on the intent, the code had to orchestrate the invocation of the next intent. As the conversation complexity and the intent count increased, managing the orchestration could become more cumbersome.
Starting today, Amazon Lex supports context management natively, so you can manage the context directly without the need for custom code. As initial prerequisite intents are filled, you can create contexts to invoke related intents. This simplifies bot design and expedites the creation of conversational experiences.
Use case
This post uses the following conversation to model a bot for financial planning:
User: What was my income in August?
Agent: Your income in August was $2345.
User: Ok. How about September?
Agent: Your income in September was $4567.
User: What were my expenses in July?
Agent: Your expenses for July were $123.
User: Ok thanks.
Building the Amazon Lex bot FinancialPlanner
In this post, we build an Amazon Lex bot called FinancialPlanner, which is available for download. Complete the following steps:
- Create the following intents:
- ExpensesIntent – Elicits information, such as account ID and period, and provides expenses detail
- IncomeIntent – Elicits information, such as account ID and period, and provides income detail
- ExpensesFollowup – Invoked after expenses intent to respond to a follow-up query, such as “How about [expenses] last month?”
- IncomeFollowup – Invoked after income intent to respond to a follow-up query about income, such as “How about [income] last month?”
- Fallback – Captures any input that the bot can’t process by the configured intents
- Set up context tags for the expenses intents.
The context management feature defines input tags and output tags that the bot developer can set. You use these tags to manage the conversation flow. For our use case, we set expenses as the output context tag in ExpensesIntent. We also use this as the input context for ExpensesFollowupIntent. We can also configure the output tag with a timeout, measured by conversation turns or seconds since the initial intent was invoked.
The following screenshot shows the Context configuration section on the Amazon Lex console.
The following screenshot shows the specific parameters for the expenses tag.
- Set up context tags for the income intents.
Similar to expenses, we now set the context for income intents. For IncomeIntent, set the output context tag as income. We use this context as the input context for IncomeFollowupIntent.
- Build the bot and test it on the Amazon Lex console.
To test the bot, provide the input “What were my expenses this year” followed by “How about last month?” For the second request, the bot selects ExpensesFollowupIntent because the expenses context is active. Alternatively, if you start with “What was my income this year?” followed by “How about last year?”, the bot invokes the IncomeFollowupIntent because the income context is active.
The following screenshot illustrates how the context tags are used to invoke the appropriate intent.
You can configure the behavior of the context attributes by editing the threshold. Editing the number of turns sets the limit for the number of interactions with the bot, and the number of seconds is from the original input tag being set. As long as the intent with the output tag occurs before the turn- or time-based timeout, the user can invoke the intent based on the input context.
Along with the context management feature, you can also set default slot values. You can set the slots to populate from a context, a session attribute, or a value. In our sample bot model, the {month} slot in ExpensesIntent is set to August as the default slot value.
Conclusion
With the new Amazon Lex context management feature, you can easily orchestrate when to enable intents based on prior intents, and pass specific user data values from one intent to another. This capability allows you to create sophisticated, multi-turn conversational experiences without having to write custom code. Context carry-over, along with default slot values, simplifies bot development and allows you to easily create more natural, conversational user experiences. For more information, see Setting Intent Context documentation.
About the Authors
Blake DeLee is a Rochester, NY-based conversational AI consultant with AWS Professional Services. He has spent five years in the field of conversational AI and voice, and has experience bringing innovative solutions to dozens of Fortune 500 businesses. Blake draws on a wide-ranging career in different fields to build exceptional chatbot and voice solutions.
As a Product Manager on the Amazon Lex team, Harshal Pimpalkhute spends his time trying to get machines to engage (nicely) with humans.
Esther Lee is a Product Manager for AWS Language AI Services. She is passionate about the intersection of technology and education. Out of the office, Esther enjoys long walks along the beach, dinners with friends and friendly rounds of Mahjong.