The Amazon Scholar received the award for his seminal and sustained contributions to the fields of computer graphics and visual computing.Read More
Automate vending Amazon SageMaker notebooks with Amazon EventBridge and AWS Lambda
Having an environment capable of delivering Amazon SageMaker notebook instances quickly allows data scientists and business analysts to efficiently respond to organizational needs. Data is the lifeblood of an organization, and analyzing that data efficiently provides useful insights for businesses. A common issue that organizations encounter is creating an automated pattern that enables development teams to launch AWS services. Organizations want to enable their developers to launch resources as they need them, but in a centralized and secure fashion.
This post demonstrates how to centralize the management of SageMaker instance notebooks using AWS services including AWS CloudFormation, AWS Serverless Application Model (AWS SAM), AWS Service Catalog, Amazon EventBridge, AWS Systems Manager Parameter Store, Amazon API Gateway, and AWS Lambda. We walk through how to use these AWS services to automate the process of vending SageMaker notebooks to end-users.
Solution overview
In our solution, a notebook user requests a notebook instance using AWS Service Catalog. The request is processed by AWS CloudFormation, which delivers the notebook instance. EventBridge monitors the AWS Service Catalog API for completion of the notebook instance resource provisioning. An event-based rule in EventBridge calls the Lambda event processor, which runs a Lambda function returning the presigned URL.
The following architectural diagram illustrates the infrastructure state as defined in the CloudFormation templates.
The process consists of the following steps:
- A user requests a new notebook via the AWS Service Catalog console.
- AWS Service Catalog launches a CloudFormation stack.
- AWS CloudFormation launches the SageMaker notebook.
- A SageMaker notebook is now running.
- An EventBridge function is triggered when a new AWS Service Catalog product is launched.
- The Amazon CloudWatch event invokes a Lambda function that generates the presigned URL and a user-specific SSM parameter.
- A user requests a new presigned URL.
- A Lambda function generates a new presigned URL and updates the user’s SSM parameter with the new URL.
Prerequisites
To implement this solution, you must have the following prerequisites:
- An AWS account with local credentials properly configured (typically under
~/.aws/credentials). - An AWS Identity and Access Management (IAM) role configured with administrative privileges.
- The latest version of the AWS Command Line Interface (AWS CLI). For more information, refer to Installing or updating the latest version of the AWS CLI.
- A Git client to clone the source code located here.
- The latest version of the AWS SAM CLI.
Deploy resources with AWS CloudFormation
To create your resources with AWS CloudFormation, complete the following steps:
- Deploy the
s3-iam-configCloudFormation template:
The output should look like the following code:
The template creates an Amazon Simple Storage Service (Amazon S3) bucket.
- Run the following command to get the S3 bucket name generated in the previous step:
The output should look like the following:
- Run the following command using the output from the previous step (update the bucket name):
The output should look like the following:
- Open the
parameters/service-catalog-params.jsonfile and update theS3BucketNameparameter to the bucket name from the previous step. Update theUserIAMPrincipalwith the ARN of the IAM role you’re using for this demo. - Deploy the
service-catalogCloudFormation template:
The output should look like the following:
Deploy resources with AWS SAM
To deploy resources with AWS SAM, complete the following steps:
- Change your directory to the
lambdadirectory: - Build the application:
The output should look like the following:
- Deploy the application:
- Respond to the questions in the CLI as shown in the following code:
The output should look like the following:
Test the solution
Now that you have deployed the solution, let’s test the workflow.
- On the AWS Service Catalog console, under Administration in the navigation pane, choose Portfolios.
- Choose your SageMaker notebook.
- Choose Launch product.
- At the bottom of the page, choose Launch product.
You should see a page similar to the following screenshot.
- Wait a few moments for the status to show as
Available.
- Open your terminal and run the following command to get the presigned URL from Parameter Store:
The output should look like the following:
EventBridge rule
EventBridge is configured with an event rule to process an API response for the AWS Service Catalog API. This rule is configured to pass the notebook instance state so that you can use Lambda to return a presigned URL response as a triggered action. The event rule is configured as follows:
The following screenshot of the EventBridge console shows your event rule.
The AWS CloudTrail API is being monitored using the event source for servicecatalog.amazonaws.com. The monitored event name is ProvisionProduct. Monitoring this event allows you to take effective action in response to AWS Service Catalog reporting back the successful delivery state of the notebook instance. When a ProvisionProduct event occurs, a Lambda function called DemoEventBridgeFunction is invoked, which returns a presigned URL to the end-user.
Lambda function for returning presigned notebook instance URLs
To ensure secure access to user-requested notebooks via AWS Service Catalog, a presigned URL is created and returned to the user. This provides a secure method of accessing the notebook instance and performing business critical functions. For this purpose, we use the EventBridgeServiceCatalogFunction function, which uses a waiter for the notebook instance state to become available. Waiters provide a means of polling a service and suspending the execution of a task until a specific condition is met. When it’s ready, the function generates a presigned URL. Finally, the function creates an SSM parameter with the generated presigned URL. The SSM parameter uses the following pattern: /SageMaker/Notebooks/%s-Notebook"%user_name/. This allows us to create a common namespace for all our SageMaker notebook SSM parameters while keeping them unique based off of user_name.
Presigned URLs have a defined expiration. The Lambda function deploys notebooks with a session expiration of 12 hours. Because of this, developers need to generate a new presigned URL when their existing presigned URL expires. The RefreshURLFunction accomplishes this by allowing users to invoke the function from calling the API Gateway. Developers can invoke this function and pass their notebook name, and it returns a presigned URL. When the RefreshURLFunction is complete, a user can make a call to Parameter Store, get the new presigned URL, and then access their notebook.
- Get the
RefreshURLFunctionAPI Gateway URL with the following code:
The output should look like the following:
- Invoke the function
RefreshURLFunctionby calling the API Gateway. Updateinput_urlwith the URL from the previous step:
The output should look like the following:
- Open a browser and navigate to the
PreSignedURLfrom the previous step.
The webpage should look like the following screenshot.
Conclusion
In this post, we demonstrated how to deploy the infrastructure components for a SageMaker notebook instance environment using AWS CloudFormation. We then illustrated how to use EventBridge to return the notebook instance state from the AWS Service Catalog API. Lastly, we showed how to use a Lambda function to return the presigned notebook instance URL for accessing the delivered resource. For more information, see the Amazon SageMaker Developer Guide. Thank you for reading!
About the Authors
Joe Keating is a Senior Customer Delivery Architect in Professional Services at Amazon Web Services. He works with AWS customers to design and implement a variety of solutions in the AWS Cloud. Joe enjoys cooking with a glass or two of wine and achieving mediocrity on the golf course.
Matt Hedges is a Cloud Application Architect at Amazon Web Services. He works closely with customers to align technology needs with business drivers to deliver their applications on AWS. With a focus on migrations and modernization, Matt works with enterprise customers around the world to pioneer changes that unlock the full potential of the cloud. Matt enjoys spending time with family, playing musical instruments, cooking, playing video games, fixing old cars, and learning new things.
Virginia Chu is a Senior DevSecOps Architect in Professional Services at Amazon Web Services. She works with enterprise-scale customers around the globe to design and implement a variety of solutions in the AWS Cloud.
Run text classification with Amazon SageMaker JumpStart using TensorFlow Hub and Hugging Face models
In December 2020, AWS announced the general availability of Amazon SageMaker JumpStart, a capability of Amazon SageMaker that helps you quickly and easily get started with machine learning (ML). JumpStart provides one-click fine-tuning and deployment of a wide variety of pre-trained models across popular ML tasks, as well as a selection of end-to-end solutions that solve common business problems. These features remove the heavy lifting from each step of the ML process, making it easier to develop high-quality models and reducing time to deployment.
All JumpStart content was previously available only through Amazon SageMaker Studio, which provides a user-friendly graphical interface to interact with the feature. Recently, we also announced the launch of easy-to-use JumpStart APIs as an extension of the SageMaker Python SDK, allowing you to programmatically deploy and fine-tune a vast selection of JumpStart-supported pre-trained models on your own datasets. This launch unlocks the usage of JumpStart capabilities in your code workflows, MLOps pipelines, and anywhere else you are interacting with SageMaker via SDK.
This post is the second in a series on using JumpStart for specific tasks. In the first post, we showed how you can run image classification use cases on JumpStart. In this post, we provide a step-by-step walkthrough on how to fine-tune and deploy a text classification model, using trained models from TensorFlow Hub. We explore two ways of obtaining the same result: via JumpStart’s graphical interface on Studio, and programmatically through JumpStart APIs. While not used in this blog post, Amazon Comprehend is a natural language processing (NLP) service that uses machine learning (ML) to find insights and relationships like people, places, sentiments, and topics in text. In a fully-managed experience, customers can use Comprehend’s custom classification API to train a custom classification model to recognize the classes that are of interest and then use that model to classify documents into your own categories.
If you want to jump straight into the JumpStart API code we explain in this post, you can refer to the following sample Jupyter notebooks:
- Introduction to JumpStart – Text Classification
- Introduction to JumpStart – Sentence Pair Classification
JumpStart overview
JumpStart is a multi-faceted product that includes different capabilities to help get you quickly started with ML on SageMaker. At the time of writing, JumpStart enables you to do the following:
- Deploy pre-trained models for common ML tasks – JumpStart enables you to address common ML tasks with no development effort by providing easy deployment of models pre-trained on large, publicly available datasets. The ML research community has put a large amount of effort into making a majority of recently developed models publicly available for use. JumpStart hosts a collection of over 300 models, spanning the 15 most popular ML tasks such as object detection, text classification, and text generation, making it easy for beginners to use them. These models are drawn from popular model hubs such as TensorFlow, PyTorch, Hugging Face, and MXNet Hub.
- Fine-tune pre-trained models – JumpStart allows you to fine-tune pre-trained models with no need to write your own training algorithm. In ML, the ability to transfer the knowledge learned in one domain to another domain is called transfer learning. You can use transfer learning to produce accurate models on your smaller datasets, with much lower training costs than the ones involved in training the original model. JumpStart also includes popular training algorithms based on LightGBM, CatBoost, XGBoost, and Scikit-learn, which you can train from scratch for tabular regression and classification.
- Use pre-built solutions – JumpStart provides a set of 17 solutions for common ML use cases, such as demand forecasting and industrial and financial applications, which you can deploy with just a few clicks. Solutions are end-to-end ML applications that string together various AWS services to solve a particular business use case. They use AWS CloudFormation templates and reference architectures for quick deployment, which means they’re fully customizable.
- Refer to notebook examples for SageMaker algorithms – SageMaker provides a suite of built-in algorithms to help data scientists and ML practitioners get started with training and deploying ML models quickly. JumpStart provides sample notebooks that you can use to quickly use these algorithms.
- Review training videos and blogs – JumpStart also provides numerous blog posts and videos that teach you how to use different functionalities within SageMaker.
JumpStart accepts custom VPC settings and AWS Key Management Service (AWS KMS) encryption keys, so you can use the available models and solutions securely within your enterprise environment. You can pass your security settings to JumpStart within Studio or through the SageMaker Python SDK.
Transformer models and the importance of fine-tuning
The attention-based Transformer architecture has become the de-facto standard for the state-of-the-art natural language processing (NLP) models. In 2018, the famous BERT model was born from an adapted Transformer encoder, pre-trained on GBs of unlabeled English text from Wikipedia and other public resources. BERT was incredibly useful for creating contextualized text representations, which you could then use in many downstream tasks by fine-tuning the model. Since then, many variants of the BERT model have been developed by way of architecture, pre-training schema, or pre-training dataset changes.
Fine-tuning means to use a model that has been pre-trained on a given task and train it again, this time for a specific task that is different but related (and on your specific data). This practice is also typically referred to as transfer learning—literally meaning to transfer knowledge gained on one task to another. Typically, Transformer-based models are pre-trained on massive amounts of unlabeled data, and comparatively much smaller labeled datasets are then used for fine-tuning. Being able to leverage the large computational investments made to pre-train such models for downstream tasks (commonly made open-source) has been one of the most important factors in the growth of NLP as a field in previous years. However, as new models get larger and more complex, so does the development effort required to fine-tune and deploy them efficiently. In this post, we show you how to use JumpStart to fine-tune a BERT model with little to no development effort involved.
Text classification
Sentiment analysis is one of the many tasks under the umbrella of text classification. It consists of predicting what sentiment should be assigned to a specific passage of text, with varying degrees of granularity. Typical applications include social media monitoring, customer support management, and analyzing customer feedback.
The input is a directory containing a data.csv file. The first column is the label (an integer between 0 and the number of classes in the dataset), and the second column is the corresponding passage of text. This means that you could even use a dataset with more degrees of sentiment than the original—for example, very negative (0), negative (1), neutral (2), positive (3), very positive (4). The following is an example of a data.csv file corresponding to the SST2 (Stanford Sentiment Treebank) dataset, and shows values in its first two columns. Note that the file shouldn’t have any header.
| Column 1 | Column 2 |
| 0 | hide new secretions from the parental units |
| 0 | contains no wit , only labored gags |
| 1 | that loves its characters and communicates something rather beautiful about human nature |
| 0 | remains utterly satisfied to remain the same throughout |
| 0 | on the worst revenge-of-the-nerds clichés the filmmakers could dredge up |
| 0 | that ‘s far too tragic to merit such superficial treatment |
| 1 | demonstrates that the director of such hollywood blockbusters as patriot games can still turn out a small , personal film with an emotional wallop . |
The SST21,2 dataset is downloaded from TensorFlow. Apache 2.0 License. Dataset Homepage.
Sentence pair classification
Sentence pair classification consists of predicting a class for a pair of sentences, which forces the model to learn semantic dependencies between sentence pairs. Among these are typically textual entailment (does the first sentence come before the second originally?), paraphrasing (are both sentences just differently worded versions of one another?), and others.
The input is a directory containing a data.csv file. The first column in the file should have integer class labels between 0 and the number of classes. The second and third columns should contain the first and second sentence corresponding to that row. The following is an example of a data.csv file for the QNLI dataset, and shows values in its first three columns. Note that the file shouldn’t have any header.
| Column 1 | Column 2 | Column 3 |
| 0 | What is the Grotto at Notre Dame? | Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. |
| 1 | What is the Grotto at Notre Dame? | It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. |
| 0 | What sits on top of the Main Building at Notre Dame? | Atop the Main Building’s gold dome is a golden statue of the Virgin Mary. |
| 1 | What sits on top of the Main Building at Notre Dame? | Next to the Main Building is the Basilica of the Sacred Heart. |
| 0 | When did the Scholastic Magazine of Notre dame begin publishing? | Begun as a one-page journal in September 1876, the Scholastic magazine is issued twice monthly and claims to be the oldest continuous collegiate publication in the United States. |
| 1 | When did the Scholastic Magazine of Notre dame begin publishing? | The newspapers have varying publication interests, with The Observer published daily and mainly reporting university and other news, and staffed by students from both Notre Dame and Saint Mary’s College. |
| 0 | In what year did the student paper Common Sense begin publication at Notre Dame? | In 1987, when some students believed that The Observer began to show a conservative bias, a liberal newspaper, Common Sense was published. |
The QNLI3.4 dataset is downloaded from TensorFlow. Apache 2.0 License. Dataset Homepage. CC BY-SA 4.0 License.
Solution overview
The following sections provide a step-by-step demo to perform sentiment analysis with JumpStart, both via the Studio UI and via JumpStart APIs. The workflow for sentence pair classification is almost identical, and we describe the changes required for that task.
We walk through the following steps:
- Access JumpStart through the Studio UI:
- Fine-tune the pre-trained model.
- Deploy the fine-tuned model.
- Use JumpStart programmatically with the SageMaker Python SDK:
- Fine-tune the pre-trained model.
- Deploy the fine-tuned model.
Access JumpStart through the Studio UI
In this section, we demonstrate how to train and deploy JumpStart models through the Studio UI.
Fine-tune the pre-trained model
The following video shows you how to find a pre-trained text classification model on JumpStart and fine-tune it on the task of sentiment analysis. The model page contains valuable information about the model, how to use it, expected data format, and some fine-tuning details.
For demonstration purposes, we fine-tune the model using the dataset provided by default, which is the Stanford Sentiment Treebank v2 (SST2) dataset. Fine-tuning on your own dataset involves taking the correct formatting of data (as explained on the model page), uploading it to Amazon Simple Storage Service (Amazon S3), and specifying its location in the data source configuration.
We use the sane hyperparameter values set by default (number of epochs, learning rate, and batch size). We also use a GPU-backed ml.p3.2xlarge as our SageMaker training instance.
You can monitor your training job running directly on the Studio console, and are notified upon its completion.
For sentence pair classification, instead of searching for text classification models in the JumpStart search bar, search for sentence pair classification. For example, choose Bert Base Uncased from the resulting model selection. Apart from the dataset format, the rest of the workflow is identical between text classification and sentence pair classification.
Deploy the fine-tuned model
After training is complete, you can deploy the fine-tuned model from the same page that holds the training job details. To deploy our model, we pick a different instance type, ml.g4dn.xlarge. It still provides the GPU acceleration needed for low inference latency, but at a lower price point. After you configure the SageMaker hosting instance, choose Deploy. It may take 5–10 minutes until your persistent endpoint is up and running.
After a few minutes, your endpoint is operational and ready to respond to inference requests!
To accelerate your time to inference, JumpStart provides a sample notebook that shows you how to run inference on your freshly deployed endpoint. Choose Open Notebook under Use Endpoint from Studio.
Use JumpStart programmatically with the SageMaker SDK
In the preceding sections, we showed how you can use the JumpStart UI to fine-tune and deploy a model interactively, in a matter of a few clicks. However, you can also use JumpStart’s models and easy fine-tuning programmatically by using APIs that are integrated into the SageMaker SDK. We now go over a quick example of how you can replicate the preceding process. All the steps in this demo are available in the accompanying notebooks Introduction to JumpStart – Text Classification and Introduction to JumpStart – Sentence Pair Classification.
Fine-tune the pre-trained model
To fine-tune a selected model, we need to get that model’s URI, as well as that of the training script and the container image used for training. Thankfully, these three inputs depend solely on the model name, version (for a list of the available models, see JumpStart Available Model Table), and the type of instance you want to train on. This is demonstrated in the following code snippet:
We retrieve the model_id corresponding to the same model we used previously (dimensions are characteristic to the base version of BERT). The tc in the identifier corresponds to text classification.
For sentence pair classification, we can set model_id to huggingface-spc-bert-base-uncased. The spc in the identifier corresponds to sentence pair classification.
You can now fine-tune this JumpStart model on your own custom dataset using the SageMaker SDK. We use a dataset that is publicly hosted on Amazon S3, conveniently focused on sentiment analysis. The dataset should be structured for fine-tuning as explained in the previous section. See the following example code:
We obtain the same default hyperparameters for our selected model as the ones we saw in the previous section, using sagemaker.hyperparameters.retrieve_default(). We then instantiate a SageMaker estimator, and call the .fit method to start fine-tuning our model, passing it the Amazon S3 URI for our training data. As you can see, the entry_point script provided is named transfer_learning.py (the same for other tasks and models), and the input data channel passed to .fit must be named training.
Deploy the fine-tuned model
When training is complete, you can deploy your fine-tuned model. To do so, all we need to obtain is the inference script URI (the code that determines how the model is used for inference once deployed) and the inference container image URI, which includes an appropriate model server to host the model we chose. See the following code:
After a few minutes, our model is deployed and we can get predictions from it in real time!
Next, we invoke the endpoint to predict the sentiment of the example text. We use the query_endpoint and parse_response helper functions, which are defined in the accompanying notebook:
Conclusion
JumpStart is a capability in SageMaker that allows you to quickly get started with ML. JumpStart uses open-source pre-trained models to solve common ML problems like image classification, object detection, text classification, sentence pair classification, and question answering.
In this post, we showed you how to fine-tune and deploy a pre-trained text classification model for sentiment analysis. We also mentioned the changes required to adapt the demo for sentence pair classification. With JumpStart, you can easily do this process with no need to code. Try out the solution on your own and let us know how it goes in the comments. To learn more about JumpStart, check out the AWS re:Invent 2020 video Get started with ML in minutes with Amazon SageMaker JumpStart.
References
- Socher et al., 2013
-
Wang et al., 2018a
-
Rajpurkar et al., 2016
-
Wang et al., 2018a
About the Authors
João Moura is an AI/ML Specialist Solutions Architect at Amazon Web Services. He is mostly focused on NLP use cases and helping customers optimize Deep Learning model training and deployment. He is also an active proponent of low-code ML solutions and ML-specialized hardware.
Dr. Vivek Madan is an Applied Scientist with Amazon SageMaker JumpStart team. He got his PhD from University of Illinois at Urbana-Champaign and was a Post Doctoral Researcher at Georgia Tech. He is an active researcher in machine learning and algorithm design and has published papers in EMNLP, ICLR, COLT, FOCS and SODA conferences.
Dr. Ashish Khetan is a Senior Applied Scientist with Amazon SageMaker JumpStart and Amazon SageMaker built-in algorithms and helps develop machine learning algorithms. He is an active researcher in machine learning and statistical inference and has published many papers in NeurIPS, ICML, ICLR, JMLR, ACL, and EMNLP conferences.
RescoreBERT: Using BERT models to improve ASR rescoring
Knowledge distillation and discriminative training enable efficient use of a BERT-based model to rescore automatic-speech-recognition hypotheses.Read More
Seamlessly connect Amazon Athena with Amazon Lookout for Metrics to detect anomalies
Amazon Lookout for Metrics is an AWS service that uses machine learning (ML) to automatically monitor the metrics that are most important to businesses with greater speed and accuracy. The service also makes it easier to diagnose the root cause of anomalies, such as unexpected dips in revenue, high rates of abandoned shopping carts, spikes in payment transaction failures, increases in new user sign-ups, and many more. Lookout for Metrics goes beyond simple anomaly detection. It allows developers to set up autonomous monitoring for important metrics to detect anomalies and identify their root cause in a matter of a few clicks to detect anomalies in its metrics—all with no ML experience required.
Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon Simple Storage Service (Amazon S3) using standard SQL. Simply point to your data in Amazon S3, define the schema, and start querying using standard SQL. Most results are delivered within seconds. With Athena, there’s no need for complex ETL jobs to prepare your data for analysis. This makes it easy for anyone with SQL skills to quickly analyze large-scale datasets.
With today’s launch, Lookout for Metrics can now seamlessly connect to your data in Athena to set up highly accurate anomaly detectors. This lets you quickly deploy state-of-the-art anomaly detection via ML with Lookout for Metrics against any datasets that are available in Athena.
Athena connectivity extends the capabilities of Lookout for Metrics by bringing the following benefits:
- It extends the capabilities of Lookout for Metrics in terms of filetype support. Prior to this, Lookout for Metrics supported CSV and JSONLines formatted files, but with Athena this has been expanded to Parquet, Avro, Plaintext, and more. If you can parse it via Athena, then it’s now possible to import and leverage with Lookout for Metrics.
- It also introduces support for data with federated queries. Prior to this launch, if your data was stored in multiple databases or sources, you would need to define a complete complex ETL process as well as manage its performance characteristics before you could export all of the data into a CSV or JSONLines file and input it into Lookout for Metrics for Anomaly Detection. With federated queries from Athena, you define the disparate sources as well as how the join should be performed and when the data has been processed and can be queried by Athena, it’s immediately ready for Lookout for Metrics. This enables you to hand over the burden for data transformation, aggregation, and delivery location to Athena and just focus on the identified anomalies from Lookout for Metrics.
Solution overview
In this post, we demonstrate how to integrate an Athena table and detect anomalies in the revenue metrics. We also track how order rate and inventory metrics are impacted. The source data lies in Amazon S3 and we’ve configured Athena tables to be able to query the data in it. An AWS Lambda is responsible for updating the partitions within Athena, which are used by Lookout for Metrics to detect anomalies. This solution enables you to use an Athena data source for Lookout for Metrics.
You could use the provided AWS CloudFormation stack to set up resources for the walkthrough. It contains resources to continuously generate live data and makes them query-able in Athena.
- Launch the stack from the following link and select next on the Create Stack page.
- On the Specify stack details page, add the values from above, give it a Stack name (for example,
L4MAthenaDetector), and select Next. - On the Configure stack options page, leave everything as-is and select Next.
Set up a new detector with Athena as the data source
Step 1
Log in to the AWS Console to get started with creating an Anomaly Detector with Lookout for Metrics. The first step is to select the “Create detector” button.
Step 2
Fill out the mandatory detector fields like name. Select the detection interval for the detector, which is determined by the frequency at which you want Lookout for Metrics to query your data and monitor them for anomalies. Encryption information is not mandatory. Encryption information allows Lookout for Metrics to encrypt your data using your AWS Key Management Service (KMS) key. In this example, we’ll skip adding an encryption key, Lookout for Metrics would use default encryption to encrypt your data if no encryption information is provided, and proceed by selecting the “Create” button.
Step 3
Upon creation of the anomaly detector, you’ll see confirmation in a banner at the top. You can proceed by selecting “Add a dataset” through either the banner or the button under “Add a dataset”.
Fill out the basic information for the data source. Timezone is an optional field. Select the dropdown to select a data source.
Lookout for Metrics supports multiple data sources as a convenience for customers. For this example, we’ll select Athena.
Once Athena is selected as the data source, you’ll have the option of selecting Backtest or Continuous mode for the detector. For this example, we’ll proceed by using the Continuous mode. Proceed by adding details for the Athena table that you want to monitor for anomalies.
You can allow the service to create a Service role or you could use an existing AWS Identity and Access Management (IAM) role in your account for federated queries. Note that Lookout for Metrics doesn’t support automated creation of IAM roles for federated queries. Therefore, you would have to create a new IAM role to allow Athena to perform the following actions on your data,
CreatePreparedStatementGetPreparedStatementGetQueryResultsStreamDeletePreparedStatementGetDatabaseGetQueryResultsGetWorkGroupGetTableMetadataStartQueryExecutionGetQueryExecution
The IAM role created by the service looks like the following:
Step 4
Now we’ll define relevant metrics for the detector. Lookout for Metrics will populate the drop-downs with the columns present in the supplied Athena table. You can select up to five metrics and five dimensions. Lookout for Metrics requires the data in your table to be partitioned as timestamps for the timestamp column. You will also have an option to estimate the cost for this detector by adding the number of values across your dimensions.
Once you have selected all of the metrics, proceed by selecting the “Next” button. Review the details and select the “Save dataset” button to save the dataset.
Step 5
Once the dataset is created, we’ll activate the detector by either selecting the “Activate” button at the top or the “Activate Detector” button under the “How it works” section.
You’ll be prompted to confirm if you want to activate the detector for continuous detection. Select “Activate” to confirm.
You’ll see a confirmation informing that the detector is activating.
Step 6
Once the Anomaly Detector is Active, you can use the “Detector log” tab on the Detector details page to review detection executions that have been performed by the service.
Step 7
You can select the “View anomalies” button from the detector details page to manually inspect anomalies that may have been detected by the service.
Step 8
On the Anomalies review page, you can adjust the severity score threshold on the threshold dial to filter anomalies above a selected score.
Review and analyze the results
When detecting an anomaly, Lookout for Metrics helps you focus on what matters most by assigning a severity score to aid prioritization. To help you find the root cause, it intelligently groups anomalies that may be related to the same incident, and then summarizes the different sources of impact.
Lookout for Metrics also lets you provide real-time feedback on the relevance of detected anomalies, thereby enabling a powerful human-in-the-loop mechanism. This information is fed back to the anomaly detection model to improve its accuracy in near-real time.
Clean up
To avoid incurring additional charges for the resource set up for the demo, you can delete the created detector under Lookout for Metrics and the stack created via CloudFormation.
Conclusion
You can seamlessly connect to your data in Athena to Lookout for Metrics to set up highly accurate anomaly detector across metrics and dimensions within your Athena tables. To get started with this capability, see Using Amazon Athena with Lookout for Metrics. You can use this capability in all Regions where Lookout for Metrics is publicly available. For more information about Region availability, see AWS Regional Services.
About the Authors
Devesh Ratho is a Software Development Engineer in the Lookout for Metrics team. His interests lie in building scalable distributed systems. In his spare time, he enjoys sim racing.
Chris King is a Senior Solutions Architect in Applied AI with AWS. He has a special interest in launching AI services and helped grow and build Amazon Personalize and Amazon Forecast before focusing on Amazon Lookout for Metrics. In his spare time he enjoys cooking, reading, boxing, and building models to predict the outcome of combat sports.
AWS contributes novel causal machine learning algorithms to DoWhy
New features go beyond conventional effect estimation by attributing events to individual components of complex systems.Read More
ACL: What comes next for natural-language processing?
Amazon Scholar and Columbia professor Kathleen McKeown on model compression, data distribution shifts, language revitalization, and more.Read More
Amazon and Max Planck Society launch Science Hub
The first Amazon Science Hub to exist outside the US will focus on driving AI research and development throughout Germany.Read More
Translating images into bird’s-eye-view maps
Reformulating the mapping problem to take advantage of sequence-to-sequence Transformers improves performance by an average of 15%.Read More
Amazon Robotics names 14 new Day One Fellowship recipients
Program empowers Black, Latinx, and Native American students to become industry leaders through scholarship, research, and career opportunities.Read More