Posts in category: Amazon AWS

Amazon SageMaker comes with two options to spin up fully managed notebooks for exploring data and building machine learning (ML) models. The first option is fast start, collaborative notebooks accessible within Amazon SageMaker Studio—a fully integrated development environment (IDE) for machine learning. You can quickly launch notebooks in Studio, easily dial up or down the underlying compute resources without interrupting your work, and even share your notebook as a link in few clicks. In addition to creating notebooks, you can perform all the ML development steps to build, train, debug, track, deploy, and monitor your models in a single pane of glass in Studio. The second option is Amazon SageMaker notebook instances—a single, fully managed ML compute instance running notebooks in the cloud, offering you more control on your notebook configurations.

Today, we are excited to announce the availability of Amazon CodeWhisperer and Amazon CodeGuru Security extensions in SageMaker notebooks. These AI-powered extensions help accelerate ML development by offering code suggestions as you type, and ensure that your code is secure and follows AWS best practices.

In this post, we show how you can get started with Amazon CodeGuru Security and CodeWhisperer in Studio and SageMaker notebook instances.

Solution overview

The CodeWhisperer extension is an AI coding companion that provides developers with real-time code suggestions in notebooks. Individual developers can use CodeWhisperer for free in Studio and SageMaker notebook instances. The coding companion generates real-time single-line or full function code suggestions. It understands semantics and context in your code and can recommend suggestions built on AWS and development best practices, improving developer efficiency, quality, and speed.

The CodeGuru Security extension offers security and code quality scans for Studio and SageMaker notebook instances. This assists notebook users in detecting security vulnerabilities such as injection flaws, data leaks, weak cryptography, or missing encryption within the notebook cells. You can also detect many common issues that affect the readability, reproducibility, and correctness of computational notebooks, such as misuse of ML library APIs, invalid run order, and nondeterminism. When vulnerabilities or quality issues are identified in the notebook, CodeGuru generates recommendations that enable you to remediate those issues based on AWS security best practices.

In the following sections, we show how to install each of the extensions and discuss the capabilities of each, demonstrating how these tools can improve overall developer productivity.

Prerequisites

If this is your first time working with Studio, you first need to create a SageMaker domain. Additionally, make sure you have appropriate access to both CodeWhisperer and CodeGuru using AWS Identity and Access Management (IAM).

You can use these extensions in any AWS Region, but requests to CodeWhisperer will be served through the us-east-1 Region. Requests will be served to CodeGuru in the Region of the Studio domain and if CodeGuru is supported in the Region. For all non-supported Regions, the requests will be served through us-east-1.

Set up CodeWhisperer with SageMaker notebooks

In this section, we demonstrate how to set up CodeWhisperer with SageMaker Studio.

Update IAM permissions to use the extension

You can use the CodeWhisperer extension in any Region, but all requests to CodeWhisperer will be served through the us-east-1 Region.

To use the CodeWhisperer extension, ensure that you have the necessary permissions. On the IAM console, add the following policy to the SageMaker user execution role:

{
"Version": "2012-10-17",
"Statement": [
          {
               	"Sid": "CodeWhispererPermissions",
               	"Effect": "Allow",
               	"Action": ["codewhisperer:GenerateRecommendations"],
				"Resource": "*"
          }
    ]
}

Install the CodeWhisperer extension

You can install the CodeWhisperer extension through the command line. In this section, we look at the steps involved. To get started, complete the following steps:

1. On the File menu, choose New and Terminal.
2. Run the following commands to install the extension:

   conda activate studio
   pip install amazon-codewhisperer-jupyterlab-ext
   jupyter server extension enable amazon_codewhisperer_jupyterlab_ext
   conda deactivate
   restart-jupyter-server

Refresh your browser, and you will have successfully installed the CodeWhisperer extension.

Use CodeWhisperer in Studio

After we complete the installation steps, we can use CodeWhisperer by opening a new notebook or Python file. For our example we will open a sample Notebook.

You will see a toolbar at the bottom of your notebook called CodeWhisperer. This shows common shortcuts for CodeWhisperer along with the ability to pause code suggestions, open the code reference log, and get a link to the CodeWhisperer documentation.

The code reference log will flag or filter code suggestions that resemble open-source training data. Get the associated open-source project’s repository URL and license so that you can more easily review them and add attributions.

To get started, place your cursor in a code block in your notebook, and CodeWhisperer will begin to make suggestions .If you don’t see suggestions, press Alt+C in Windows or Option+C in Mac to manually invoke suggestions.

The following video shows how to use CodeWhisperer to read and perform descriptive statistics on a data file in Studio.

Use CodeWhisperer in SageMaker Notebook Instances

Complete the following steps to use CodeWhisperer in notebook instances:

1. Navigate to your SageMaker notebook instance.
2. Make sure you have attached the CodeWhisperer policy from earlier to the notebook instance IAM role.
3. When the permissions are added, choose Open JupyterLab.
   
4. Install the extension. by using a terminal, on the File menu, choose New and Terminal, and enter the following commands:

   pip install amazon-codewhisperer-jupyterlab-ext
   jupyter server extension enable amazon_codewhisperer_jupyterlab_ext

5. Once the commands complete, on the File menu, choose Shut Down to restart our Jupyter Server.
   
6. Refresh the browser window.

You will now see the CodeWhisperer extension installed and ready to use.

Let’s test it out in a Python file.

7. On the File menu, choose New and Python File.

The following video shows how to create a function to convert a JSON file to a CSV.

Set up CodeGuru Security with SageMaker notebooks

In this section, we demonstrate how to set up CodeGuru Security with SageMaker Studio.

Update IAM permissions to use the extension

To use the CodeGuru Security extension, ensure that you have the necessary permissions. Complete the following steps to update permission policies with IAM:

1. Preferred: On the IAM console, you can attach the AmazonCodeGuruSecurityScanAccess managed policy to your IAM identities. This policy grants permissions that allow a user to work with scans, including creating scans, viewing scan information, and viewing scan findings.
2. For custom policies, enter the following permissions:

   { 
       "Version": "2012-10-17", 
       "Statement": [ 
           { 
               "Sid": "AmazonCodeGuruSecurityScanAccess", 
               "Effect": "Allow", 
               "Action": [ 
                   "codeguru-security:CreateScan", 
                   "codeguru-security:CreateUploadUrl", 
                   "codeguru-security:GetScan", 
                   "codeguru-security:GetFindings" 
               ], 
               "Resource": "arn:aws:codeguru-security:*:*:scans/*" 
           } 
       ] 
   }

3. Attach the policy to any user or role that will use the CodeGuru Security extension.

For more information, see Policies and permissions in IAM.

Install the CodeGuru Security extension

You can install the CodeGuru Security extension through the command line. To get started, complete the following steps:

1. On the File menu, choose New and Terminal.
2. Run the following commands to install the extension in the conda environment:

   conda activate studio
   pip install amazon-codeguru-jupyterlab-extension
   conda deactivate

Refresh your browser, and you will have successfully installed the CodeGuru extension.

Run a code scan

The following steps demonstrate running your first CodeGuru Security scan using an example file:

1. Create a new notebook called example.ipynb with the following code for testing purposes:

   import torch
   # import tensorflow as tf
   
       
   def tensorflow_avoid_using_nondeterministic_api_noncompliant():
       data = tf.ones((1, 1))
       # Noncompliant: Determinism of tf.compat.v1.Session
       # can not be guaranteed in TF2.
       Ítf.config.experimental.enable_op_determinism()
       tf.compat.v1.Session(
           target='', graph=None, config=None
       )
       layer = tf.keras.layers.Input(shape=[1])
       model = tf.keras.models.Model(inputs=layer, outputs=layer)
       model.compile(loss="categorical_crossentropy", metrics="AUC")
       model.fit(x=data, y=data)
       
   def pytorch_sigmoid_before_bceloss_compliant():
       # Compliant: `BCEWithLogitsLoss` function integrates a `Sigmoid`
       # layer and the `BCELoss` into one class
       # and is numerically robust.
       loss = nn.BCEWithLogitsLoss()
   
       input = torch.randn(3, requires_grad=True)
       target = torch.empty(3).random_(2)
       output = loss(input, target)
       output.backward()

The below code has intentionally incorporated common bad practices to showcase the capabilities of Amazon CodeGuru Security.

2. Important: Please confirm that the CodeGuru-Security extension is installed and if the LSP server says Fully initialized as shown below when you open your notebook.

If you don’t see the extension fully initialized, return to the previous section to install the extension and complete the installation steps.

3. Initiate the scan. You can initiate a scan in one of the following ways:
   • Choose any code cell in your file, then choose the lightbulb icon.
     
   • Choose (right-click) any code cell in your file, then choose Run CodeGuru scan.
     

When the scan is started, the scan status will show as CodeGuru: Scan in progress.

After a few seconds, when the scan is complete, the status will change to CodeGuru: Scan completed.

View and address findings

After the scan is finished, your code may have some underlined findings. Hover over the underlined code, and a pop-up window appears with a brief summary of the finding. To access additional details about the findings, right-click on any cell and choose Show diagnostics panel.

This will open a panel containing additional information and suggestions related to the findings, located at the bottom of the notebook file.

After making changes to your code based on the recommendations, you can rerun the scan to check if the issue has been resolved. It’s important to note that the scan findings will disappear after you modify your code, and you’ll need to rerun the scan to view them again.

Enable automatic code scans

Automatic scans are disabled by default. Optionally, you can enable automatic code scans and set the frequency and AWS Region for your scan runs. To enable automatic code scans, complete the following steps.

1. In Studio, on the Settings menu, choose Advanced Settings Editor.
   
2. For Auto scans, choose Enabled.
3. Specify the scan frequency in seconds and the Region for your CodeGuru Security scan.

For our example, we configure CodeGuru to perform an automatic security scan every 240 seconds in the us-east-1 Region. You can modify this value for any region that CodeGuru Security is supported.

Conclusion

SageMaker Studio and SageMaker Notebook Instances now support AI-powered CodeWhisperer and CodeGuru extensions that help you write secure code faster. We encourage you to try out both extensions. To learn more about CodeGuru Security for SageMaker, refer to Get started with the Amazon CodeGuru Extension for JupyterLab and SageMaker Studio, and to learn more about CodeWhisperer for SageMaker, refer to Setting up CodeWhisperer with Amazon SageMaker Studio. Please share any feedback in the comments!

About the authors

Raj Pathak is a Senior Solutions Architect and Technologist specializing in Financial Services (Insurance, Banking, Capital Markets) and Machine Learning. He specializes in Natural Language Processing (NLP), Large Language Models (LLM) and Machine Learning infrastructure and operations projects (MLOps).

Gaurav Parekh is a Solutions Architect helping AWS customers build large scale modern architecture. His core area of expertise include Data Analytics, Networking and Technology strategy. Outside of work, Gaurav enjoys playing cricket, soccer and volleyball.

Arkaprava De is a Senior Software Engineer at AWS. He has been at Amazon for over 7 years and is currently working on improving the Amazon SageMaker Studio IDE experience. You can find him on LinkedIn.

Prashant Pawan Pisipati is a Principal Product Manager at Amazon Web Services (AWS). He has built various products across AWS and Alexa, and is currently focused on helping Machine Learning practitioners be more productive through AWS services.

Read More

Amazon Kendra is an intelligent search service powered by machine learning (ML). Amazon Kendra reimagines enterprise search for your websites and applications so your employees and customers can easily find the content they’re looking for, even when it’s scattered across multiple locations and content repositories within your organization. Keywords or natural language questions can be used to search most relevant documents powered by ML to deliver answers and rank documents. Amazon Kendra can index data from Amazon Simple Storage Service (Amazon S3) or from a third-party document repository. Amazon S3 is an object storage service that offers scalability and availability where you can store large amounts of data, including product manuals, project and research documents, and more.

In this post, you can learn how to deploy a provided AWS CloudFormation template to index your documents in an Amazon S3 bucket. The template creates an Amazon Kendra data source for an index and synchronizes your data source according to your needs: on-demand, hourly, daily, weekly or monthly. AWS CloudFormation allows us to provision infrastructure as code (IaC) so you can spend less time managing resources, replicate your infrastructure quickly, and control and track changes in the infrastructure.

Overview of the solution

The CloudFormation template sets up an Amazon Kendra data source with a connection to Amazon S3. The template also creates one role for the Amazon Kendra data source service. You can specify an S3 bucket, synchronization schedule, and inclusion/exclusion patterns. When the synchronization job has finished, you can search the indexed content through the Search console. The following diagram illustrates this workflow.

This post guides you to the following steps:

1. Deploy the provided template.
2. Upload the documents to the S3 bucket that you create. If you provide a bucket with documents, you can omit this step.
3. Wait until the index finishes crawling the data source.

Prerequisites

For this walkthrough, you should have the following prerequisites:

• An AWS account where the proposed solution can be deployed.
• An Amazon Kendra index for attaching a data source to the stack.
• The set of documents that are used to create the Amazon Kendra index. In this solution, you are using a compressed file of AWS whitepapers.

Deploy the solution with AWS CloudFormation

To deploy the CloudFormation template, complete the following steps:

1. Choose
   

You’re redirected to the AWS CloudFormation console.

2. You can modify the parameters or use the default values:
   • The Amazon Kendra data source name is automatically set using the stack name and associated bucket name.
   • For KendraIndexId, enter the Amazon Kendra index ID where you will attach the data source.
   • You can also choose when you want to run the data source synchronization using KendraSyncSchedule. By default, it’s set to OnDemand.
   • For S3BucketName, you can either enter a bucket you have already created or leave it empty. If you leave it empty, a bucket will be created for you. Either way, the bucket is used as the Amazon Kendra data source. For this post, we leave it empty.

It takes around 5 minutes for the stack to deploy the Amazon Kendra data source attached to the Amazon Kendra index.

3. On the Outputs tab of the CloudFormation stack, copy the name of the created bucket, data source name, and ID.

The created stack deploys one role: <stack-name>-KendraDataSourceRole. It’s a best practice to deploy a role for each data source you create. This role gives Amazon Kendra data source to add or remove files from Amazon Kendra index, to get objects from Amazon S3 bucket.

Upload files to the S3 bucket

Amazon Kendra can handle multiple document types, such as .html, .pdf, .csv, .json, .docx, and .ppt. You can also have a combination of documents on a single index. The text contained in those documents is indexed to the provided Amazon Kendra index. You can search for keywords on AWS topics on best practices, databases, machine learning, security, and more using over 60 pdf files that you can download. For example, if you want to know where you can find more information about caching in the AWS whitepapers, Amazon Kendra can help you find documents related to databases and best practices.

When you download the AWS Whitepapers.zip file and uncompress the file, you see these six folders: Best_Practices, Databases, General, Machine_Learning, Security, Well_Architected. Upload these folders to your S3 bucket.

Synchronize the Amazon Kendra data source

Amazon Kendra data source data can synchronize your data based on preconfigured schedule or can be be manually triggered on-demand. By default, CloudFormation template configures the data source to on-demand synchronization schedule to be triggered manually as required.

To manually trigger the synchronization job from the AWS Amazon Kendra console, navigate to the Amazon Kendra index used as part of CloudFormation stack deployment, under Data Management in the navigation pane, choose Data Sources and then choose Sync now. This makes the S3 bucket synchronize with the data source.

When the Amazon Kendra data source starts syncing, you should see the Current sync state as Syncing.

When the data source has finished, the Last sync status appears as Succeeded and Current sync state as Idle. You can now search the indexed content.

Configure synchronization schedule

The template allows you to run the schedule every hour at minute 0, for example, 13:00, 14:00, or 15:00. You also have the option to run it daily at 00:00 UTC. The Weekly setting runs Mondays at 00:00 UTC, and the Monthly setting runs every first day of the month at 00:00 UTC.

To change the schedule after the Amazon Kendra data source has been created, on the Actions menu, choose Edit. Under Configure sync settings, you find the Sync rule schedule section.

Under Frequency, you can select hourly, daily, weekly, monthly, or custom, all of which allow you to schedule your sync down to the minute.

Add exclusion patterns

The provided CloudFormation template allows you to add exclusion patterns. By default, .png and .jpg files will be added to the ExclusionPatterns parameter. Additional file formats can be added as a comma separated list to the exclusion pattern. Similarly, InclusionPatterns parameter may be used add comma list file formats to set up an inclusion pattern. If you don’t provide an inclusion pattern, all files are indexed except for the ones included in the exclusion parameter.

Clean up

To avoid costs, you can delete the stack from the AWS CloudFormation console. On the Stacks page, select the stack you created, choose Delete, and confirm the deletion of the stack.

If you haven’t provided a S3 bucket, the stack creates a bucket. If the bucket is empty, it’s automatically deleted. Otherwise, you need to empty the folder and manually delete it. If you provided a bucket, even if it’s empty, it won’t be deleted. Amazon Kendra index won’t be deleted. Only the Amazon Kendra data source created by the stack will be deleted.

Conclusion

In this post, we provided an CloudFormation template to easily synchronize your text documents on an S3 bucket to your Amazon Kendra index. This solution is helpful if you have multiple S3 buckets you want to index because you can create all the necessary components to query the documents with a few clicks in a consistent and repeatable manner. You can also see how image-based text documents can be handled in Amazon Kendra. To learn more about specific schedule patterns, refer to Schedule Expressions for Rules.

Leave a comment and learn more about Amazon Kendra index creation in the following Amazon Kendra Essentials+ workshop.

Special thanks to Jose Mauricio Mani Yanez for his help creating the example code and compiling the content for this post.

About the author

Rajesh Kumar Ravi is an AI/ML Specialist Solutions Architect at Amazon Web Services specializing in intelligent document search with Amazon Kendra and generative AI. He is a builder and problem solver, and contributes to development of new ideas. He enjoys walking and loves to go on short hiking trips outside of work.

Read More

You can now register machine learning (ML) models built in Amazon SageMaker Canvas with a single click to the Amazon SageMaker Model Registry, enabling you to operationalize ML models in production. Canvas is a visual interface that enables business analysts to generate accurate ML predictions on their own—without requiring any ML experience or having to write a single line of code. Although it’s a great place for development and experimentation, to derive value from these models, they need to be operationalized—namely, deployed in a production environment where they can be used to make predictions or decisions. Now with the integration with the model registry, you can store all model artifacts, including metadata and performance metrics baselines, to a central repository and plug them into your existing model deployment CI/CD processes.

The model registry is a repository that catalogs ML models, manages various model versions, associates metadata (such as training metrics) with a model, manages the approval status of a model, and deploys them to production. After you create a model version, you typically want to evaluate its performance before you deploy it to a production endpoint. If it performs to your requirements, you can update the approval status of the model version to approved. Setting the status to approved can initiate CI/CD deployment for the model. If the model version doesn’t perform to your requirements, you can update the approval status to rejected in the registry, which prevents the model from being deployed into an escalated environment.

A model registry plays a key role in the model deployment process because it packages all model information and enables the automation of model promotion to production environments. The following are some ways that a model registry can help in operationalizing ML models:

• Version control – A model registry allows you to track different versions of your ML models, which is essential when deploying models in production. By keeping track of model versions, you can easily revert to a previous version if a new version causes issues.
• Collaboration – A model registry enables collaboration among data scientists, engineers, and other stakeholders by providing a centralized location for storing, sharing, and accessing models. This can help streamline the deployment process and ensure that everyone is working with the same model.
• Governance – A model registry can help with compliance and governance by providing an auditable history of model changes and deployments.

Overall, a model registry can help streamline the process of deploying ML models in production by providing version control, collaboration, monitoring, and governance.

Overview of solution

For our use case, we are assuming the role of a business user in the marketing department of a mobile phone operator, and we have successfully created an ML model in Canvas to identify customers with the potential risk of churn. Thanks to the predictions generated by our model, we now want to move this from our development environment to production. However, before our model gets deployed to a production endpoint, it needs to be reviewed and approved by a central MLOps team. This team is responsible for managing model versions, reviewing all associated metadata (such as training metrics) with a model, managing the approval status of every ML model, deploying approved models to production, and automating model deployment with CI/CD. To streamline the process of deploying our model in production, we take advantage of the integration of Canvas with the model registry and register our model for review by our MLOps team.

The workflow steps are as follows:

1. Upload a new dataset with the current customer population into Canvas. For the full list of supported data sources, refer to Import data into Canvas.
2. Build ML models and analyze their performance metrics. Refer to the instructions to build a custom ML model in Canvas and evaluate the model’s performance.
3. Register the best performing versions to the model registry for review and approval.
4. Deploy the approved model version to a production endpoint for real-time inferencing.

You can perform Steps 1–3 in Canvas without writing a single line of code.

Prerequisites

For this walkthrough, make sure that the following prerequisites are met:

1. To register model versions to the model registry, the Canvas admin must give the necessary permissions to the Canvas user, which you can manage in the SageMaker domain that hosts your Canvas application. For more information, refer to the Amazon SageMaker Developer Guide. When granting your Canvas user permissions, you must choose whether to allow the user to register their model versions in the same AWS account.

2. Implement the prerequisites mentioned in Predict customer churn with no-code machine learning using Amazon SageMaker Canvas.

You should now have three model versions trained on historical churn prediction data in Canvas:

• V1 trained with all 21 features and quick build configuration with a model score of 96.903%
• V2 trained with all 19 features (removed phone and state features) and quick build configuration and improved accuracy of 97.403%
• V3 trained with standard build configuration with 97.03% model score
  

Use the customer churn prediction model

Enable Show advanced metrics and review the objective metrics associated with each model version so that we can select the best performing model for registration to the model registry.

Based on the performance metrics, we select version 2 to be registered.

The model registry tracks all the model versions that you train to solve a particular problem in a model group. When you train a Canvas model and register it to the model registry, it gets added to a model group as a new model version.

At the time of registration, a model group within the model registry is automatically created. Optionally, you can rename it to a name of your choice or use an existing model group in the model registry.

For this example, we use the autogenerated model group name and choose Add.

Our model version should now be registered to the model group in the model registry. If we were to register another model version, it would be registered to the same model group.

The status of the model version should have changed from Not Registered to Registered.

When we hover over the status, we can review the model registry details, which include the model group name, model registry account ID, and approval status. Right after registration, the status changes to Pending Approval, which means that this model is registered in the model registry but is pending review and approval from a data scientist or MLOps team member and can only be deployed to an endpoint if approved.

Now let’s navigate to Amazon SageMaker Studio and assume the role of an MLOps team member. Under Models in the navigation pane, choose Model registry to open the model registry home page.

We can see the model group canvas-Churn-Prediction-Model that Canvas automatically created for us.

Choose the model to review all the versions registered to this model group and then review the corresponding model details.

If you open the details for version 1, we can see that the Activity tab keeps track of all the events happening on the model.

On the Model quality tab, we can review the model metrics, precision/recall curves, and confusion matrix plots to understand the model performance.

On the Explainability tab, we can review the features that influenced the model’s performance the most.

After we have reviewed the model artifacts, we can change the approval status from Pending to Approved.

We can now see the updated activity.

The Canvas business user will now be able to see that the registered model status changed from Pending Approval to Approved.

As the MLOps team member, because we have approved this ML model, let’s deploy it to an endpoint.

In Studio, navigate to the model registry home page and choose the canvas-Churn-Prediction-Model model group. Choose the version to be deployed and go to the Settings tab.

Browse to get the model package ARN details from the selected model version in the model registry.

Open a notebook in Studio and run the following code to deploy the model to an endpoint. Replace the model package ARN with your own model package ARN.

from sagemaker import ModelPackage
from time import gmtime, strftime
import boto3
import os
import pandas as pd
import sagemaker
import time
from datetime import datetime
from sagemaker import ModelPackage

boto_session = boto3.session.Session()
sagemaker_client = boto_session.client("sagemaker")
sagemaker_session = sagemaker.session.Session(
boto_session=boto_session, sagemaker_client=sagemaker_client
)
role=(sagemaker.get_execution_role())
model_package_arn = 'arn:aws:sagemaker:us-west-2:1234567890:model-package/canvas-churn-prediction-model/3'
model = ModelPackage(role=role,
model_package_arn=model_package_arn,
sagemaker_session=sagemaker_session)
model.deploy(initial_instance_count=1, instance_type='ml.m5.xlarge')

After the endpoint gets created, you can see it tracked as an event on the Activity tab of the model registry.

You can double-click on the endpoint name to get its details.

Now that we have an endpoint, let’s invoke it to get a real-time inference. Replace your endpoint name in the following code snippet:

import boto3, sys

endpoint_name = '3-2342-12-21-12-07-09-075'
sm_rt = boto3.Session().client('runtime.sagemaker')

payload = [163, 806, 'no', 'yes', 300, 8.162204, 3, 3.933, 2.245779, 4, 6.50863, 6.05194, 5, 4.948816088, 1.784764, 2, 5.135322, 8]
l = ",".join([str(p) for p in payload])

response = sm_rt.invoke_endpoint(EndpointName=endpoint_name, ContentType='text/csv', Accept='text/csv', Body=l)

response = response['Body'].read().decode("utf-8")
print(response)

Clean up

To avoid incurring future charges, delete the resources you created while following this post. This includes logging out of Canvas and deleting the deployed SageMaker endpoint. Canvas bills you for the duration of the session, and we recommend logging out of Canvas when you’re not using it. Refer to Logging out of Amazon SageMaker Canvas for more details.

Conclusion

In this post, we discussed how Canvas can help operationalize ML models to production environments without requiring ML expertise. In our example, we showed how an analyst can quickly build a highly accurate predictive ML model without writing any code and register it to the model registry. The MLOps team can then review it and either reject the model or approve the model and initiate the downstream CI/CD deployment process.

To start your low-code/no-code ML journey, refer to Amazon SageMaker Canvas.

Special thanks to everyone who contributed to the launch:

Backend:

• Huayuan (Alice) Wu
• Krittaphat Pugdeethosapol
• Yanda Hu
• John He
• Esha Dutta
• Prashanth

Front end:

• Kaiz Merchant
• Ed Cheung

About the Authors

Janisha Anand is a Senior Product Manager in the SageMaker Low/No Code ML team, which includes SageMaker Autopilot. She enjoys coffee, staying active, and spending time with her family.

Krittaphat Pugdeethosapol is a Software Development Engineer at Amazon SageMaker and mainly works with SageMaker low-code and no-code products.

Huayuan(Alice) Wu is a Software Development Engineer at Amazon SageMaker. She focuses on building ML tools and products for customers. Outside of work, she enjoys the outdoors, yoga, and hiking.

Read More

Today, data scientists who are training deep learning models need to identify and remediate model training issues to meet accuracy targets for production deployment, and require a way to utilize standard tools for debugging model training. Among the data scientist community, TensorBoard is a popular toolkit that allows data scientists to visualize and analyze various aspects of their machine learning (ML) models and training processes. It provides a suite of tools for visualizing training metrics, examining model architectures, exploring embeddings, and more. TensorFlow and PyTorch projects both endorse and use TensorBoard in their official documentation and examples.

Amazon SageMaker with TensorBoard is a capability that brings the visualization tools of TensorBoard to SageMaker. Integrated with SageMaker training jobs and domains, it provides SageMaker domain users access to the TensorBoard data and helps domain users perform model debugging tasks using the SageMaker TensorBoard visualization plugins. When they create a SageMaker training job, domain users can use TensorBoard using the SageMaker Python SDK or Boto3 API. SageMaker with TensorBoard is supported by the SageMaker Data Manager plugin, with which domain users can access many training jobs in one place within the TensorBoard application.

In this post, we demonstrate how to set up a training job with TensorBoard in SageMaker using the SageMaker Python SDK, access SageMaker TensorBoard, explore training output data visualized in TensorBoard, and delete unused TensorBoard applications.

Solution overview

A typical training job for deep learning in SageMaker consists of two main steps: preparing a training script and configuring a SageMaker training job launcher. In this post, we walk you through the required changes to collect TensorBoard-compatible data from SageMaker training.

Prerequisites

To start using SageMaker with TensorBoard, you need to set up a SageMaker domain with an Amazon VPC under an AWS account. Domain user profiles for each individual user are required to access the TensorBoard on SageMaker, and the AWS Identity and Access Management (IAM) execution role needs a minimum set of permissions, including the following:

• sagemaker:CreateApp
• sagemaker:DeleteApp
• sagemaker:DescribeTrainingJob
• sagemaker:Search
• s3:GetObject
• s3:ListBucket

For more information on how to set up SageMaker Domain and user profiles, see Onboard to Amazon SageMaker Domain Using Quick setup and Add and Remove User Profiles.

Directory structure

When using Amazon SageMaker Studio, the directory structure can be organized as follows:

.
├── script
│	└── train.py
└── simple_tensorboard.ipynb

Here, script/train.py is your training script, and simple_tensorboard.ipynb launches the SageMaker training job.

Modify your training script

You can use any of the following tools to collect tensors and scalars: TensorBoardX, TensorFlow Summary Writer, PyTorch Summary Writer, or Amazon SageMaker Debugger, and specify the data output path as the log directory in the training container (log_dir). In this sample code, we use TensorFlow to train a simple, fully connected neural network for a classification task. For other options, refer to Prepare a training job with a TensorBoard output data configuration. In the train() function, we use the tensorflow.keras.callbacks.TensorBoard tool to collect tensors and scalars, specify /opt/ml/output/tensorboard as the log directory in the training container, and pass it to model training callbacks argument. See the following code:

import argparse
import json
import tensorflow as tf


def parse_args():
    cmdline = argparse.ArgumentParser(
        formatter_class=argparse.ArgumentDefaultsHelpFormatter
    )
    cmdline.add_argument("--epochs", default=5, type=int, help="""Number of epochs.""")
    cmdline.add_argument(
        "--optimizer", default="adam", type=str, help="""Optimizer type"""
    )
    cmdline.add_argument(
        "--loss",
        default="sparse_categorical_crossentropy",
        type=str,
        help="""Optimizer type""",
    )
    cmdline.add_argument(
        "--metrics",
        action="store",
        dest="metrics",
        type=json.loads,
        default="['accuracy']",
        help="List of metrics to be evaluated by the model during training and testing.",
    )
    return cmdline


def create_model():
    return tf.keras.models.Sequential(
        [
            tf.keras.layers.Flatten(input_shape=(28, 28)),
            tf.keras.layers.Dense(512, activation="relu"),
            tf.keras.layers.Dropout(0.2),
            tf.keras.layers.Dense(10, activation="softmax"),
        ]
    )


def train(args):
    mnist = tf.keras.datasets.mnist
    (x_train, y_train), (x_test, y_test) = mnist.load_data()
    x_train, x_test = x_train / 255.0, x_test / 255.0

    model = create_model()
    model.compile(optimizer=args.optimizer, loss=args.loss, metrics=args.metrics)

    # setup TensorBoard Callback
    LOG_DIR = "/opt/ml/output/tensorboard"
    tensorboard_callback = tf.keras.callbacks.TensorBoard(
        log_dir=LOG_DIR,
        histogram_freq=1,
        update_freq=1,
        embeddings_freq=5,
        write_images=True,
    )

    # pass TensorBoard Callback into the Model fit
    model.fit(
        x=x_train,
        y=y_train,
        epochs=args.epochs,
        validation_data=(x_test, y_test),
        callbacks=[tensorboard_callback],
    )


if __name__ == "__main__":
    cmdline = parse_args()
    args, unknown_args = cmdline.parse_known_args()
    train(args)

Construct a SageMaker training launcher with a TensorBoard data configuration

Use sagemaker.debugger.TensorBoardOutputConfig while configuring a SageMaker framework estimator, which maps the Amazon Simple Storage Service (Amazon S3) bucket you specify for saving TensorBoard data with the local path in the training container (for example, /opt/ml/output/tensorboard). You can use a different container local output path. However, it must be consistent with the value of the LOG_DIR variable, as specified in the previous step, to have SageMaker successfully search the local path in the training container and save the TensorBoard data to the S3 output bucket.

Next, pass the object of the module to the tensorboard_output_config parameter of the estimator class. The following code snippet shows an example of preparing a TensorFlow estimator with the TensorBoard output configuration parameter.

The following is the boilerplate code:

import os
from datetime import datetime
import boto3
import sagemaker

time_str = datetime.now().strftime("%d-%m-%Y-%H-%M-%S")

region = boto3.session.Session().region_name
boto_sess = boto3.Session()
role = sagemaker.get_execution_role()
sm = sagemaker.Session()

base_job_name = "simple-tensorboard"
date_str = datetime.now().strftime("%d-%m-%Y")
time_str = datetime.now().strftime("%d-%m-%Y-%H-%M-%S")
job_name = f"{base_job_name}-{time_str}"

s3_output_bucket = os.path.join("s3://", sm.default_bucket(), base_job_name)

output_path = os.path.join(s3_output_bucket, "sagemaker-output", date_str, job_name)
code_location = os.path.join(s3_output_bucket, "sagemaker-code", date_str, job_name)

The following code is for the training container:

instance_type = "ml.c5.xlarge"
instance_count = 1

image_uri = sagemaker.image_uris.retrieve(
    framework="tensorflow",
    region=region,
    version="2.11",
    py_version="py39",
    image_scope="training",
    instance_type=instance_type,
)

The following code is the TensorBoard configuration:

from sagemaker.tensorflow import TensorFlow

tensorboard_output_config = sagemaker.debugger.TensorBoardOutputConfig(
    s3_output_path=os.path.join(output_path, "tensorboard"),
    container_local_output_path="/opt/ml/output/tensorboard",
)

hyperparameters = {
    "epochs": 5,
    "optimizer": "adam",
    "loss": "sparse_categorical_crossentropy",
    "metrics": "'["accuracy"]'",
}

estimator = TensorFlow(
    entry_point="train.py",
    source_dir="script",
    role=role,
    image_uri=image_uri,
    instance_count=1,
    instance_type="ml.c5.xlarge",
    base_job_name=job_name,
    tensorboard_output_config=tensorboard_output_config,
    hyperparameters=hyperparameters,
)

Launch the training job with the following code:

estimator.fit(
    inputs=None,
    wait=False,
    job_name=job_name,
)

Access TensorBoard on SageMaker

You can access TensorBoard with two methods: programmatically using the sagemaker.interactive_apps.tensorboard module that generates the URL or using the TensorBoard landing page on the SageMaker console. After you open TensorBoard, SageMaker runs the TensorBoard plugin and automatically finds and loads all training job output data in a TensorBoard-compatible file format from S3 buckets paired with training jobs during or after training.

The following code autogenerates the URL to the TensorBoard console landing page:

from sagemaker.interactive_apps import tensorboard
app = tensorboard.TensorBoardApp(region)

print("Navigate to the following URL:")
if app._is_studio_user:
    print(app.get_app_url(job_name))
else:
    print(app.get_app_url())

This returns the following message with a URL that opens the TensorBoard landing page.

>>> Navigate to the following URL: https://<sagemaker_domain_id>.studio.<region>.sagemaker.aws/tensorboard/default/data/plugin/sa

For opening TensorBoard from the SageMaker console, please refer to How to access TensorBoard on SageMaker.

When you open the TensorBoard application, TensorBoard opens with the SageMaker Data Manager tab. The following screenshot shows the full view of the SageMaker Data Manager tab in the TensorBoard application.

On the SageMaker Data Manager tab, you can select any training job and load TensorBoard-compatible training output data from Amazon S3.

1. In the Add Training Job section, use the check boxes to choose training jobs from which you want to pull data and visualize for debugging.
2. Choose Add Selected Jobs.

The selected jobs should appear in the Tracked Training Jobs section.

Refresh the viewer by choosing the refresh icon in the upper-right corner, and the visualization tabs should appear after the job data is successfully loaded.

Explore training output data visualized in TensorBoard

On the Time Series tab and other graphics-based tabs, you can see the list of Tracked Training Jobs in the left pane. You can also use the check boxes of the training jobs to show or hide visualizations. The TensorBoard dynamic plugins are activated dynamically depending on how you have set your training script to include summary writers and pass callbacks for tensor and scalar collection, and the graphics tabs also appear dynamically. The following screenshots show example views of each tab with visualizations of the collected metrics of two training jobs. The metrices include time series, scalar, graph, distribution, and histogram plugins.

The following screenshot is the Time Series tab view.

The following screenshot is the Scalars tab view.

The following screenshot is the Graphs tab view.

The following screenshot is the Distributions tab view.

The following screenshot is the Histograms tab view.

Clean up

After you are done with monitoring and experimenting with jobs in TensorBoard, shut the TensorBoard application down:

1. On the SageMaker console, choose Domains in the navigation pane.
2. Choose your domain.
3. Choose your user profile.
4. Under Apps, choose Delete App for the TensorBoard row.
5. Choose Yes, delete app.
6. Enter delete in the text box, then choose Delete.

A message should appear at the top of the page: “Default is being deleted”.

Conclusion

TensorBoard is a powerful tool for visualizing, analyzing, and debugging deep learning models. In this post, we provide a guide to using SageMaker with TensorBoard, including how to set up TensorBoard in a SageMaker training job using the SageMaker Python SDK, access SageMaker TensorBoard, explore training output data visualized in TensorBoard, and delete unused TensorBoard applications. By following these steps, you can start using TensorBoard in SageMaker for your work.

We encourage you to experiment with different features and techniques.

About the authors

Dr. Baichuan Sun is a Senior Data Scientist at AWS AI/ML. He is passionate about solving strategic business problems with customers using data-driven methodology on the cloud, and he has been leading projects in challenging areas including robotics computer vision, time series forecasting, price optimization, predictive maintenance, pharmaceutical development, product recommendation system, etc. In his spare time he enjoys traveling and hanging out with family.

Manoj Ravi is a Senior Product Manager for Amazon SageMaker. He is passionate about building next-gen AI products and works on software and tools to make large-scale machine learning easier for customers. He holds an MBA from Haas School of Business and a Masters in Information Systems Management from Carnegie Mellon University. In his spare time, Manoj enjoys playing tennis and pursuing landscape photography.

Read More

Amazon SageMaker provides a broad selection of machine learning (ML) infrastructure and model deployment options to help meet your ML inference needs. It’s a fully-managed service and integrates with MLOps tools so you can work to scale your model deployment, reduce inference costs, manage models more effectively in production, and reduce operational burden. SageMaker provides multiple inference options so you can pick the option that best suits your workload.

New generations of CPUs offer a significant performance improvement in ML inference due to specialized built-in instructions. In this post, we focus on how you can take advantage of the AWS Graviton3-based Amazon Elastic Compute Cloud (EC2) C7g instances to help reduce inference costs by up to 50% relative to comparable EC2 instances for real-time inference on Amazon SageMaker. We show how you can evaluate the inference performance and switch your ML workloads to AWS Graviton instances in just a few steps.

To cover the popular and broad range of customer applications, in this post we discuss the inference performance of PyTorch, TensorFlow, XGBoost, and scikit-learn frameworks. We cover computer vision (CV), natural language processing (NLP), classification, and ranking scenarios for models and ml.c6g, ml.c7g, ml.c5, and ml.c6i SageMaker instances for benchmarking.

Benchmarking results

AWS measured up to 50% cost savings for PyTorch, TensorFlow, XGBoost, and scikit-learn model inference with AWS Graviton3-based EC2 C7g instances relative to comparable EC2 instances on Amazon SageMaker. At the same time, the latency of inference is also reduced.

For comparison, we used four different instance types:

• c7g.4xlarge
• c6g.4xlarge
• c6i.4xlarge
• c5.4xlarge

All four instances have 16 vCPUs and 32 GiB of memory.

In the following graph, we measured the cost per million inference for the four instance types. We further normalized the cost per million inference results to a c5.4xlarge instance, which is measured as 1 on the Y-axis of the chart. You can see that for the XGBoost models, cost per million inference for c7g.4xlarge (AWS Graviton3) is about 50% of the c5.4xlarge and 40% of c6i.4xlarge; for the PyTorch NLP models, the cost savings is about 30–50% compared to c5 and c6i.4xlarge instances. For other models and frameworks, we measured at least 30% cost savings compared to c5 and c6i.4xlarge instances.

Similar to the preceding inference cost comparison graph, the following graph shows the model p90 latency for the same four instance types. We further normalized the latency results to the c5.4xlarge instance, which is measured as 1 in the Y-axis of the chart. The c7g.4xlarge (AWS Graviton3) model inference latency is up to 50% better than the latencies measured on c5.4xlarge and c6i.4xlarge.

Migrate to AWS Graviton instances

To deploy your models to AWS Graviton instances, you can either use AWS Deep Learning Containers (DLCs) or bring your own containers that are compatible with the ARMv8.2 architecture.

The migration (or new deployment) of your models to AWS Graviton instances is straightforward because not only does AWS provide containers to host models with PyTorch, TensorFlow, scikit-learn, and XGBoost, but the models are architecturally agnostic as well. You can also bring your own libraries, but be sure that your container is built with an environment that supports the ARMv8.2 architecture. For more information, see Building your own algorithm container.

You will need to complete three steps in order to deploy your model:

1. Create a SageMaker model. This will contain, among other parameters, the information about the model file location, the container that will be used for the deployment, and the location of the inference script. (If you have an existing model already deployed in a compute optimized inference instance, you can skip this step.)
2. Create an endpoint configuration. This will contain information about the type of instance you want for the endpoint (for example, ml.c7g.xlarge for AWS Graviton3), the name of the model you created in the previous step, and the number of instances per endpoint.
3. Launch the endpoint with the endpoint configuration created in the previous step.

For detailed instructions, refer to Run machine learning inference workloads on AWS Graviton-based instances with Amazon SageMaker

Benchmarking methodology

We used Amazon SageMaker Inference Recommender to automate performance benchmarking across different instances. This service compares the performance of your ML model in terms of latency and cost on different instances and recommends the instance and configuration that gives the best performance for the lowest cost. We have collected the aforementioned performance data using Inference Recommender. For more details, refer to the GitHub repo.

You can use the sample notebook to run the benchmarks and reproduce the results. We used the following models for benchmarking:

• PyTorch – ResNet50 image classification, DistilBERT sentiment analysis, RoBERTa fill mask, and RoBERTa sentiment analysis
• TensorFlow – TF Hub ResNet 50 and ML Commons TensorFlow BERT
• XGBoost and scikit-learn – We tested four models to cover the classifiers, rankers, and linear regression scenarios

Conclusion

AWS measured up to 50% cost savings for PyTorch, TensorFlow, XGBoost, and scikit-learn model inference with AWS Graviton3-based EC2 C7g instances relative to comparable EC2 instances on Amazon SageMaker. You can migrate your existing inference use cases or deploy new ML models on AWS Graviton by following the steps provided in this post. You can also refer to the AWS Graviton Technical Guide, which provides the list of optimized libraries and best practices that will help you achieve cost benefits with AWS Graviton instances across different workloads.

If you find use cases where similar performance gains are not observed on AWS Graviton, please reach out us. We will continue to add more performance improvements to make AWS Graviton the most cost-effective and efficient general-purpose processor for ML inference.

About the authors

Sunita Nadampalli is a Software Development Manager at AWS. She leads Graviton software performance optimizations for machine learning, HPC, and multimedia workloads. She is passionate about open-source development and delivering cost-effective software solutions with Arm SoCs.

Jaymin Desai is a Software Development Engineer with the Amazon SageMaker Inference team. He is passionate about taking AI to the masses and improving the usability of state-of-the-art AI assets by productizing them into features and services. In his free time, he enjoys exploring music and traveling.

Mike Schneider is a Systems Developer, based in Phoenix AZ. He is a member of Deep Learning containers, supporting various Framework container images, to include Graviton Inference. He is dedicated to infrastructure efficiency and stability.

Mohan Gandhi is a Senior Software Engineer at AWS. He has been with AWS for the last 10 years and has worked on various AWS services like EMR, EFA and RDS. Currently, he is focused on improving the SageMaker Inference Experience. In his spare time, he enjoys hiking and marathons.

Qingwei Li is a Machine Learning Specialist at Amazon Web Services. He received his Ph.D. in Operations Research after he broke his advisor’s research grant account and failed to deliver the Nobel Prize he promised. Currently he helps customers in financial service and insurance industry build machine learning solutions on AWS. In his spare time, he likes reading and teaching.

Wayne Toh is a Specialist Solutions Architect for Graviton at AWS. He focuses on helping customers adopt ARM architecture for large scale container workloads. Prior to joining AWS, Wayne worked for several large software vendors, including IBM and Red Hat.

Lauren Mullennex is a Solutions Architect based in Denver, CO. She works with customers to help them architect solutions on AWS. In her spare time, she enjoys hiking and cooking Hawaiian cuisine.

Read More

This is a guest post co-written with Trey Robinson, CTO at Sleepme Inc.

Sleepme is an industry leader in sleep temperature management and monitoring products, including an Internet of Things (IoT) enabled sleep tracking sensor suite equipped with heart rate, respiration rate, bed and ambient temperature, humidity, and pressure sensors.

Sleepme offers a smart mattress topper system that can be scheduled to cool or heat your bed using the companion application. The system can be paired with a sleep tracker that gathers insights such as heart rate, respiration rate, humidity in the room, wake up times, and when the user was in and out of bed. At the end of a given sleep session, it will aggregate sleep tracker insights, along with sleep stage data, to produce a sleep quality score.

This smart mattress topper works like a thermostat for your bed and gives customers control of their sleep climate. Sleepme products help you cool your body temperature, which is linked with falling into a deep sleep, while being hot can reduce the likelihood of falling and staying asleep.

In this post, we share how Sleepme used Amazon SageMaker to developed a machine learning (ML) model proof of concept that recommends temperatures to maximize your sleep score.

“The adoption of AI opens new avenues to improve customers’ sleeping experience. These changes will be implemented in the Sleepme product line, allowing the client to leverage the technical and marketing value of the new features during deployment.”

– Trey Robinson, Chief Technology Officer of Sleepme.

Using ML to improve sleep in real time

Sleepme is a science-driven organization that uses scientific studies, international journals, and cutting-edge research to bring customers the latest in sleep health and wellness. Sleepme provides sleep science information on their website.

Sleepme discusses how only 44% of Americans report a restful night’s sleep almost every night, and that 35% of adults sleep less than 7 hours per night. Getting a full night’s sleep helps you feel more energized and has proven benefits to your mind, weight, and heart. This represents a huge population of people with opportunities to improve their sleep and health.

Sleepme saw an opportunity to improve the sleep of their users by changing the user’s sleep environment during the night. By capturing environment data like temperature and humidity and connecting it with personalized user data like restlessness, heart rate, and sleep cycle, Sleepme determined they were able to change the user’s environment to optimize their rest. This use case demanded an ML model that served real-time inference.

Sleepme needed a highly available inference model that provides low-latency recommendations. With a focus on delivering new features and products for their customers, Sleepme needed an out-of-the-box solution that doesn’t require infrastructure management.

To address these challenges, Sleepme turned to Amazon SageMaker.

Using Amazon SageMaker to build an ML model for sleep temperature recommendations

SageMaker accelerates the deployment of ML workloads by simplifying the ML build process. It provides a set of ML capabilities that run on a managed infrastructure on AWS. This reduces the operational overhead and complexity associated with ML development.

Sleepme chose SageMaker because of the capabilities it provides in model training, endpoint deployment process, and infrastructure management. The following diagram illustrates their AWS architecture.

Sleepme is focused on delivering new products and features for their customers. They didn’t want to dedicate their resources to a lengthy ML model training process.

SageMaker’s Model Training allowed Sleepme to use their historical data to quickly develop a proprietary machine learning model. SageMaker Model Training provides dozens of built-in training algorithms and hundreds of pre-trained models, increasing Sleepme’s agility in model creation. By managing the underlying compute instances, SageMaker Model Training enabled Sleepme to focus on enhancing model performance.

This ML model needed to make sleep environment adjustments in real time. To achieve this, Sleepme used a SageMaker Real-time inference to manage the hosting of their model. This endpoint receives data from Sleepme’s smart mattress topper and sleep tracker to make a temperature recommendation for the user’s sleep in real time. Additionally, with the option for automatic scaling of models, SageMaker inference offered Sleepme the option to add or remove instances to meet demand.

SageMaker also provides Sleepme with useful features as their workload evolves. They could use shadow tests to evaluate model performance of new versions before they are deployed to customers, SakeMaker Model Registry to manage model versions and automate model deployment, and SageMaker Model Monitoring to monitor the quality of their model in production. These features provide Sleepme with the opportunity to take their ML use cases to next level, without developing new capabilities on their own.

Conclusion

With Amazon SageMaker, Sleepme was able to build and deploy a custom ML model in a matter of weeks that identifies the recommended temperature adjustment, which the Sleepme devices mirror to the user’s environment.

Sleepme IoT devices capture sleep data and can now make adjustments to a customer’s bed in minutes. This capability proved to be a business differentiator. Now, users sleep can be optimized to provide a higher-quality sleep in real time.

To learn more about how you can quickly build ML models, refer to the Train Models or get started on the SageMaker Console.

About the Authors

Trey Robinson has been a mobile and IoT-focused software engineer leading teams as the CTO of Sleepme Inc and Director of Engineering at Passport Inc. He has worked on dozens of mobile apps, backends, and IoT projects over the years. Before moving to Charlotte, NC, Trey grew up in Ninety Six, South Carolina, and studied Computer Science at Clemson University.

Benon Boyadjian is a Solutions Architect in the Private Equity group at Amazon Web Services. Benon works directly with Private Equity Firms and their portfolio companies, helping them leverage AWS to achieve business objectives and increase enterprise value.

Read More

Understanding business trends, customer behavior, sales revenue, increase in demand, and buyer propensity all start with data. Exploring, analyzing, interpreting, and finding trends in data is essential for businesses to achieve successful outcomes.

Business analysts play a pivotal role in facilitating data-driven business decisions through activities such as the visualization of business metrics and the prediction of future events. Quick iteration and faster time-to-value can be achieved by providing these analysts with a visual business intelligence (BI) tool for simple analysis, supported by technologies like machine learning (ML).

Amazon QuickSight is a fully managed, cloud-native BI service that makes it easy to connect to your data, create interactive dashboards and reports, and share these with tens of thousands of users, either within QuickSight or embedded in your application or website. Amazon SageMaker Canvas is a visual interface that enables business analysts to generate accurate ML predictions on their own, without requiring any ML experience or having to write a single line of code.

In this post, we show how you can publish predictive dashboards in QuickSight using ML-based predictions from Canvas, without explicitly downloading predictions and importing into QuickSight. This solution will help you send predictions from Canvas to QuickSight, enabling you with accelerated decision-making using ML to achieve effective business outcomes.

Solution overview

In the following sections, we discuss steps that will help administrators configure the right permissions to seamlessly redirect users from Canvas to QuickSight. Then we detail how to build a model and run predictions, and demonstrate the business analyst experience.

Prerequisites

The following prerequisites are needed to implement this solution:

• An AWS account with permissions to create AWS Identity and Access Management (IAM) policies and roles.
• Access to Amazon SageMaker, an instance of Amazon SageMaker Studio, and a user for Studio. For more information about prerequisites, see Getting started with using Amazon SageMaker Canvas.
• A QuickSight subscription. For this post, we only use QuickSight features included in the Standard subscription.
• Access to the QuickSight dashboard to author and analyze the inferred data.

Make sure to use the same QuickSight Region as Canvas. You can change the Region by navigating from the profile icon on the QuickSight console.

Administrator setup

In this section, we detail the steps to set up IAM resources, prepare the data, train the data with the training dataset, and infer the validation dataset. Thereafter, we send the data to QuickSight for further analysis.

Create a new IAM policy for QuickSight access

To create an IAM policy, complete the following steps:

1. On the IAM console, choose Policies in the navigation pane.
2. Choose Create policy.
3. On the JSON tab, enter the following permissions policy into the editor:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "quicksight:CreateDataSet",
                "quicksight:ListNamespaces",
                "quicksight:CreateDataSource",
                "quicksight:PassDataSet",
                "quicksight:PassDataSource"
            ],
            "Resource":[
                "arn:aws:quicksight:*:<AWS-account-id>:datasource/*", #replace account id
                "arn:aws:quicksight:*:<AWS-account-id>:user/*", #replace account id
                "arn:aws:quicksight:*:<AWS-account-id>:namespace/*", #replace account id
                "arn:aws:quicksight:*:<AWS-account-id>:dataset/*" #replace account id
            ]
        }
    ]
}

For details about the IAM policy language, see IAM JSON policy reference.

4. Choose Next: Tags.
5. You can add metadata to the policy by attaching tags as key-value pairs, then choose Next: Review.

For more information about using tags in IAM, see Tagging IAM resources.

6. On the Review policy page, enter a name (for example, canvas-quicksight-access-policy) and an optional description of the policy.
7. Review the Summary section to see the permissions that are granted by your policy.
8. Choose Create policy to save your work.

After you create a policy, you can attach it to your execution role that grants your users the necessary permissions to send batch predictions to users in QuickSight.

Attach the policy to your Studio execution role

To attach the policy to your Studio execution role, complete the following steps:

1. On the SageMaker console, choose Domains in the navigation pane.
2. Choose your domain.
3. Choose Domain settings.
4. Copy the role name under Execution role.

5. On the IAM console, choose Roles in the navigation pane.
6. In the search bar, enter the execution role you copied, then choose the role.

7. On the page for the user’s role, navigate to the Permissions policies section.
8. On the Add permissions menu, choose Attach policies.
9. Search for the previously created policy (canvas-quicksight-access-policy), select it, and choose Add permissions.

Now you have an IAM policy attached to your execution role that grants your users the necessary permissions to send batch predictions to users in QuickSight.

Download the datasets

Let’s download the datasets that we use to train the model and make the predictions:

• Training data
• Test data

Build a model and run predictions

In this section, we cover how we can build a model and run predictions on the loan dataset. Then we send the data to the QuickSight dashboard to get business insights.

Launch Canvas

To launch Canvas, complete the following steps:

1. On the SageMaker console, choose Domains in the navigation pane.
2. Choose your domain.
3. On the Launch menu, choose Canvas.

Upload training and validation datasets

Complete the following steps to upload your datasets to Canvas:

1. On the Canvas home page, choose Datasets.
2. Choose Import data, then upload lending_club_loan_data_train.csv and lending_club_loan_data_test.csv.
   
3. Choose Save & Close, then choose Import data.

Now let’s create new model.

4. Choose My models in the navigation pane.
5. Choose New model.
6. Enter a name to your model (Loan_Prediction) and choose Create.

If this is the first time creating a Canvas model, you will be welcomed by an informative pop-up about how to build your first model in four simple steps. You can read this through, then come back to this guide.

7. In the model view, on the Select tab, select the lending_club_loan_data_train dataset.

This dataset has 18 columns and 32,000 rows.

8. Choose Select dataset.

9. On the Build tab, choose the target column, in our case loan_status.

Canvas will automatically detect that this is a 3+ category prediction problem (also known as multi-class classification).

10. If another model type is detected, change it manually by choosing Change type.

11. Choose Quick build, and select Start quick build from the pop-up.

You can also choose Standard build, which goes through the complete AutoML cycle, generating multiple models before recommending the best model.

Now your model is being built. Quick build usually takes 2–15 minutes.

After the model is built, you can find the model status on the Analyze tab.

Make predictions with the model

After we build and train the model, we can generate predictions on this model.

1. Choose Predict on the Analyze tab, or choose the Predict tab.
2. Run a single prediction by choosing Single prediction and providing entries.

You will see the loan_status prediction on the right side of the page. You can copy the prediction by choosing Copy, or download it by choosing Download prediction. This is ideal for generating what-if scenarios and testing how different columns impact the predictions of our model.

3. To run batch predictions, choose Batch prediction.

This is best when you’d like to make predictions for an entire dataset. You should make predictions with a dataset that matches your input dataset.

For each prediction or set of predictions, Canvas returns the predicted values and the probability of the predicted value being correct.

Let’s make predictions from the trained model using the validation dataset.

4. Choose Select the dataset.
5. Select lending_club_loan_data_test and choose Generate predictions.

When your predictions are ready, you can find them in the Dataset section. You can preview the prediction, download it to a local machine, delete it, or send it to QuickSight.

Send predictions to QuickSight

You can now share predictions from these ML models as QuickSight datasets that will serve as a new source for enterprise-wide dashboards. You can analyze trends, risks, and business opportunities. Through this capability, ML becomes more accessible to business teams so they can accelerate data-driven decision-making. Sharing data with QuickSight users grants them owner permissions on the dataset. Multiple inferred datasets can be sent at once to QuickSight.

Note that you can only send predictions to users in the default namespace of the QuickSight account, and the user must have the Author or Admin role in QuickSight. Predictions sent to QuickSight are available in the same Region as Canvas.

1. Select the inferred batch dataset and choose Send to Amazon QuickSight.

1. Enter one or multiple QuickSight user names to share the dataset with and press Enter.
2. Choose Send to share data.

After you send your batch predictions, the QuickSight field for the datasets you sent shows as Sent.

4. In the confirmation box, you can choose Open Amazon QuickSight to open your QuickSight application.
5. If you’re done using Canvas, log out of the Canvas application.

You can send batch predictions to QuickSight for numeric, categorical prediction, and time series forecasting models. You can also send predictions generated with the bring your own model (BYOM) method. Single-label image prediction and multi-category text prediction models are excluded.

The QuickSight users that you’ve sent datasets to can open their QuickSight console and view the Canvas datasets that have been shared with them. Then they can create predictive dashboards with the data. For more information, see Getting started with Amazon QuickSight data analysis.

By default, all the users to whom you send predictions have owner permissions for the dataset in QuickSight. Owners are able to create analyses, refresh, edit, delete, and reshare datasets. The changes that owners make to a dataset change the dataset for all users with access. To change the permissions, go to the dataset in QuickSight and manage its permissions. For more information, see Viewing and editing the permissions users that a dataset is shared with.

Business analysts experience

With QuickSight, you can visualize your data to better understand it. We start by getting some high-level information.

1. On the QuickSight console, choose Datasets in the navigation pane.
2. Create an analysis on the batch prediction dataset shared from Canvas by choosing Create analysis on the drop-down options menu (three vertical dots).

3. On the analysis page, choose the sheet name and rename to it Loan Data Analysis.

Let’s create a visual to show the count by loan status.

4. For Visual types, choose Donut chart.
5. Use the loan_status field for Group/Color.

We can see that 99% are fully paid, 1% are current, and 0% are charged off.

Now we add a second visual to show the amount of loans by status.

6. On the top-left corner, choose the plus sign and choose Add visual.
7. For Visual types, choose Waterfall chart.
8. Use the loan_status field for Category.
9. Use the loan_amount field for Value.

We can see that the total loan amount is around $88 million, with around $221,000 charged off.

Let’s try to detect some risk drivers for defaulting on loans.

10. Choose the plus sign and choose Add visual.
11. For Visual types, choose Horizontal bar chart.
12. Use the loan_status field for Y axis.
13. Use the loan_amount field for Value.
14. Modify the Value field aggregation from Sum to Average.

We can see that on average, the loan amount was around $3,500 lower for the fully paid loans compared to the current loans, and around $3,500 lower for the fully paid loans compared to the charged off loans. There seems to be a correlation between the loan amount and the credit risk.

15. To duplicate the visual, choose the options menu (three dots), choose Duplicate visual to, and choose This sheet.
16. Choose the duplicated visual to modify its configuration.
17. For Visual types, choose Horizontal bar chart.
18. Use the loan_status field for Y axis.
19. Use the loan_amount field for Value.
20. Modify the Value field aggregation from Sum to Average.

You can create additional visuals to check for additional risk drivers. For example:

• Loan term
• Open credit lines
• Revolving line utilization rate
• Total credit lines

21. After you add the visuals, publish the dashboard using the Share option on the analyses page and share the dashboard with the business stakeholders.

Clean up

To avoid incurring future charges, delete or shut down the resources you created while following this post. Refer to Logging out of Amazon SageMaker Canvas for more details.

Conclusion

In this post, we trained an ML model using Canvas without writing a single line of code thanks to its user-friendly interfaces and clear visualizations. We then generated single and batch predictions for this model in Canvas. To assess the trends, risks, and business opportunities across the enterprise, we sent the predictions of this ML model to QuickSight. As business analysts, we created various visualizations to assess the trends in QuickSight.

This capability is available in all Regions where Canvas is now supported. You can learn more on the Canvas product page and documentation.

About the Authors

Ajjay Govindaram is a Senior Solutions Architect at AWS. He works with strategic customers who are using AI/ML to solve complex business problems. His experience lies in providing technical direction as well as design assistance for modest to large-scale AI/ML application deployments. His knowledge ranges from application architecture to big data, analytics, and machine learning. He enjoys listening to music while resting, experiencing the outdoors, and spending time with his loved ones.

Varun Mehta is a Solutions Architect at AWS. He is passionate about helping customers build enterprise-scale well-architected solutions on the AWS Cloud. He works with strategic customers who are using AI/ML to solve complex business problems.

Shyam Srinivasan is a Principal Product Manager on the AWS AI/ML team, leading product management for Amazon SageMaker Canvas. Shyam cares about making the world a better place through technology and is passionate about how AI and ML can be a catalyst in this journey.

Read More