Monthly Archives: August 2022

Amazon Rekognition Custom Labels is a fully managed computer vision service that allows developers to build custom models to classify and identify objects in images that are specific and unique to your business. Rekognition Custom Labels doesn’t require you to have any prior computer vision expertise. For example, you can find your logo in social media posts, identify your products on store shelves, classify machine parts in an assembly line, distinguish healthy and infected plants, or detect animated characters in videos.

Developing a custom model to analyze images is a significant undertaking that requires time, expertise, and resources, often taking months to complete. Additionally, it often requires thousands or tens of thousands of hand-labeled images to provide the model with enough data to accurately make decisions. Generating this data can take months to gather and requires large teams of labelers to prepare it for use in machine learning (ML).

Rekognition Custom Labels builds off of the existing capabilities of Amazon Rekognition, which are already trained on tens of millions of images across many categories. Instead of thousands of images, you simply need to upload a small set of training images (typically a few hundred images or less) that are specific to your use case using the Amazon Rekognition console. If the images are already labeled, you can begin training a model in just a few clicks. If not, you can label them directly on the Rekognition Custom Labels console, or use Amazon SageMaker Ground Truth to label them. Rekognition Custom Labels uses transfer learning to automatically inspect the training data, select the right model framework and algorithm, optimize the hyperparameters, and train the model. When you’re satisfied with the model accuracy, you can start hosting the trained model with just one click.

Today we’re happy to announce the launch of the Rekognition Custom Labels model copy feature. This feature allows you to copy your Rekognition Custom Labels models across projects, which can be in the same AWS account or across AWS accounts in the same AWS Region, without retraining the models from scratch. This new capability makes it easier for you to move Rekognition Custom Labels models through various environments such as development, quality assurance, integration, and production without needing to copy the original training and test datasets and retraining the model. You can use the AWS Command Line Interface (AWS CLI) to copy trained models across projects, which can be in the same AWS account or across AWS accounts.

In this post, we show you how to copy models between different AWS accounts in the same AWS Region.

Benefits of the model copy feature

This new feature has the following benefits:

• Multi-account ML-Ops best practices – You can train a model one time and ensure predictable deployment with consistent results across multiple accounts mapped to various environments such as development, quality assurance, integration, and production allowing you to follow ML-Ops best practices within your organization.
• Cost savings and faster deployment – You can quickly copy a trained model between accounts, avoiding the time taken to retrain in every account and saving on the model retraining cost.
• Protect sensitive datasets – You no longer need to share the datasets between different AWS accounts or users. The training data needs to be available only on the AWS account where model training is done. This is very important for certain industries, where data isolation is essential to meet business or regulatory requirements.
• Easy collaboration – Partners or vendors can now easily train Amazon Rekognition Custom Labels model in their own AWS account and share the models with users across AWS accounts.
• Consistent performance – Model performance is now consistent across different AWS accounts. Model training is generally non-deterministic and two models trained with the same dataset does not guarantee the same performance scores and the same predictions. Copying the model helps make sure that the behavior of the copied model is consistent with the source model eliminating the need to re-test the model.

Solution overview

The following diagram illustrates our solution architecture.

This post assumes you have a trained a Rekognition Custom Labels model in your source account. For instructions, refer to Training a custom single class object detection model with Amazon Rekognition Custom Labels. In this post, we used the image classification “Rooms” project from the Rekognition Custom Labels sample projects list and trained a room classification model in the source account to classify images of kitchens, bathrooms, living rooms, and more.

To demonstrate the functionality of the model copy feature, we go through the following steps in the source account:

1. Start the model and run inferences on sample images.
2. Define a resource-based policy to allow cross-account access to copy the Rekognition Custom Labels model.

Then we copy the source model to the target account.

3. Create an Amazon Simple Storage Service (Amazon S3) bucket, which serves as a container for the model evaluation and performance statistics.
4. Create a project.
5. Copy the trained model from the source account to the target account.
6. Start the model and run inference on the sample images.
7. Verify the inference results match the results of the source account model.

Prerequisites

In addition to having a trained model in your source account, make sure you complete the following prerequisite steps:

1. Install the AWS CLI V2.
2. Configure your AWS CLI with the following code and enter your Region:

   aws configure

3. Run the following commands to ensure you have AWS CLI version 2.xx installed on your local host:

   aws --version

4. Update the AWS credentials file under $HOME/.aws/credentials with the following entry:

   [source-account]
   aws_access_key_id = ####
   aws_secret_access_key = #######
   
   [target-account]
   aws_access_key_id = ####
   aws_secret_access_key = #######

5. Get the ProjectArn and ProjectVersionArn for the source AWS account.ProjectArn is the project associated with your source model. ProjectVersionArn is the version of the model you’re interested in copying to the target account.You can find the SourceProjectArn using the following command:

   aws rekognition describe-projects 
   --region us-east-1 
   --profile source-account
   
   {
       "ProjectDescriptions": [{
           "ProjectArn": "arn:aws:rekognition:us-east-1::111111111111:project/rooms_1/1657588855531",
           .
           .
       }]
   }

   If you see multiple lines of output, pick the ProjectArn associated with the model you’re going to copy.

   You can find the SourceProjectVersionArn for the model you trained using the SourceProjectArn (the preceding output). Replace the SourceProjectArn in the following command:

   aws rekognition describe-project-versions 
   --project-arn SourceProjectArn 
   --region us-east-1 
   --profile source-account

   The command returns the SourceProjectVersionArn. If you see multiple lines of output, pick the ProjectVersionArn of interest.

   {
       "ProjectVersionDescriptions": [
           {
               "ProjectVersionArn": "arn:aws:rekognition:us-east-1:111111111111:project/rooms_1/version/rooms_1.2022-07-12T09.39.36/1657643976475",
               .
               .
           }
       ]
   }

You’re now ready to run the steps to implement the solution. Replace the values of SourceProjectArn and SourceProjectVersionArn in the following commands with the values you generated.

1. Start the model and run inference on sample images

In the source account, enter the following code to start the model:

aws rekognition start-project-version 
--project-version-arn SourceProjectVersionArn 
--min-inference-units 1 
--region us-east-1 
--profile source-account
{
    "Status": "STARTING"
}

After the model is hosted and in the running state, you can run inference.

We used the following images (demo1.jpeg and demo2.jpeg) to run inference. These images are located in our local file system in the same directory where the AWS CLI commands are being run from.

The following image is demo1.jpeg, which shows a backyard.

See the following inference code and output:

aws rekognition detect-custom-labels 
--project-version-arn SourceProjectVersionArn   
--image-bytes fileb://demo1.jpeg 
--region us-east-1 
--profile source-account
{
    "Name": "backyard",
    "Confidence": 45.77000045776367
 }

The following image is demo2.jpeg, which shows a bedroom.

See the following inference code and output:

aws rekognition detect-custom-labels 
--project-version-arn SourceProjectVersionArn   
--image-bytes fileb://demo2.jpeg 
--region us-east-1 
--profile source-account
{
    "Name": "bedroom",
    "Confidence": 61.84600067138672
 }

The inference results show the image belongs to the classes backyard and bedroom, with a confidence score of 45.77 and 61.84, respectively.

2. Define the IAM resource policy for the trained model to allow cross-account access

To create your resource-based IAM policy, complete the following steps in the source account:

1. Allow your specific AWS account to access resources using the provided IAM resource policy (for more information, refer to Creating a project policy document. Replace the values for TargetAWSAccountId and SourceProjectVersionArn in the following policy:

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Principal": {
                   "AWS": [ "TargetAWSAccountId" ]
               },
               "Action": "Rekognition:CopyProjectVersion",
               "Resource": "SourceProjectVersionArn",
               "Effect": "Allow"
           }
       ]
   }

2. Attach the policy to the project in the source account by calling the following command.

   aws rekognition put-project-policy 
   --project-arn SourceProjectArn 
   --policy-name PolicyName 
   --policy-document '{
       "Version": "2012-10-17",
       "Statement": [
           {
               "Principal": {
                   "AWS": [ "TargetAWSAccountId" ]
               },
               "Action": "Rekognition:CopyProjectVersion",
               "Resource": "SourceProjectVersionArn",
               "Effect": "Allow"
           }
       ]
   }' 
   --region us-east-1 
   --profile source-account

   Replace SourceProjectArn, PolicyName, TargetAWSAccountId, and SourceProjectVersionArn.

   The output shows the policy revision ID created:

   {
       "PolicyRevisionId": "f95907f9c1472c114f61b0e1f31ed131"
   }

Now we’re ready to copy the trained model from the source account to the target account.

3. Create an S3 bucket in the target account

You can use an existing S3 bucket in your account or create a new S3 bucket. For this post, we call this S3 bucket DestinationS3Bucket.

4. Create a new Rekognition Custom Labels project

Create a new project with the following code:

aws rekognition create-project 
--project-name target_rooms_1 
--region us-east-1 
--profile target-account 

This creates a TargetProjectArn in the target account:

{
    "ProjectArn": "arn:aws:rekognition:us-east-1:222222222222:project/target_rooms_1/1657599660206"
}

Note the value of the destination project ProjectArn field. We use this value in the following copy model command.

5. Copy the model from the source account to the target account

Supply the source and target ProjectArn, source ProjectVersionArn, and target S3 bucket and S3 key prefix in the following code:

aws rekognition copy-project-version 
--source-project-arn SourceProjectArn 
--source-project-version-arn SourceProjectVersionArn 
--destination-project-arn TargetProjectArn 
--version-name TargetVersionName 
--output-config '{"S3Bucket":"DestinationS3Bucket", "S3KeyPrefix":"DestinationS3BucketPrefix"}' 
--region us-east-1 
--profile target-account

This creates a copied model TargetProjectVersionArn in the target account. The TargetVersionName in our case has been named copy_rooms_1:

{
    "ProjectVersionArn": "arn:aws:rekognition:us-east-1:222222222222:project/target_rooms_1/version/copy_rooms_1/1657667877079"
}

Check the status of the model copy process:

aws rekognition describe-project-versions 
--project-arn TargetProjectArn 
--version-names TargetVersionName 
--region us-east-1 
--profile target-account

The model copy from the source account to the target account is complete when the Status changes to COPYING_COMPLETED:

 {
    "ProjectVersionDescriptions": [
        {
            "ProjectVersionArn": "arn:aws:rekognition:us-east-1:222222222222:project/target_rooms_1/version/copy_rooms_1/1657667877079",
            "CreationTimestamp": "2022-07-12T16:17:57.079000-07:00",
            "Status": "COPYING_COMPLETED",
            "StatusMessage": "Model copy operation was successful",
            ..........
            ..........
            "EvaluationResult": {
                "F1Score": 0.0,
                "Summary": {

6. Start the model and run inference

Enter the following code to start the model in the target account:

aws rekognition start-project-version 
--project-version-arn TargetProjectArn 
--min-inference-units 1 
--region us-east-1 
--profile target-account
{
    "Status": "STARTING"
}

Check the status of the model:

aws rekognition describe-project-versions 
--project-arn TargetProjectArn 
--version-names copy_rooms_1 
--region us-east-1 
--profile target-account

The model is now hosted and running:

{
    "ProjectVersionDescriptions": [
        {
            "ProjectVersionArn": "arn:aws:rekognition:us-east-1:222222222222:project/target_rooms_1/version/copy_rooms_1/1657667877079",
            "CreationTimestamp": "2022-07-12T16:17:57.079000-07:00",
            "MinInferenceUnits": 1,
            "Status": "RUNNING",
            "StatusMessage": "The model is running.",
            ..........
            ..........
        }
    ]
}

Run inference with the following code:

aws rekognition detect-custom-labels 
 --project-version-arn TargetProjectVersionArn 
 --image-bytes fileb://demo1.jpeg 
 --region us-east-1 
 --profile target-account
{
    "Name": "backyard",
    "Confidence": 45.77000045776367
 }
aws rekognition detect-custom-labels 
 --project-version-arn TargetProjectVersionArn 
 --image-bytes fileb://demo2.jpeg 
 --region us-east-1 
 --profile target-account
{
    "Name": "bedroom",
    "Confidence": 61.84600067138672

7. Verify the inference results match

The classes and the confidence scores for the images demo1.jpg and demo2.jpg in the target account should match the results in the source account.

Conclusion

In this post, we demonstrated the Rekognition Custom Label model copy feature. This feature enables you to train a classification or object detection model in one account and then share the model with another account in the same Region. This simplifies the multi-account strategy where the model can be trained one time and shared between accounts within the same Region without having to retrain or share the training datasets. This allows for a predicable deployment in every account as part of your MLOps workflow. For more information, refer to Copying an Amazon Rekognition Custom Labels model, or try out the walkthrough in this post using a cloud shell with the AWS CLI.

As of this writing, the model copy feature in Amazon Rekognition Custom Labels is available in the following Regions:

• US East (Ohio)
• US East (N. Virginia)
• US West (Oregon)
• Asia Pacific (Mumbai)
• Asia Pacific (Seoul)
• Asia Pacific (Singapore)
• Asia Pacific (Sydney)
• Asia Pacific (Tokyo)
• EU (Frankfurt)
• EU (Ireland)
• EU (London)

Give the feature a try, and please send us feedback either via the AWS forum for Amazon Rekognition or through your AWS support contacts.

About the authors

Amit Gupta is a Senior AI Services Solutions Architect at AWS. He is passionate about enabling customers with well-architected machine learning solutions at scale.

Yogesh Chaturvedi is a Solutions Architect at AWS with a focus in computer vision. He works with customers to address their business challenges using cloud technologies. Outside of work, he enjoys hiking, traveling, and watching sports.

Aakash Deep is a Senior Software Engineer with AWS. He enjoys working on computer vision, AI, and distributed systems. Outside of work, he enjoys hiking and traveling.

Pashmeen Mistry is the Senior Product Manager for Amazon Rekognition Custom Labels. Outside of work, Pashmeen enjoys adventurous hikes, photography, and spending time with his family.

Read More

Medical imaging techniques like computed tomography (CT), magnetic resonance imaging (MRI), medical x-ray imaging, ultrasound imaging, and others are commonly used by doctors for various reasons. Some examples include detecting changes in the appearance of organs, tissues, and vessels, and detecting abnormalities such as tumors and various other type of pathologies.

Before doctors can use the data from those techniques, the data needs to be transformed from its native raw form to a form that can be displayed as an image on a computer screen.

This process is known as image reconstruction, and it plays a crucial role in a medical imaging workflow—it’s the step that creates diagnostic images that can be then reviewed by doctors.

In this post, we discuss a use case of MRI reconstruction, but the architectural concepts can be applied to other types of image reconstruction.

Advances in the field of image reconstruction have led to the successful application of AI-based techniques within magnetic resonance (MR) imaging. These techniques are aimed at increasing the accuracy of the reconstruction and in the case of MR modality, and decreasing the time required for a full scan.

Within MR, applications using AI to work with under-sampled acquisitions have been successfully employed, achieving nearly ten times reduction in scan times.

Waiting times for tests like MRIs and CT scans have increased rapidly in the last couple of years, leading to wait times as long as 3 months. To ensure good patient care, the increasing need for quick availability of reconstructed images along with the need to reduce operational costs has driven the need of a solution capable of scaling according to storage and computational needs.

In addition to computational needs, data growth has seen a steady increase in the last few years. For example, looking at the datasets made available by the Medical Image Computing and Computer-Assisted Intervention (MICCAI), it’s possible to gather that the annual growth is 21% for MRI, 24% for CT, and 31% for functional MRI (fMRI). (For more information, refer to Dataset Growth in Medical Image Analysis Research.)

In this post, we show you a solution architecture that addresses these challenges. This solution can enable research centers, medial institutions, and modality vendors to have access to unlimited storage capabilities, scalable GPU power, fast data access for machine learning (ML) training and reconstruction tasks, simple and fast ML development environments, and the ability to have on-premises caching for fast and low-latency image data availability.

Solution overview

This solution uses an MRI reconstruction technique known as Robust Artificial-neural-networks for k-space Interpolation (RAKI). This approach is advantageous because it’s scan-specific and doesn’t require prior data to train the neural network. The drawback to this technique is that it requires a lot of computational power to be effective.

The AWS architecture outlined shows how a cloud-based reconstruction approach can effectively perform computational-heavy tasks like the one required by the RAKI neural network, scaling according to the load and accelerating the reconstruction process. This opens the door to techniques that can’t realistically be implemented on premises.

Data layer

The data layer has been architected around the following principles:

• Seamless integration with modalities that store data generated into an attached storage drive via a network share on a NAS device
• Limitless and secure data storage capabilities to scale to the continuous demand of storage space
• Fast storage availability for ML workloads such as deep neural training and neural image reconstruction
• The ability to archive historic data using a low-cost, scalable approach
• Permit availability to the most frequently accessed reconstructed data while simultaneously keeping less frequently accessed data archived at a lower cost

The following diagram illustrates this architecture.

This approach uses the following services:

• AWS Storage Gateway for a seamless integration with the on-premises modality that exchanges information via a file share system. This allows transparent access to the following AWS Cloud storage capabilities while maintaining how the modality exchanges data:
  • Fast cloud upload of the volumes generated by the MR modality.
  • Low-latency access to frequently used reconstructed MR studies via local caching offered by Storage Gateway.
• Amazon SageMaker for unlimited and scalable cloud storage. Amazon S3 also provides low-cost, historical raw MRI data deep archiving with Amazon S3 Glacier, and an intelligent storage tier for the reconstructed MRI with Amazon S3 Intelligent-Tiering.
• Amazon FSx for Lustre for fast and scalable intermediate storage used for ML training and reconstruction tasks.

The following figure shows a concise architecture describing the data exchange between the cloud environments.

Using Storage Gateway with the caching mechanism allows on-premises applications to quickly access data that’s available on the local cache. This occurs while simultaneously giving access to scalable storage space on the cloud.

With this approach, modalities can generate raw data from acquisition jobs, as well as write the raw data into a network share handled from Storage Gateway.

If the modality generates multiple files that belong to the same scan, it’s recommended to create a single archive (.tar for example), and perform a single transfer to the network share to accelerate the data transfer.

Data decompression and transformation layer

The data decompression layer receives the raw data, automatically performs decompression, and applies potential transformations to the raw data before submitting the preprocessed data to the reconstruction layer.

The adopted architecture is outlined in the following figure.

In this architecture, raw MRI data lands in the raw MRI S3 bucket, thereby triggering a new entry in Amazon Simple Queue Service (Amazon SQS).

An AWS Lambda function retrieves the raw MRI Amazon SQS queue depth, which represents the amount of raw MRI acquisitions uploaded to the AWS Cloud. This is used with AWS Fargate to automatically modulate the size of an Amazon Elastic Container Service (Amazon ECS) cluster.

This architecture approach lets it automatically scale up and down accordingly to the number of raw scans landed into the raw input bucket.

After the raw MRI data is decompressed and preprocessed, it’s saved into another S3 bucket so that it can be reconstructed.

Neural model development layer

The neural model development layer consists of a RAKI implementation. This creates a neural network model to allow the fast image reconstruction of under-sampled magnetic resonance raw data.

The following figure shows the architecture that realizes the neural model development and container creation.

In this architecture, Amazon SageMaker is used to develop the RAKI neural model, and simultaneously to create the container that is later used to perform the MRI reconstruction.

Then, the created container is included in the fully managed Amazon Elastic Container Registry (Amazon ECR) repository so that it can then spin off reconstruction tasks.

Fast data storage is guaranteed by the adoption of Amazon FSx for Lustre. It provides sub-millisecond latencies, up to hundreds of GBps of throughput, and up to millions of IOPS. This approach gives SageMaker access to a cost-effective, high-performance, and scalable storage solution.

MRI reconstruction layer

The MRI reconstruction based on the RAKI neural network is handled by the architecture shown in the following diagram.

With the same architectural pattern adopted in the decompression and preprocessing layer, the reconstruction layer automatically scales up and down by analyzing the depth of the queue responsible for holding all the reconstruction requests. In this case, to enable GPU support, AWS Batch is used to run the MRI reconstruction jobs.

Amazon FSx for Lustre is used to exchange the large amount of data involved in MRI acquisition. Furthermore, when a reconstruction job is complete and the reconstructed MRI data is stored in the target S3 bucket, the architecture employed automatically requests a refresh of the storage gateway. This makes the reconstructed data available to the on-premises facility.

Overall architecture and results

The overall architecture is shown in the following figure.

We applied the described architecture on MRI reconstruction tasks with datasets approximately 2.4 GB in size.

It took approximately 210 seconds to train 221 datasets, for a total of 514 GB of raw data on a single node equipped with a Nvidia Tesla V100-SXM2-16GB.

The reconstruction, after the RAKI network has been trained, took an average of 40 seconds on a single node equipped with a Nvidia Tesla V100-SXM2-16GB.

The application of the preceding architecture to a reconstruction job can yield the results in the following figure.

The image shows that good results can be obtained via reconstruction techniques such as RAKI. Moreover, adopting cloud technology can make these computation-heavy approaches available without the limitations found in on-premises solutions where storage and computational resources are always limited.

Conclusions

With tools such as Amazon SageMaker, Amazon FSx for Lustre, AWS Batch, Fargate, and Lambda, we can create a managed environment that is scalable, secure, cost-effective, and capable of performing complex tasks such as image reconstruction at scale.

In this post, we explored a possible solution for image reconstruction from raw modality data using a computationally intensive technique known as RAKI: a database free deep learning technique for fast image reconstruction.

To learn more about how AWS is accelerating innovation in healthcare, visit AWS for Health.

References

• Scan-specific Robust Artificial-neural-networks for k-space Interpolation (RAKI): Database free Deep Learning Reconstruction for Fast Imaging
• MRI datasets used in this post

About the author

Benedetto Carollo is the Senior Solution Architect for medical imaging and healthcare at Amazon Web Services in Europe, Middle East, and Africa. His work focuses on helping medical imaging and healthcare customers solve business problems by leveraging technology. Benedetto has over 15 years of experience of technology and medical imaging and has worked for companies like Canon Medical Research and Vital Images. Benedetto received his summa cum laude MSc in Software Engineering from the University of Palermo – Italy.

Read More