Monthly Archives: September 2024

Time series data is a distinct category that incorporates time as a fundamental element in its structure. In a time series, data points are collected sequentially, often at regular intervals, and they typically exhibit certain patterns, such as trends, seasonal variations, or cyclical behaviors. Common examples of time series data include sales revenue, system performance data (such as CPU utilization and memory usage), credit card transactions, sensor readings, and user activity analytics.

Time series anomaly detection is the process of identifying unexpected or unusual patterns in data that unfold over time. An anomaly, also known as an outlier, occurs when a data point deviates significantly from an expected pattern.

For some time series, like those with well-defined expected ranges such as machine operating temperatures or CPU usage, a threshold-based approach might suffice. However, in areas like fraud detection and sales, where simple rules fall short due to their inability to catch anomalies across complex relationships, more sophisticated techniques are required to identify unexpected occurrences.

In this post, we demonstrate how to build a robust real-time anomaly detection solution for streaming time series data using Amazon Managed Service for Apache Flink and other AWS managed services.

Solution overview

The following diagram illustrates the core architecture of the Anomaly Detection Stack solution.

This solution employs machine learning (ML) for anomaly detection, and doesn’t require users to have prior AI expertise. It offers an AWS CloudFormation template for straightforward deployment in an AWS account. With the CloudFormation template, you can deploy an application stack with the necessary AWS resources required for detecting anomalies. Setting up one stack creates an application with one anomaly detection task or detector. You can set up multiple such stacks to run them simultaneously, with each one analyzing the data and reporting back the anomalies.

The application, once deployed, constructs an ML model using the Random Cut Forest (RCF) algorithm. It initially sources input time series data from Amazon Managed Streaming for Apache Kafka (Amazon MSK) using this live stream for model training. Post-training, the model continues to process incoming data points from the stream. It evaluates these points against the historical trends of the corresponding time series. The model also generates an initial raw anomaly score while processing and maintains an internal threshold to eliminate noisy data points. Subsequently, the model generates a normalized anomaly score for each data point that the model treats as an anomaly. These scores, ranging from 0–100, indicate the deviation from typical patterns; scores closer to 100 signify higher anomaly levels. You have the flexibility to set a custom threshold on these anomaly scores, allowing you to define what you consider anomalous.

This solution uses a CloudFormation template, which takes inputs such as MSK broker endpoint and topics, AWS Identity and Access Management (IAM) roles, and other parameters related to virtual private cloud (VPC) configuration. The template creates the essential resources like the Apache Flink application and Amazon SageMaker real-time endpoint in the customer account.

To request the access to this solution, send an email to anomalydetection-support-canvas@amazon.com.

In this post, we outline how you can build an end-to-end solution with the Anomaly Detection Stack. Consider a hypothetical sales scenario where AnyBooks, an on-campus bookstore at a large university, sells various supplies to college students. Due to the timing of class schedules, their seasonality is such that they sell around 20 Item-A units and 30 Item-B units during even hours, and approximately half that during odd hours throughout the day. Recently, there have been some unexplained spikes in the quantity of items sold, and the management team wants to start tracking these quantity anomalies so that they can better plan their staffing and inventory levels.

The following diagram shows the detailed architecture for the end-to-end solution.

In the following sections, we discuss each layer shown in the preceding diagram.

Ingestion

In the ingestion layer, an AWS Lambda function retrieves sales transactions for the current minute from a PostgreSQL transactional database, transforms each record into a JSON message, and publishes it to an input Kafka topic. This Lambda function is configured to run every minute using Amazon EventBridge Scheduler.

Anomaly detection stack

The Flink application initiates the process of reading raw data from the input MSK topic, training the model, and commencing the detection of anomalies, ultimately recording them to the MSK output topic. The following code is the output results JSON:

{"detectorName":"canvas-ad-blog-demo-1","measure":"quantity","timeseriesId":"f3c7f14e7a445b79a3a9877dfa02064d56533cc29fb0891945da4512c103e893","anomalyDecisionThreshold":70,"dimensionList":[{"name":"product_name","value":"item-A"}],"aggregatedMeasureValue":14.0,"anomalyScore":0.0,"detectionPeriodStartTime":"2024-08-29 13:35:00","detectionPeriodEndTime":"2024-08-29 13:36:00","processedDataPoints":1261,"anomalyConfidenceScore":80.4674989791107,"anomalyDecision":0,"modelStage":"INFERENCE","expectedValue":0.0}

The following is a brief explanation of the output fields:

• measure – This represents the metric we are tracking for anomalies. In our case, the measure field is the quantity of sales for Item-A.
• aggregatedMeasureVaue – This represents the aggregated value of quantity in the time window.
• timeseriesid – This unique identifier corresponds to a combination of unique values for the dimensions and the metric. In this scenario, it’s the product name, Item-A, within the product_name
• anomalyConfidenceScore – As the model evolves through learning and inference, this confidence score will progressively improve.
• anomalyScore – This field represents the score for anomaly detection. With an anomalyThreshold set at 70, any value exceeding 70 is considered a potential anomaly.
• modelStage – When the model is in the learning phase, the anomalyScore is 0.0 and the value of this field is set to LEARNING. After the learning is complete, the value of this field changes to INFERENCE.
• anomalyDecisionThreshold – The decision threshold is provided as input in the CloudFormation stack. If you determine there are too many false positives, you can increase this threshold to change the sensitivity.
• anomalyDecision – If the anomalyScore exceeds the anomalyDecisionThreshold, this field is set to 1, indicating an anomaly is detected.

Transform

In the transformation layer, an Amazon Data Firehose stream is configured to consume data from the output Kafka topic and invoke a Lambda function for transformation. The Lambda function flattens the nested JSON data from the Kafka topic. The transformed results are then partitioned by date and stored in an Amazon Simple Storage Service (Amazon S3) bucket in Parquet format. An AWS Glue crawler is used to crawl the data in the Amazon S3 location and catalog it in the AWS Glue Data Catalog, making it ready for querying and analysis.

Visualize

To visualize the data, we’ve created an Amazon QuickSight dashboard that connects to the data in Amazon S3 through the Data Catalog and queries it using Amazon Athena. The dashboard can be refreshed to display the latest detected anomalies, as shown in the following screenshot.

In this example, the darker blue line in the line graph represents the seasonality of the quantity measure for Item-A over time, showing higher values during even hours and lower values during odd hours. The pink line represents the anomaly detection score, plotted on the right Y-axis. The anomaly score approaches 100 when the quantity value significantly deviates from its seasonal pattern. The blue line represents the anomaly threshold, set at 70. When anomalyScore exceeds this threshold, anomalyDecision is set to 1.

The “Number of Timeseries Tracked” KPI displays how many time series the model is currently monitoring. In this case, because we’re tracking two products (Item-A and Item-B), the count is 2. The “Number of Datapoints Processed” KPI shows the total number of data points the model has processed, and the “Anomaly Confidence Score” indicates the confidence level in predicting anomalies. Initially, this score is low, but will approach 100 as the model matures over time.

Notification

Although visualization is valuable for investigating anomalies, data analysts often prefer to receive near real-time notifications for critical anomalies. This is achieved by adding a Lambda function that reads results from the output Kafka topic and analyzes them. If the anomalyScore value exceeds the defined threshold, the function invokes an Amazon Simple Notification Service (Amazon SNS) topic to send email or SMS notifications to a designated list, alerting the team about the anomaly in near real time.

Conclusion

This post demonstrated how to build a robust real-time anomaly detection solution for streaming time series data using Managed Service for Apache Flink and other AWS services. We walked through an end-to-end architecture that ingests data from a source database, passes it through an Apache Flink application that trains an ML model and detects anomalies, and then lands the anomaly data in an S3 data lake. The anomaly scores and decisions are visualized through a QuickSight dashboard connected to the Amazon S3 data using AWS Glue and Athena. Additionally, a Lambda function analyzes the results and sends notifications in near real time.

With AWS managed services like Amazon MSK, Data Firehose, Lambda, and SageMaker, you can rapidly deploy and scale this anomaly detection solution for your own time series use cases. This allows you to automatically identify unexpected behaviors or patterns in your data streams in real time without manual rules or thresholds.

Give this solution a try, and explore how real-time anomaly detection on AWS can unlock insights and optimize operations across your business!

About the Authors

Noah Soprala is a Solutions Architect based out of Dallas. He is a trusted advisor to his customers and helps them build innovative solutions using AWS technologies. Noah has over 20 years of experience in consulting, development, and solution architecture and delivery.

Dan Sinnreich is a Sr. Product Manager for Amazon SageMaker, focused on expanding no-code / low-code services. He is dedicated to making ML and generative AI more accessible and applying them to solve challenging problems. Outside of work, he can be found playing hockey, scuba diving, and reading science fiction.

Syed Furqhan is a Senior Software Engineer for AI and ML at AWS. He was part of many AWS service launches like Amazon Lookout for Metrics, Amazon Sagemaker and Amazon Bedrock. Currently, he is focusing on generative AI initiatives as part of Amazon Bedrock Core Systems. He is a clean code advocate and a subject-matter expert on server-less and event-driven architecture. You can follow him on linkedin, syedfurqhan

Nirmal Kumar is Sr. Product Manager for the Amazon SageMaker service. Committed to broadening access to AI/ML, he steers the development of no-code and low-code ML solutions. Outside work, he enjoys travelling and reading non-fiction.

Read More

Technology operations (TechOps) refers to the set of processes and activities involved in managing and maintaining an organization’s IT infrastructure and services. There are several terminologies used with reference to managing information technology operations, including ITOps, SRE, AIOps, DevOps, and SysOps. For the context of this post, we refer to these terminologies as TechOps. This includes tasks such as managing servers, networks, databases, and applications to maintain reliability, performance, and security of IT systems. However, certain tasks require manual and repetitive efforts such as incident detection and response, analyzing incoming tickets from disparate service providers, finding standard operating procedures for known and unknown issues, and managing support case resolution. In recent years, TechOps has been using AI capabilities—called AIOps—for operational data collection, aggregation, and correlation to generate actionable insights, identity root causes, and more.

This post describes how AWS generative AI solutions (including Amazon Bedrock, Amazon Q Developer, and Amazon Q Business) can further enhance TechOps productivity, reduce time to resolve issues, enhance customer experience, standardize operating procedures, and augment knowledge bases. The ability of generative AI technology to interpret complex situations on a nuanced, case-by-case basis implies that generative AI can solve challenges that other approaches—including traditional artificial intelligence and machine learning (AI/ML)-based pattern matching—couldn’t handle. The following table depicts a few examples of how AWS generative AI services can help with some of the day-to-day TechOps activities.

A typical day in the life of a TechOps team includes issue resolution, root cause analysis, maintenance activities, and updating knowledge bases to provide a positive customer experience. In the following sections, we discuss some of these areas and how generative AI can help enhance TechOps.

Event management

By monitoring systems and analyzing patterns in performance data, an AI model can predict issues before they cause outages or degraded service. When incidents do occur, generative AI models can generate preliminary documentation of the event, including details on impacted systems, potential root causes, and troubleshooting steps. This allows engineers to quickly get up to speed on new incidents and accelerate response efforts.

Generative AI can also generate summary reports of past incidents to help teams identify recurring problems and opportunities for preventative measures. Furthermore, it can help with formatting inbound maintenance notifications from various service providers into a standard format, which can speed up understanding the impact of upcoming maintenance. Similarly, generative AI can automatically generate outbound cases to service providers if it detects an anomaly.

By taking over basic documentation and prediction tasks, generative AI can help infrastructure teams spend less time on repetitive work and more time resolving issues to improve overall system reliability.

To learn more about using Amazon Bedrock for summary tasks, refer to Create summaries of recordings using generative AI with Amazon Bedrock and Amazon Transcribe. To learn how Wiz uses Amazon Bedrock to address security risks, see How Wiz is empowering organizations to remediate security risks faster with Amazon Bedrock. To learn how HappyFox uses Anthropic Claude in Amazon Bedrock, refer to HappyFox Automates Support Agent Responses with Claude in Amazon Bedrock, Increasing Ticket Resolution by 40%.

Knowledge base management

Generative AI has the potential to help engineers automatically create operational documents such as standard operating procedures (SOPs) and supplemental documents, such as server hardening, security policies for external IPs allow lists and operating system patching, and more.

Using natural language models trained on large datasets of existing SOPs and similar content, generative AI systems can understand the common structure and language used in these types of documents. Engineers can then provide the system with high-level requirements or parameters for a new procedure, and generative AI can automatically generate a draft document formatted with the appropriate sections, level of detail, and terminology. This allows engineers to spend less time on documentation and more time focused on other engineering tasks. The initial drafts from AI also provide a strong starting point that engineers can refine.

Overall, generative AI offers a more efficient way for engineers to develop standardized procedural content at scale.

To learn how to use Amazon Bedrock to generate product descriptions, see Automating product description generation with Amazon Bedrock. Additionally, refer to How Skyflow creates technical content in days using Amazon Bedrock to learn how Skyflow Inc.—a data privacy company—uses Amazon Bedrock to streamline the creation of technical content, reducing the process from weeks to days while maintaining the highest standards of data privacy and security.

Automation

Generative AI can assist engineers and automate certain tasks that would otherwise require manual work. One area this could help in is script code generation for repetitive automation processes. By training AI models on large datasets of existing code examples for common programming tasks like file operations or system configuration, generative models can learn patterns and syntax.

An Amazon Q customization is a set of elements that enables Amazon Q to provide you with suggestions based on your company’s code base. Engineers can then provide high-level descriptions or specifications of what they need automated, such as “Generate a script to back up and archive files older than 30 days in this directory.” The AI model would be able to produce working code to accomplish this automatically based on its training. This would save engineers considerable time writing and testing scripts for routine jobs, allowing them to focus on more creative and challenging aspects of their work. As generative AI techniques advance, more complex engineering automation may also be achieved.

Refer to Upgrade your Java applications with Amazon Q Code Transformation to learn about the Amazon Q Code Transformation feature. Also, refer to Using Amazon Bedrock Agents to interactively generate infrastructure as code to learn how to configure Amazon Bedrock Agents to generate infrastructure as code. Lastly, refer to TymeX Accelerates Clean Coding by 40% by Implementing Generative AI on AWS to learn how TymeX uses generative AI on AWS.

Customer experience

Generative AI can analyze large volumes of customer service data, like call logs and support tickets, and identify patterns in issues customers frequently report. This insight allows operations teams to proactively address common problems before they severely impact customers. Generative AI assistants can also automate many routine service tasks, freeing up human agents to focus on more complex inquiries that require personalization. With AI assistance, infrastructure services can be restored more quickly when outages occur. This helps make sure operations are more efficient and transparent, directly enhancing the experience for the customers that infrastructure teams aim to support.

Amazon Q Business offers a conversational experience with generative prompts and tasks that can act as a front-line support engineer, answering customer questions and resolving known issues efficiently. The feature can use data from enterprise systems to provide accurate and timely responses, reducing the burden on human engineers and improving customer satisfaction.

With Amazon Bedrock, you can perform sentiment analysis to help analyze customer emotions and provide context to human engineers, enabling them to provide better support and improve customer loyalty, retention, and growth.

Refer to Develop advanced generative AI chat-based assistants by using RAG and ReAct prompting to learn one way to develop generative AI assistants. Refer to Building a Generative AI Contact Center Solution for DoorDash Using Amazon Bedrock, Amazon Connect, and Anthropic’s Claude to learn how DoorDash built a generative AI contact center solution using AWS services. To learn how PGA TOUR built a generative AI virtual assistant, see The journey of PGA TOUR’s generative AI virtual assistant, from concept to development to prototype.

Staff productivity

An all-day infrastructure operations team faces challenges in maintaining staff productivity during off-hours and nights when the volume of support requests is lower. A generative AI assistant can help improve staff productivity in these periods and streamline the shift-handover process.

The assistant can be trained on historical support conversations to understand and resolve a large percentage of routine queries independently. It can communicate with customers on messaging platforms to provide instant assistance. Simple requests that the assistant can address free up the team to focus on complex issues requiring human expertise. The AI system can escalate any queries it can’t resolve on its own to the on-call staff. This allows the night and weekend crew to work with fewer interruptions. They can work through tasks more efficiently knowing the assistant is handling basic support needs independently. Generative AI-powered contact center solutions can improve an agent’s ability to interact with customers more precisely and speed up issue resolution, increasing overall productivity.

To learn how to automate document and data retrieval for AI assistants, see Automate chatbot for document and data retrieval using Amazon Bedrock Agents and Knowledge Bases. Refer to How LeadSquared accelerated chatbot deployments with generative AI using Amazon Bedrock and Amazon Aurora PostgreSQL to learn how LeadSquared uses Amazon Bedrock and Amazon Aurora PostgreSQL-Compatible Edition to deploy generative AI-powered assistants on their Converse platform, which personalize interactions based on customer-specific training data. This integration reduces customer onboarding costs, minimizes manual effort, and improves chatbot responses, transforming customer support and engagement by providing swift and relevant assistance.

Reporting

Generative AI has the potential to help infrastructure operations teams streamline reporting processes. By using ML algorithms trained on past report examples, a generative AI system can automatically generate draft reports based on incoming data from monitoring systems and other operational tools. This can save teams significant time spent compiling information into standardized report formats. The AI-generated reports could include summary data visualizations, descriptive analyses, and recommendations tailored to each recipient.

Teams would still need to review the drafts for accuracy before finalizing and distributing them. However, having an initial version generated automatically could cut down on routine reporting tasks so engineers have more time for higher-value problem-solving and strategic planning work. The use of AI could help infrastructure teams meet their reporting obligations more efficiently.

Amazon Q in QuickSight is your generative AI assistant that makes it straightforward to build and consume insights. For more information, see Amazon Q is now generally available in Amazon QuickSight, bringing Generative BI capabilities to the entire organization. Also, refer to Anthology uses embedded analytics offered by Amazon QuickSight to democratize decision making for higher education to learn how Anthology is using Amazon Q in QuickSight to offer institutions self-serve options for analytics needs that aren’t directly addressed by the central dashboards.

You can explore more customer stories and case studies at Generative AI Customer Stories to learn how customers are using AWS generative AI services. Refer to Derive meaningful and actionable operational insights from AWS Using Amazon Q Business to learn how to use AWS generative AI services, like Amazon Q Business, with AWS Support cases, AWS Trusted Advisor, and AWS Health data to derive actionable insights based on common patterns, issues, and resolutions while using the AWS recommendations and best practices enabled by support data.

Conclusion

Integrating generative AI into TechOps represents a transformative leap in the management and optimization of IT infrastructure and services. By using AWS generative AI solutions such as Amazon Bedrock, Amazon Q Developer, and Amazon Q Business, organizations can significantly enhance productivity, reduce the time required to resolve issues, and improve overall customer experience. Generative AI’s sophisticated capabilities in predicting and preventing outages, automating documentation, and generating actionable insights from operational data position it as a critical tool for modern TechOps teams.

You can unlock unimagined possibilities with generative AI by using the AWS Generative AI Innovation Center program, which pairs you with AWS science and strategy experts with deep experience in AI/ML and generative AI techniques. To get started, contact your AWS Account Manager. If you don’t have an AWS Account Manager, contact AWS Sales.

About the Authors

Raman Pujani is a Solutions Architect at Amazon Web Services, where he helps customers to accelerate their business transformation journey with AWS. He builds simplified and sustainable solutions for complex business problems with innovative technology. Raman has 25+ years of experience in IT Transformation. Besides work, he enjoys spending time with family, vacationing in the mountains, and music.

Rachanee Singprasong is a Principal Customer Solutions Manager in Strategic Accounts at Amazon Web Services. Her role is focused on enabling customer in their cloud adoption and digital transformation journey. She has a Ph.D. in Operations Research and her passion is to solve complex customer challenges using creative solutions.

Vijay Sivaji is a Senior Technical Account Manager in Strategic Accounts at Amazon Web Services. He helps customers in solving architectural, operational and cost optimization challenges. In his spare time he enjoys playing tennis.

Read More

Machine learning operations (MLOps) are a set of practices that automate and simplify machine learning (ML) workflows and deployments. What is MLOps provides a detailed description of this concept. As ML workloads become increasingly complex and consume more energy and resources, a growing number of companies are looking for ways to manage both the costs and the carbon footprint associated with these workloads. AWS published Guidance for Optimizing MLOps for Sustainability on AWS to help customers maximize utilization and minimize waste in their ML workloads.

In this blog post, you will learn how to optimize MLOps for sustainability.

There are three main workflows in the overall process for building, deploying and using ML models, as shown in the following figure. The process begins with data preparation, followed by model training and tuning, and then model deployment and management.

Data preparation

The workflow starts with data preparation, which includes four components: your data stream, Amazon SageMaker Processing job, Amazon SageMaker Feature Store and an Amazon Simple Storage Service (Amazon S3) bucket for raw data, as shown in the following figure.

Data preparation is essential for model training and is also the first phase in the MLOps lifecycle. Optimizing the artificial intelligence and machine learning (AI/ML) data preparation workload on AWS with sustainability best practices helps reduce the carbon footprint and the cost.

The data preparation process can be complex and energy-intensive because of the vast amount of data processing and computations involved. This leads to substantial resource consumption. There are a few things to consider that can help reduce energy consumption.

Start with the AWS Region you choose for your workload. If possible, choose a Region that has low carbon intensity or where the electricity is attributed to 100% renewable energy sources. In addition, consider storing data and training models in the same Region if possible. This reduces the data movement and latency across the network, optimizing the networking resources required.

Using a serverless architecture can help further reduce resource consumption and remove maintenance overhead by provisioning resources only when required. It’s also important to avoid duplication and re-run of code across teams. Look for services such as Amazon SageMaker Feature Store which helps achieve this goal. Finally, choosing the right storage type for the data used for model training can limit the carbon impact of your workload.

For example, by using S3 One Zone-Infrequent Access to store data that isn’t frequently accessed, such as test data and training data, you can optimize the carbon impact of the data stored. Also, using S3 Intelligent-Tiering can help move the data to more energy-efficient tiers based on access patterns.

Model training and tuning

The second area for you to consider is model training and tuning, shown in the following figure.

While data preparation isn’t unique to AI/ML workloads, the model training and tuning workflow is specific to AI/ML. It’s an important step in making the models functionally useful while also reducing the resources required to run them at scale. There are costs in terms of both operations and sustainability. The good news is that optimization for sustainability also helps to optimizing operations.

For example, SageMaker provides the model parallel library to help efficiently distribute and train models on multiple compute nodes. The library has multiple features that can be combined to more efficiently train models from relatively small parameter sets up to sets with hundreds of billions of parameters. The library can also help use the features of Elastic Fabric Adapter (EFA) supported devices to maximize throughput and minimize latency across nodes. Further optimization is possible using SageMaker Training Compiler to compile deep learning models for training on supported GPU instances. SageMaker Training Compiler converts deep learning models from high-level language representation to hardware-optimized instructions. Hardware-optimized instructions can speed up model training by up to 50% by more efficiently using the GPU memory and using a larger batch size per iteration, all without altering the final trained model.

To reduce the time and energy required to tune a model, SageMaker automatic model tuning (AMT) runs multiple training jobs on a given dataset; it then uses the results to converge on a set of hyperparameter values to create the best performing model for a given metric. There are multiple approaches to the process of searching for the right hyperparameter ranges. For example, Bayesian optimization typically requires 10 times fewer jobs to find the best set of values compared to other methods, reducing the resource usage and carbon footprint of the process.

Right-sizing is another method for managing resource usage and minimizing the environmental impact of your workloads. SageMaker debugger helps to optimize resource consumption by detecting under-utilization of system resources, identifying training problems, and using built-in rules to monitor and stop training jobs as soon as bugs are detected.

Data pre- and post-processing and model evaluation tasks can be run as Amazon SageMaker Processing jobs. In addition to evaluating the accuracy of your models, processing jobs help you to make informed decisions about the tradeoffs between a model’s accuracy and its carbon footprint. Thus, you can establish performance criteria that support your sustainability goals while meeting your business requirements. SageMaker Processing also provides Amazon CloudWatch logs and metrics that can be used for monitoring and right-sizing jobs based on CPU, memory, GPU, GPU memory, and disk metrics.

Dedicated Amazon Elastic Compute Cloud (Amazon EC2) Trn1 instances provide both efficiency and environmental benefits for running your training jobs. These instances use Trainium processors: purpose-built chips designed specifically for deep learning training of models that can exceed 100 billion parameters. Each Trn1 instance provides up to 16 Trainium accelerators, ensuring that jobs will be both efficient and cost optimized. EC2 Trn1 instances offer up to 52% cost-to-train savings compared to comparable EC2 instance types.

Next, you can use governance to share information about the environmental impact of your model. Amazon SageMaker Model Cards provide versioned records documenting various aspects and attributes of your model. This allows you to share the intended uses and assessed carbon impact of a model so that data scientists, ML engineers, and other teams can make informed decisions when choosing and running models.

Model deployment and management

The last area of MLOps is deployment and management, shown in the following figure.

Automating the deployment of ML models provides several sustainability benefits. The deployed model can use a lot of resources when data or code is updated and retrained. You want to ensure that the deployed model is as efficient as possible to reduce the carbon footprint of the workload.

One approach is to use Amazon SageMaker Model Registry. This feature helps improve sustainability and resource optimization by providing a centralized repository for cataloging ML models and reducing redundancy. This approach improves model reusability by allowing existing models to be fine-tuned, rather than training new models from scratch. Consider running your deployment code using AWS CodePipeline to ensure repeatability and version control and optimize resource utilization by running only the necessary stages in the pipeline. This helps your workloads remove the waste associated with manual processes and supports incremental improvements over time.

If your workloads can tolerate latency, consider deploying your model on Amazon SageMaker Asynchronous Inference with auto-scaling groups. This can help minimize idle resources and reduce the impact of load spikes. This also means you pay for compute only when the endpoint is actively handling inference requests. Alternatively, if you don’t need real-time inference, use batch transform. Unlike persistent endpoints, clusters are decommissioned when a batch transform job is complete. Batch transform automatically partitions large datasets and distributes workloads across compute to ensure efficient resource utilization.

To simplify deployment and management and increase resource utilization, use multi-model endpoints instead of separate endpoints for each model. One example for this approach is models with different data formats, such as recommendation systems that process text and images using separate endpoints. Or deploying a variety of models that include PyTorch, Scikit-learn, and TensorFlow models. Automatic scaling can amplify resource optimization for your hosted models. Auto scaling dynamically adjusts the number of instances provisioned for a model in response to changes in your workload. This helps you avoid cost and consumes less energy and resources. If your workload has intermittent or unpredictable traffic with idle periods between traffic peaks and can tolerate cold starts, use Amazon SageMaker Serverless Inference endpoints, which automatically launch compute resources and scale depending on traffic. Optionally, you can use Provisioned Concurrency with Serverless Inference when you have predictable bursts in your traffic.

AWS offers a few different options to better utilize your resources and lower emissions when working with inference workloads. AWS Inferentia is designed to deliver high performance at the lowest cost in EC2 instances for your deep learning and generative AI inference applications. AWS Inferentia is built for sustainability and provides up to 50% better performance per watt over comparable EC2 instances. You can further optimize resource utilization by combining AWS Inferentia and Amazon Elastic Inference to attach the right amount of GPU-powered inference acceleration to any EC2 or SageMaker instance type.

After training a model for high accuracy, developers often turn to more expensive large instances with lots of memory and processing power to achieve better throughput. You can reduce resource usage and avoid the need for more powerful instances by using pre-trained models and compiling them into optimized executables that can be hosted in SageMaker or edge devices for inference with Amazon SageMaker Neo.

Monitoring CPU, memory, and GPU resource utilization is critical to optimize model performance and avoid wasted resources. AWS offers a variety of tools that you can use to optimize MLOps for sustainability, such as CloudWatch, SageMaker Inference recommender, and SageMaker Model Monitor. Inference Recommender helps you choose the optimal instance type and configuration for ML models and workloads. You can use SageMaker Model Monitor to automate drift detection of your ML model in production, and only retrain it when prediction performance drops below predetermined key performance indicators (KPIs). This approach improves operational efficiency and retrains the model based on your business metrics.

Conclusion

Sustainability and ML are redefining how many companies deliver value for their customers. Incorporating sustainability into the design, development and deployment of ML models is a crucial long-term consideration. AWS is investing in the sustainability of the cloud and providing resources to assist customers in transforming their workloads to be more energy efficient. In this post, we have reviewed the Guidance for Optimizing MLOps for Sustainability on AWS, providing service-specific practices to understand and reduce the environmental impact of these workloads. MLOps consists of several distinct phases that can be independently optimized for sustainability. Regular reviews using tools such as AWS Well-Architected Machine Learning Lens help you identify optimization opportunities and provide a mechanism for you to meet your sustainability goals.

About the Authors

Archana Srinivasan is a Senior Technical Account Manager within Enterprise Support at Amazon Web Services (AWS). Archana provides strategic technical guidance for independent software vendors (ISVs) to innovate and operate their workloads efficiently on AWS.

Chris Procunier is a Senior Technical Account Manager at AWS, based out of Washington DC. He has been managing systems and infrastructure for 25 years as an entrepreneur, IT Director and architect. Outside of work Chris is passionate about family, friends, music, cooking and cycling.

Meghana Reddy is a Technical Account Manager at AWS, where she offers strategic technical guidance to Independent Software Vendors (ISVs) for optimizing their workloads on AWS. She is passionate about environmental sustainability and actively promotes sustainable practices within the cloud.

Steven David is a Principal Solutions Architect at Amazon Web Services (AWS). He has over 20 years of experience designing solutions for large enterprises. Through these engagements he has developed deep expertise in application development technologies and methodologies.

Read More

In June, I started a series of posts that highlight the key factors that are driving customers to choose Amazon Bedrock. The first covered building generative AI apps securely with Amazon Bedrock, while the second explored building custom generative AI applications with Amazon Bedrock. Now I’d like to take a closer look at Amazon Bedrock Agents, which empowers our customers to build intelligent and context-aware generative AI applications, streamlining complex workflows and delivering natural, conversational user experiences. The advent of large language models (LLMs) has enabled humans to interact with computers using natural language. However, many real-world scenarios demand more than just language comprehension. They involve executing complex multi-step workflows, integrating external data sources, or seamlessly orchestrating diverse AI capabilities and data workflows. In these real-world scenarios, agents can be a game changer, delivering more customized generative AI applications—and transforming the way we interact with and use LLMs.

Answering more complex queries

Amazon Bedrock Agents enables a developer to take a holistic approach in improving scalability, latency, and performance when building generative AI applications. Generative AI solutions that use Amazon Bedrock Agents can handle complex tasks by combining an LLM with other tools. For example, imagine that you are trying to create a generative AI-enabled assistant that helps people plan their vacations. You want it to be able to handle simple questions like “What’s the weather like in Paris next week?” or “How much does it cost to fly to Tokyo in July?” A basic virtual assistant might be able to answer those questions drawing from preprogrammed responses or by searching the Internet. But what if someone asks a more complicated question, like “I want to plan a trip to three countries next summer. Can you suggest a travel itinerary that includes visiting historic landmarks, trying local cuisine, and staying within a budget of $3,000?” That is a harder question because it involves planning, budgeting, and finding information about different destinations.

Using Amazon Bedrock Agents, a developer can quickly build a generative assistant to help answer this more complicated question by combining the LLM’s reasoning with additional tools and resources, such as natively integrated knowledge bases to propose personalized itineraries. It could search for flights, hotels, and tourist attractions by querying travel APIs, and use private data, public information for destinations, and weather—while keeping track of the budget and the traveler’s preferences. To build this agent, you would need an LLM to understand and respond to questions. But you would also need other modules for planning, budgeting, and accessing travel information.

Agents in action

Our customers are using Amazon Bedrock Agents to build agents—and agent-driven applications—quickly and effectively. Consider Rocket, the fintech company that helps people achieve home ownership and financial freedom:

“Rocket is poised to revolutionize the homeownership journey with AI technology, and agentic AI frameworks are key to our mission. By collaborating with AWS and leveraging Amazon Bedrock Agents, we are enhancing the speed, accuracy, and personalization of our technology-driven communication with clients. This integration, powered by Rocket’s 10 petabytes of data and industry expertise, ensures our clients can navigate complex financial moments with confidence.”

– Shawn Malhotra, CTO of Rocket Companies.

A closer look at how agents work

Unlike LLMs that provide simple lookup or content-generation capabilities, agents integrate various components with an LLM to create an intelligent orchestrator capable of handling sophisticated use cases with nuanced context and specific domain expertise. The following figure outlines the key components of Amazon Bedrock Agents:

The process starts with two parts—the LLM and the orchestration prompt. The LLM—often implemented using models like those in the Anthropic Claude family or Meta Llama models—provides the basic reasoning capabilities. The orchestration prompt is a set of prompts or instructions that guide the LLM when driving the decision-making process.

In the following sections, we discuss the key components of Amazon Bedrock Agents in depth:

Planning: A path from task to goals

The planning component for LLMs entails comprehending tasks and devising multi-step strategies to address a problem and fulfill the user’s need. In Amazon Bedrock Agents, we use chain-of-thought prompting in combination with ReAct in the orchestration prompt to improve an agent’s ability to solve multi-step tasks. In task decomposition, the agent must understand the intricacies of an abstract request. Continuing to explore our travel scenario, if a user wants to book a trip, the agent must recognize that it encompasses transportation, accommodation, reservations for sightseeing attractions, and restaurants. This ability to split up an abstract request, such as planning a trip, into detailed, executable actions, is the essence of planning. However, planning extends beyond the initial formulation of a plan, because during execution, the plan may get dynamically updated. For example, when the agent has completed arranging transportation and progresses to booking accommodation, it may encounter circumstances where no suitable lodging options align with the original arrival date. In such scenarios, the agent must determine whether to broaden the hotel search or revisit alternative booking dates, adapting the plan as it evolves.

Memory: Home for critical information

Agents have both long-term and short-term memory. Short-term memory is detailed and exact. It is relevant to the current conversation and resets when the conversation is over. Long-term memory is episodic and remembers important facts and details in the form of saved summaries. These summaries serve as the memory synopses of previous dialogues. The agent uses this information from the memory store to better solve the current task. The memory store is separate from the LLM, with a dedicated storage and a retrieval component. Developers have the option to customize and control which information is stored (or excluded) in memory. An identity management feature, which associates memory with specific end-users, gives developers the freedom to identify and manage end-users—and enable further development on top of Amazon Bedrock agents’ memory capabilities. The industry-leading memory retention functionality of Amazon Bedrock—launched at the recent AWS New York Summit—allows agents to learn and adapt to each user’s preferences over time, enabling more personalized and efficient experiences across multiple sessions for the same user. It is straightforward to use, allowing users to get started in a single click.

Communication: Using multiple agents for greater efficiency and effectiveness

Drawing from the powerful combination of the capabilities we’ve described, Amazon Bedrock Agents makes it effortless to build agents that transform one-shot query responders into sophisticated orchestrators capable of tackling complex, multifaceted use cases with remarkable efficiency and adaptability. But what about using multiple agents? LLM-based AI agents can collaborate with one another to improve efficiency in solving complex questions. Today, Amazon Bedrock makes it straightforward for developers to connect them through LangGraph, part of LangChain, the popular open source tool set. The integration of LangGraph into Amazon Bedrock empowers users to take advantage of the strengths of multiple agents seamlessly, fostering a collaborative environment that enhances the overall efficiency and effectiveness of LLM-based systems.

Tool Integration: New tools mean new capabilities

New generations of models such as Anthropic Claude Sonnet 3.5, Meta Llama 3.1, or Amazon Titan Text Premier are better equipped to use reources. Using these resources requires that developers keep up with ongoing updates and changes, requiring new prompts every time. To reduce this burden, Amazon Bedrock simplifies interfacing with different models, making it effortless to take advantage of all the features a model has to offer. For example, the new code interpretation capability announced at the recent AWS New York Summit allows Amazon Bedrock agents to dynamically generate and run code snippets within a secure, sandboxed environment to address complex tasks like data analysis, visualization, text processing, and equation solving. It also enables agents to process input files in various formats—including CSV, Excel, JSON—and generate charts from data.

Guardrails: Building securely

Accuracy is critical when dealing with complex queries. Developers can enable Amazon Bedrock Guardrails to help reduce inaccuracies. Guardrails improve the behavior of the applications you’re building, increase accuracy, and help you build responsibly. They can prevent both malicious intent from users and potentially toxic content generated by AI, providing a higher level of safety and privacy protection.

Amplifying and extending the capabilities of generative AI with Amazon Bedrock Agents

Enterprises, startups, ISVs, and systems integrators can take advantage of Amazon Bedrock Agents today because it provides development teams with a comprehensive solution for building and deploying AI applications that can handle complex queries, use private data sources, and adhere to responsible AI practices. Developers can start with tested examples—so-called golden utterances (input prompts) and golden responses (expected outputs). You can continuously evolve agents to fit your top use cases and kickstart your generative AI application development. Agents unlock significant new opportunities to build generative AI applications to truly transform your business. It will be fascinating to see the solutions—and results—that Amazon Bedrock Agents inspires.

Resources

For more information on customization with Amazon Bedrock, see the following resources:

• Learn about Amazon Bedrock
• Learn more about Amazon Bedrock Agents
• Learn how to associate a guardrail with your agent
• Read the announcement post on code interpretation and memory retention in Amazon Bedrock Agents
• Get hands-on insights on using Amazon Bedrock Agents

About the author

Vasi Philomin is VP of Generative AI at AWS. He leads generative AI efforts, including Amazon Bedrock and Amazon Titan.

Read More

This post is co-written with Francisco Azuaje from Genomics England.

Genomics England analyzes sequenced genomes for The National Health Service (NHS) in the United Kingdom, and then equips researchers to use data to advance biological research. As part of its goal to help people live longer, healthier lives, Genomics England is interested in facilitating more accurate identification of cancer subtypes and severity, using machine learning (ML). To explore whether such ML models can perform at higher accuracy when using multiple modalities, such as genomic and imaging data, Genomics England has launched a multi-modal program aimed at enhancing its dataset and also partnered with the the AWS Global Health and Non-profit Go-to-Market (GHN-GTM) Data Science and AWS Professional Services teams to create an automatic cancer sub-typing and survival detection pipeline and explore its accuracy on publicly available data.

In this post, we detail our collaboration in creating two proof of concept (PoC) exercises around multi-modal machine learning for survival analysis and cancer sub-typing, using genomic (gene expression, mutation and copy number variant data) and imaging (histopathology slides) data. We provide insights on interpretability, robustness, and best practices of architecting complex ML workflows on AWS with Amazon SageMaker. These multi-modal pipelines are being used on the Genomics England cancer cohort to enhance our understanding of cancer biomarkers and biology.

1. Data

The PoCs have used the publicly available cancer research data from The Cancer Genome Atlas (TCGA), which contain paired high-throughput genome analysis and diagnostic whole slide images with ground-truth survival outcome and histologic grade labels. Specifically, the PoCs focus on whole slide histopathology images of tissue samples as well as gene expression, copy number variations, and the presence of deleterious genetic variants to perform analysis on two cancer types: Breast cancer (BRCA) and gastrointestinal cancer types (Pan-GI). Table 1 shows the sample sizes for each cancer type.

Table 1. Overview of input data sizes across the different cancer types investigated.

2. Multi-modal machine learning frameworks

The ML pipelines tackling multi-modal subtyping and survival prediction have been built in three phases throughout the PoC exercises. First, a state-of-the-art framework, namely Pathology-Omic Research Platform for Integrative Survival Estimation (PORPOISE) (Chen et al., 2022) was implemented (Section 2.1). This was followed by the proposal, development, and implementation of a novel architecture based on Hierarchical Extremum Encoding (HEEC) (Section 2.2) by AWS, which aimed to mitigate the limitations of PORPOISE. The final phase improved on the results of HEEC and PORPOISE—both of which have been trained in a supervised fashion—using a foundation model trained in a self-supervised manner, namely Hierarchical Image Pyramid Transformer (HIPT) (Chen et al., 2023).

2.1 Pathology-Omic Research Platform for Integrative Survival Estimation (PORPOISE)

PORPOISE (Chen et al., 2022) is a multi-modal ML framework that consists of three sub-network components (see Figure 1 at Chen et al., 2022):

1. CLAM component; an attention-based multiple-instance learning network trained on pre-processed whole slid image (WSI) inputs (CLAM, Lu et al., 2021). CLAM extracts features from image patches of size 256×256 using a pre-trained ResNet50.
2. A self-normalizing network component for extracting deep molecular features.
3. A multi-modal fusion layer for integrating feature representations from 1) and 2) by modelling their pairwise interactions. The joint representations obtained from 3) are then used for undertaking the downstream tasks such as survival analysis and cancer-subtyping.

Despite being performant, PORPOISE was observed to output reduced multi-modal performance than single best modality (imaging) performance alone when gene expression data was excluded from the genomic features while performing survival analysis for Pan-GI data (Figure 2). A possible explanation is that the model has difficulty dealing with the extremely high dimensional, sparse genomic data without overfitting.

2.2. Hierarchical Extremum Encoding (HEEC): A novel supervised multi-modal ML framework

To mitigate the limitations of PORPOISE, AWS has developed a novel model structure, HEEC, which is based on three ideas:

1. Using tree ensembles (LightGBM) to mitigate the sparsity and overfitting issue observed when training PORPOISE (as observed by Grinsztajn et al., 2022, tree-based models tend to overfit less when confronted with high-dimensional data with many largely uninformative features).
2. Representation construction using a novel encoding scheme (extremum encoding) that preserves spatial relationships and thus interpretability.
3. Hierarchical learning to allow representations at multiple spatial scales.

Figure 1. Hierarchical Extremum Encoding (HEEC) of pathomic representations.

Figure 1 summarizes the HEEC architecture: starting from the bottom (and clockwise): Every input WSI is cut up into patches of size 4096×4096 and 256×256 pixels in a hierarchical manner and all stacks of patches are fed through ResNet50 to obtain embedding vectors. Additionally, nucleus-level representations (of size 64×64 pixels) are extracted by a graph neural network (GNNs), allowing local nucleus neighborhoods and their spatial relationships to be taken into account. This is followed by filtering for redundancy: Patch embeddings that are important are selected using positive-unlabeled learning, and GNN importance filtering is used for retaining the top nuclei features. The resulting hierarchical embeddings are coded using extremum encoding: the maxima and minima across the embeddings are taken in each vector entry, resulting in a single vector of maxima and minima per WSI. This encoding scheme allows keeping exact track of spatial relationships for each entry in the resulting representation vectors because the model can backtrack each vector entry to a specific patch, and thus to a specific coordinate in the image.

On the genomics side, importance filtering is applied based on excluding features that don’t correlate with the prediction target. The remaining features are horizontally appended to the pathology features, and a gradient boosted decision tree classifier (LightGBM) is applied to achieve predictive analysis.

HEEC architecture is interpretable out of the box, because HEEC embeddings possess implicit spatial information and the LightGBM model supports feature importance, allowing the filtering of the most important features for accurate prediction and backtracking to their location of origin. This location can be visually highlighted on the histology slide to be presented to expert pathologists for verification. Table 2 and Figure 2 show performance results of PORPOISE and HEEC, which show that HEEC is the only algorithm that outperforms the results of the best-performing single modality by combining multiple modalities.

Table 2. Classification and survival prediction performance of the two implemented multi-modal ML models on TCGA data. *Although Chen et al., 2022 provide some interpretability, the proposed attention visualization heatmaps have been deemed difficult to interpret from the pathologist point of view by Genomics England domain experts.

Figure 2. Comparison of performance (AUC) across individual modalities for survival analysis, when excluding the gene expression data. This matches the setting encountered by GEL in practice (GEL’s internal dataset has no gene expression data)

2.3. Improvements using foundation models

Despite yielding promising results, PORPOISE and HEEC algorithms use backbone architectures trained using supervised learning (for example, ImageNet pre-trained ResNet50). To further improve performance, a self-supervised learning-based approach, namely Hierarchical Image Pyramid Transformer (HIPT) (Chen et al., 2023), has been investigated in the final stage of the PoC exercises. Note that HIPT is currently limited to the hierarchical self-supervised learning of the imaging modality (WSIs) and further work includes expansion of self-supervised learning for the genomic modality.

HIPT starts by defining a hierarchy of patches composed of non-overlapping regions of size 16×16, 256×256, and 4096×4096 pixels (see Figure 2 at Chen et al., 2023). The lowest-layer features are extracted from the smallest patches (16×16) using a self-supervised learning algorithm based on DINO with a Vision Transformer (ViT) backbone. For each 256×256 region, the lowest-layer features are then aggregated using a global pooling layer. The aggregated features constitute the (new input) features for the middle-level in the hierarchy, where the process of self-supervised learning followed by global pooling is repeated and the aggregated output features form the input features belonging to the 4096×4096 region. These input features go through self-supervised learning one last time, and the final embeddings are obtained using global attention pooling. After pre-training is completed, fine-tuning is employed only on the final layer of the hierarchy (acting on 4096×4096 regions) using multiple instance learning.

Genomics England investigated whether using HIPT embeddings would be better than using the ImageNet pretrained ResNet50 encoder, and initial experiments have shown a gain in Harrels C-index of approximately 0.05 per cancer type in survival analysis. The embeddings offer other benefits as well. Such as being smaller—meaning that models train faster and the features have a smaller footprint.

3. Architecture on AWS

As part of the PoCs, we built a reference architecture (illustrated in Figure 3) for multi-modal ML using SageMaker, a platform for building training, and deploying ML models, with fully managed infrastructure, tools, and workflows. We aimed to demonstrate some general, reusable patterns that are independent of the specific algorithms:

• Decouple data pre-processing and feature computation from model training: In our use case, we process the pathology images into numerical feature representations once, we then store the resulting feature vectors in Amazon Simple Storage Service (Amazon S3) and reuse them to train different models. Analogously, we have a second processing branch that processes and extracts features from the genomic data.
• Decouple model training from inference: As we experiment with different model structures and hyperparameters, we keep track of model versions, hyperparameters, and metrics in SageMaker model registry. We refer to the registry to review our experiments and choose which models to deploy for inference.
• Wrap long-running computations inside containers and delegate their execution to SageMaker: Any long-running computation benefits from this pattern, whether it’s for data processing, model training, or batch inference. In this way, there’s no need to manage the underlying compute resources for running the containers. Cost is reduced through a pay-as-you-go model (resources are destroyed after a container has finished running) and the architecture is easily scalable to run multiple jobs in parallel.
• Orchestrate multiple containerized jobs into SageMaker pipelines: We build a pipeline once and run it multiple times with different parametrization. Hence, pipeline invocations can be referred to at a higher-level of abstraction, without having to constantly monitor the status of its long-running constituent jobs.
• Delegate hyperparameter tuning to SageMaker using a hyperparameter tuning job: A tuning job is a family of related training jobs (all managed by SageMaker) that efficiently explore the hyperparameter space. These training jobs take the same input data for training and validation, but each one is run with different hyperparameters for the learning algorithm. Which hyperparameter values to explore at each iteration are automatically chosen by SageMaker.

3.1 Separation between development and production environments

In general, we advise to do all development work outside of a production environment, because this minimizes the risk of leakage and corruption of sensitive production data and the production environment isn’t contaminated with intermediate data and software artifacts that obfuscate lineage tracking. If data scientists require access to production data during developmental stages, for tasks such as exploratory analysis and modelling work, there are numerous strategies that can be employed to minimize risk. One effective strategy is to employ data masking or synthetic data generation techniques in the testing environment to simulate real-world scenarios without compromising sensitive data. Furthermore, production level data can be securely moved into an independent environment for analysis. Access controls and permissions can be implemented to restrict the flow of data between environments, maintaining separation and ensuring minimal access rights.

Genomics England has created two separate ML environments for testing and production level interaction with data. Each environment sits in its own isolated AWS account. The test environment mimics the production environment in its data storage strategy, but contains synthetic data void of personally identifiable information (PII) or protected health information (PHI), instead of production-level data. This test environment is used for developing essential infrastructure components and refining best practices in a controlled setting, which can be tested with synthetic data before deploying to production. Strict access controls, including role-based permissions employing principles of least privilege, are implemented in all environments to ensure that only authorized personnel can interact with sensitive data or modify deployed resources.

3.2 Automation with CI/CD pipelines

On a related note, we advise ML developers to use infrastructure-as-code to describe the resources that are deployed in their AWS accounts and use continuous integration and delivery (CI/CD) pipelines to automate code quality checks, unit testing, and the creation of artifacts, such as container images. Then, also configure the CI/CD pipelines to automatically deploy the created artifacts into the target AWS accounts, whether they’re for development or for production. These well-established automation techniques minimize errors related to manual deployments and maximize the reproducibility between development and production environments.

Genomics England has investigated the use of CI/CD pipelines for automated deployment of platform resources, as well as automated testing of code.

Figure 3. Overview of the AWS reference architecture employed for multi-modal ML in the cloud

4. Conclusion

Genomics England has a long history of working with genomics data, however the inclusion of imaging data adds additional complexity and potential. The two PoCs outlined in this post have been essential in launching Genomics England’s efforts in creating a multi-modal environment that facilitates ML development for the purpose of tackling cancer. The implementation of state-of-the-art models in Genomics England’s multi-modal environment and assistance in developing robust practices will ensure that users are maximally enabled in their research.

“At Genomics England, our mission is to realize the enormous potential of genomic and multi-modal information to further precision medicine and push the boundaries to realize the enormous potential of AWS cloud computing in its success”.

– Dr Prabhu Arumugam, Director of Clinical data and imaging, Genomics England

Acknowledgements

The results published in this blog post are in whole or part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga.

About the Authors

Cemre Zor, PhD, is a senior healthcare data scientist at Amazon Web Services. Cemre holds a PhD in theoretical machine learning and postdoctoral experiences on machine learning for computer vision and healthcare. She works with healthcare and life sciences customers globally to support them with machine learning modelling and advanced analytics approaches while tackling real-world healthcare problems.

Tamas Madl, PhD, is a former senior healthcare data scientist and business development lead at Amazon Web Services, with academic as well as industry experience at the intersection between healthcare and machine learning. Tamas helped customers in the Healthcare and Life Science vertical to innovate through the adoption of Machine Learning. He received his PhD in Computer Science from the University of Manchester.

Epameinondas Fritzilas, PhD, is a senior consultant at Amazon Web Services. He works hands-on with customers to design and build solutions for data analytics and AI applications in healthcare. He holds a PhD in bioinformatics and fifteen years of industry experience in the biotech and healthcare sectors.

Lou Warnett is a healthcare data scientist at Amazon Web Services. He assists healthcare and life sciences customers from across the world in tackling some of their most pressing challenges using data science, machine learning and AI, with a particular emphasis more recently on generative AI. Prior to joining AWS, Lou received a master’s in Mathematics and Computing at Imperial College London.

Sam Price is a Professional Services consultant specializing in AI/ML and data analytics at Amazon Web Services. He works closely with public sector customers in healthcare and life sciences to solve challenging problems. When not doing this, Sam enjoys playing guitar and tennis, and seeing his favorite indie bands.

Shreya Ruparelia is a data & AI consultant at Amazon Web Services, specialising in data science and machine learning, with a focus on developing GenAI applications. She collaborates with public sector healthcare organisations to create innovative AI-driven solutions. In her free time, Shreya enjoys activities such as playing tennis, swimming, exploring new countries and taking walks with the family dog, Buddy.

Pablo Nicolas Nunez Polcher, MSc, is a senior solutions architect working for the Public Sector team with Amazon Web Services. Pablo focuses on helping healthcare public sector customers build new, innovative products on AWS in accordance with best practices. He received his M.Sc. in Biological Sciences from Universidad de Buenos Aires. In his spare time, he enjoys cycling and tinkering with ML-enabled embedded devices.

Matthew Howard is the head of Healthcare Data Science and part of the Global Health and Non-Profits team in Amazon Web Services. He focuses on how data, machine learning and artificial intelligence can transform health systems and improve patient outcomes. He leads a team of applied data scientists who work with customers to develop AI-based healthcare solutions. Matthew holds a PhD in Biological Sciences from Imperial College London.

Tom Dyer is a Senior Product Manager at Genomics England. And was previously an Applied Machine Learning Engineer working within the Multimodal squad. His work focussed on building multimodal learning frameworks that allow users to rapidly scale research in the cloud. He also works on developing ML tooling to organise pathology image datasets and explain model predictions on a cohort level

Samuel Barnett is an applied machine learning engineer with Genomics England working on improving healthcare with machine learning. He is embedded with the Multimodal squad and is part of an ongoing effort to show the value of combing genomic, imaging, and text based data in machine learning models.

Prabhu Arumugam is the former Director of Clinical Data Imaging at Genomics England. Having joined the organization in 2019, Prabhu trained in medicine St. Bartholomew’s and the Royal London. He trained in Histopathology and completed his PhD at The Barts Cancer Institute on pancreatic pathology.

Francisco Azuaje, PhD, is the director of bioinformatics at Genomics England, where he provides cross-cutting leadership in strategy and research with a focus on data science and AI. With a career covering academia, the pharmaceutical industry, and the public sector, he has wide experience leading multidisciplinary teams in solving challenges involving diverse data sources and computational modelling approaches. With his expertise in bioinformatics and applied AI, Dr. Azuaje enables the translation of complex data into insights that can improve patient outcomes.

Read More