Monthly Archives: February 2024

This post is co-written with Ilan Geller, Shuyu Yang and Richa Gupta from Accenture.

Bringing innovative new pharmaceuticals drugs to market is a long and stringent process. Companies face complex regulations and extensive approval requirements from governing bodies like the US Food and Drug Administration (FDA). A key part of the submission process is authoring regulatory documents like the Common Technical Document (CTD), a comprehensive standard formatted document for submitting applications, amendments, supplements, and reports to the FDA. This document contains over 100 highly detailed technical reports created during the process of drug research and testing. Manually creating CTDs is incredibly labor-intensive, requiring up to 100,000 hours per year for a typical large pharma company. The tedious process of compiling hundreds of documents is also prone to errors.

Accenture built a regulatory document authoring solution using automated generative AI that enables researchers and testers to produce CTDs efficiently. By extracting key data from testing reports, the system uses Amazon SageMaker JumpStart and other AWS AI services to generate CTDs in the proper format. This revolutionary approach compresses the time and effort spent on CTD authoring. Users can quickly review and adjust the computer-generated reports before submission.

Because of the sensitive nature of the data and effort involved, pharmaceutical companies need a higher level of control, security, and auditability. This solution relies on the AWS Well-Architected principles and guidelines to enable the control, security, and auditability requirements. The user-friendly system also employs encryption for security.

By harnessing AWS generative AI, Accenture aims to transform efficiency for regulated industries like pharmaceuticals. Automating the frustrating CTD document process accelerates new product approvals so innovative treatments can get to patients faster. AI delivers a major leap forward.

This post provides an overview of an end-to-end generative AI solution developed by Accenture for regulatory document authoring using SageMaker JumpStart and other AWS services.

Solution overview

Accenture built an AI-based solution that automatically generates a CTD document in the required format, along with the flexibility for users to review and edit the generated content​. The preliminary value is estimated at a 40–45% reduction in authoring time.

This generative AI-based solution extracts information from the technical reports produced as part of the testing process and delivers the detailed dossier in a common format required by the central governing bodies. Users then review and edit the documents, where necessary, and submit the same to the central governing bodies. This solution uses the SageMaker JumpStart AI21 Jurassic Jumbo Instruct and AI21 Summarize models to extract and create the documents.

The following diagram illustrates the solution architecture.

The workflow consists of the following steps:

1. A user accesses the regulatory document authoring tool from their computer browser.
2. A React application is hosted on AWS Amplify and is accessed from the user’s computer (for DNS, use Amazon Route 53).
3. The React application uses the Amplify authentication library to detect whether the user is authenticated.
4. Amazon Cognito provides a local user pool or can be federated with the user’s active directory.
5. The application uses the Amplify libraries for Amazon Simple Storage Service (Amazon S3) and uploads documents provided by users to Amazon S3.
6. The application writes the job details (app-generated job ID and Amazon S3 source file location) to an Amazon Simple Queue Service (Amazon SQS) queue. It captures the message ID returned by Amazon SQS. Amazon SQS enables a fault-tolerant decoupled architecture. Even if there are some backend errors while processing a job, having a job record inside Amazon SQS will ensure successful retries.
7. Using the job ID and message ID returned by the previous request, the client connects to the WebSocket API and sends the job ID and message ID to the WebSocket connection.
8. The WebSocket triggers an AWS Lambda function, which creates a record in Amazon DynamoDB. The record is a key-value mapping of the job ID (WebSocket) with the connection ID and message ID.
9. Another Lambda function gets triggered with a new message in the SQS queue. The Lambda function reads the job ID and invokes an AWS Step Functions workflow for processing data files.
10. The Step Functions state machine invokes a Lambda function to process the source documents. The function code invokes Amazon Textract to analyze the documents. The response data is stored in DynamoDB. Based on specific requirements with processing data, it can also be stored in Amazon S3 or Amazon DocumentDB (with MongoDB compatibility).
11. A Lambda function invokes the Amazon Textract API DetectDocument to parse tabular data from source documents and stores extracted data into DynamoDB.
12. A Lambda function processes the data based on mapping rules stored in a DynamoDB table.
13. A Lambda function invokes the prompt libraries and a series of actions using generative AI with a large language model hosted through Amazon SageMaker for data summarization.
14. The document writer Lambda function writes a consolidated document in an S3 processed folder.
15. The job callback Lambda function retrieves the callback connection details from the DynamoDB table, passing the job ID. Then the Lambda function makes a callback to the WebSocket endpoint and provides the processed document link from Amazon S3.
16. A Lambda function deletes the message from the SQS queue so that it’s not reprocessed.
17. A document generator web module converts the JSON data into a Microsoft Word document, saves it, and renders the processed document on the web browser.
18. The user can view, edit, and save the documents back to the S3 bucket from the web module. This helps in reviews and corrections needed, if any.

The solution also uses SageMaker notebooks (labeled T in the preceding architecture) to perform domain adaption, fine-tune the models, and deploy the SageMaker endpoints.

Conclusion

In this post, we showcased how Accenture is using AWS generative AI services to implement an end-to-end approach towards a regulatory document authoring solution. This solution in early testing has demonstrated a 60–65% reduction in the time required for authoring CTDs. We identified the gaps in traditional regulatory governing platforms and augmented generative intelligence within its framework for faster response times, and are continuously improving the system while engaging with users across the globe. Reach out to the Accenture Center of Excellence team to dive deeper into the solution and deploy it for your clients.

This joint program focused on generative AI will help increase the time-to-value for joint customers of Accenture and AWS. The effort builds on the 15-year strategic relationship between the companies and uses the same proven mechanisms and accelerators built by the Accenture AWS Business Group (AABG).

Connect with the AABG team at accentureaws@amazon.com to drive business outcomes by transforming to an intelligent data enterprise on AWS.

For further information about generative AI on AWS using Amazon Bedrock or SageMaker, refer to Generative AI on AWS: Technology and Get started with generative AI on AWS using Amazon SageMaker JumpStart.

You can also sign up for the AWS generative AI newsletter, which includes educational resources, blogs, and service updates.

About the Authors

Ilan Geller is a Managing Director in the Data and AI practice at Accenture.  He is the Global AWS Partner Lead for Data and AI and the Center for Advanced AI.  His roles at Accenture have primarily been focused on the design, development, and delivery of complex data, AI/ML, and most recently Generative AI solutions.

Shuyu Yang is Generative AI and Large Language Model Delivery Lead and also leads CoE (Center of Excellence) Accenture AI (AWS DevOps professional) teams.

Richa Gupta is a Technology Architect at Accenture, leading various AI projects. She comes with 18+ years of experience in architecting Scalable AI and GenAI solutions. Her expertise area is on AI architecture, Cloud Solutions and Generative AI. She plays and instrumental role in various presales activities.

Shikhar Kwatra is an AI/ML Specialist Solutions Architect at Amazon Web Services, working with a leading Global System Integrator. He has earned the title of one of the Youngest Indian Master Inventors with over 500 patents in the AI/ML and IoT domains. Shikhar aids in architecting, building, and maintaining cost-efficient, scalable cloud environments for the organization, and supports the GSI partner in building strategic industry solutions on AWS. Shikhar enjoys playing guitar, composing music, and practicing mindfulness in his spare time.

Sachin Thakkar is a Senior Solutions Architect at Amazon Web Services, working with a leading Global System Integrator (GSI). He brings over 23 years of experience as an IT Architect and as Technology Consultant for large institutions. His focus area is on Data, Analytics and Generative AI. Sachin provides architectural guidance and supports the GSI partner in building strategic industry solutions on AWS.

Read More

Do your employees wait for hours on the telephone to open an IT ticket? Do they wait for an agent to triage an issue, which sometimes only requires restarting the computer? Providing excellent IT support is crucial for any organization, but legacy systems have relied heavily on human agents being available to intake reports and triage issues. Conversational AI (or chatbots) can help triage some of these common IT problems and create a ticket for the tasks when human assistance is needed. Chatbots quickly resolve common business issues, improve employee experiences, and free up agents’ time to handle more complex problems.

QnABot on AWS is an open source solution built using AWS native services like Amazon Lex, Amazon OpenSearch Service, AWS Lambda, Amazon Transcribe, and Amazon Polly. QnABot version 5.4+ is also enhanced with generative AI capabilities.

According to Gartner Magic Quadrant 2023, ServiceNow is one of the leading IT Service Management (ITSM) providers on the market. ServiceNow’s Incident Management uses workflows to identify, track, and resolve high‑impact IT service incidents.

In this post, we demonstrate how to integrate the QnABot on AWS chatbot solution with ServiceNow. With this integration, users can chat with QnABot to triage their IT service issues and open an incident ticket in ServiceNow in real time by providing details to QnABot.

Watch the following video to see how users can ask questions to an IT service desk chatbot and get answers. For most frequently asked questions, chatbot answers can help resolve the issue. When a user determines that the answers provided are not useful, they can request the creation of a ticket in ServiceNow.

Solution overview

QnABot on AWS is a multi-channel, multi-language chatbot that responds to your customer’s questions, answers, and feedback. QnABot on AWS is a complete solution and can be deployed as part of your IT Service Desk ticketing workflow. Its distributed architecture allows for integrations with other systems like ServiceNow. If you wish to build your own chatbot using Amazon Lex or add only Amazon Lex as part of your application, refer to Integrate ServiceNow with Amazon Lex chatbot for ticket processing.

The following diagram illustrates the solution architecture.

The workflow includes the following steps:

1. A QnABot administrator can configure the questions using the Content Designer UI delivered by Amazon API Gateway and Amazon Simple Storage Service (Amazon S3).
2. The Content Designer Lambda function saves the input in OpenSearch Service in a question’s bank index.
3. When QnABot users ask questions prompting ServiceNow integration, Amazon Lex fetches the questions and requests the user to provide a description of the issue. When the description is provided, it invokes a Lambda function.
4. The Lambda function fetches secrets from AWS Secrets Manager, where environment variables are stored, and makes an HTTP call to create a ticket in ServiceNow. The ticket number is then returned to the user.

When building a diagnostic workflow, you may require inputs to different questions before you can create a ticket in ServiceNow. You can use response bots and the document chaining capabilities of QnABot to achieve this capability.

Response bots are bots created to elicit a response from users and store them as part of session variables or as part of slot values. You can use built-in response bots or create a custom response bot. Response chatbot names must start with the letters “QNA.”

This solution provides a set of built-in response bots. Refer to Configuring the chatbot to ask the questions and use response bots for implementation details.

You can use document chaining to elicit the response and invoke Lambda functions. The chaining rule is a JavaScript programming expression used to test the value of the session attribute set to elicit a response and either route to another bot or invoke Lambda functions. You can identify the next question in the document by identifying the question ID (QID) specified in the Document Chaining:Chaining Rule field as ‘QID::‘ followed by the QID value of the document. For example, a rule that evaluates to “QID::Admin001” will chain to item Admin.001.

When using a chaining rule for Lambda, the function name must start with the letters “QNA,” and is specified in the Document Chaining:Chaining Rule field as ‘Lambda::FunctionNameorARN’. All chaining rules must be enclosed in a single quote.

Deploy the QnABot solution

Complete the following steps to deploy the solution:

1. Choose Launch Solution on the QnABot implementation guide to deploy the latest QnABot template via AWS CloudFormation.
2. Provide a name for the bot.
3. Provide an email where you will receive an email to reset your password.
4. Make sure that EnableCognitoLogin is set to true.
5. For all other parameters, accept the defaults (see the implementation guide for parameter definitions), and launch the QnABot stack.

This post uses a static webpage hosted on Amazon CloudFront, and the QnABot chatbot is embedded in the page using the Amazon Lex web UI sample plugin. We also provide instructions for testing this solution using the QnABot client page.

Create a ServiceNow account

This section walks through the steps to create a ServiceNow account and ServiceNow developer instance:

1. First, sign up for a ServiceNow account.

2. Go to your email and confirm this email address for your ServiceNow ID.
3. As part of the verification, you’ll will be asked to provide the six-digit verification code sent to your email.
4. You can skip the page that asks you to set up two-factor authentication. You’re redirected to the landing page with the ServiceNow Developer program.
5. In the Getting Started steps, choose Yes, I need a developer oriented IDE.

6. Choose Start Building to set up an instance.

When the build is complete, which may take couple of seconds to minutes, you will be provided with the instance URL, user name, and password details. Save this information to use in later steps.

7. Log in to the site using the following URL (provide your instance): https://devXXXXXX.service-now.com/now/nav/ui/classic/params/target/change_request_list.do.

Be sure to stay logged in to the ServiceNow developer instance throughout the process.

If logged out, use your email and password to log back in and wake up the instance and prevent hibernation.

8. Choose All in the navigation bar, then choose Incidents.

9. Select All to remove all of the filters.

All incidents will be shown on this page.

Create users in ServiceNow and an Amazon Cognito pool

You can create an incident using the userid of the chatbot user. For that, we need to confirm that the userId of the chatbot user exists in ServiceNow. First, we create the ServiceNow user, then we create a user with the same ID in an Amazon Cognito user pool. Amazon Cognito is an AWS service to authenticate clients and provide temporary AWS credentials.

1. Create a ServiceNow user. Be sure to include a first name, last name, and email.

Note down the user ID of the newly created user. You will need this when creating an Amazon Cognito user in a user pool.

2. On the Amazon Cognito console, choose User pools in the navigation pane.

If you have deployed the Amazon Lex web UI plugin, you will see two user pool names; if you did not, you’ll see only one user pool name.

3. Select the user pool that has your QnABot name and create a new user. Use the same userId as that of the ServiceNow user.
4. If you are using the Amazon Lex web UI, create a user in the appropriate Amazon Cognito user pool by following the preceding steps.

Note that the userId you created will be used for the QnABot client and Amazon Lex Web UI client.

Create a Lambda function for invoking ServiceNow

In this step, you create a Lambda function that invokes the ServiceNow API to create a ticket.

1. On the Lambda console, choose Functions in the navigation pane.
2. Choose Create function.

3. Select Author from scratch.
4. For Function name, enter a name, such as qna-ChatBotLambda. (Remember that QnABot requires the prefix qna- in the name.)
5. For Runtime, choose Node.js 18.x.

This Lambda function creates new role. If you want to use an existing role, you can change the default AWS Identity and Access Management (IAM) execution role by selecting Use existing role.

6. Choose Create function.
7. After you create the function, use the inline editor to edit the code for index.js.
8. Right-click on index.js and rename it to index.mjs.
9. Enter the following code, which is sample code for the function that you’re using as the compute layer for our logic:

import AWS from '@aws-sdk/client-secrets-manager';

const incident="incident";
const secret_name = "servicenow/password";

export const handler = async (event, context) => {
    console.log('Received event:',JSON.stringify(event, null,2));
    // make async call createticket which creates serviceNow ticket
    await createTicket( event).then(response => event=response);
    return event;
    
};

// async function to create servicenow ticket
async function createTicket( event){
 
    var password='';
    await getSecretValue().then(response => password=response);
    
    // fetch description and userid from event
      var shortDesc =  event.req._event.inputTranscript;
    console.log("received slots value", shortDesc);
    // userName of the logged in user
    var userName= event.req._userInfo.UserName;
    console.log("userId", userName);
    
    console.log("password from secrets manager::", password);
    // description provided by user is added to short_description
    var requestData = {
        "short_description": shortDesc,
        "caller_id": userName
      };
      var postData = JSON.stringify(requestData);

    // create url from hostname fetched from envrionment variables. Remaining path is constant.
    const url = "https://"+process.env.SERVICENOW_HOST+":443/api/now/table/"+incident;

    // create incident in servicenow and return event with ticket information
    try {
            await fetch(url,{
                method: 'POST',
            headers: {
                'Content-Type': 'application/json',
                'Accept': 'application/json',
                'Authorization': 'Basic ' + Buffer.from(process.env.SERVICENOW_USERNAME + ":" + password).toString('base64'),
                'Content-Length': Buffer.byteLength(postData),
            },
            'body': postData
            }).then(response=>response.json())
            .then(data=>{ console.log(data); 
                var ticketNumber = data.result.number;
                var ticketType = data.result.sys_class_name;
                event.res.message="Done! I've opened an " + ticketType + " ticket for you in ServiceNow. Your ticket number is: " + ticketNumber + ".";
            });  
            return event;
        }
        catch (e) {
            console.error(e);
            return 500;
        }

}

// get secret value from secrets manager
async function getSecretValue(){
    var secret;
    var client = new AWS.SecretsManager({
        region: process.env.AWS_REGION
    });
   // await to get secret value
    try {
        secret = await client.getSecretValue({SecretId: secret_name});
    }
    catch (err) {
        console.log("error", err);
    
    }   
   const secretString = JSON.parse(secret.SecretString);
    return secretString.password;
}

This function uses the ServiceNow Incident API. For more information, refer to Create an incident.

10. Choose Deploy to deploy this code to the $LATEST version of the Lambda function.
11. On the Configuration tab, in the Environment variables section, add the following:
    1. • Add SERVICENOW_HOST with the value devXXXXXX.service-now.com.
       • Add SERVICENOW_USERNAME with the value admin.

    

12. Copy the Lambda function ARN. You will need it at later stage.

The next step is to store your ServiceNow user name and password in Secrets Manager.

13. On the Secrets Manager console, create a new secret.
14. Select Other type of secret.
15. Add your key-value pairs as shown and choose Next.

16. For Secret name, enter a descriptive name (for this post, servicenow/password). If you choose a different name, update the value of const secret_name in the Lambda function code.
17. Choose Next.
18. Leave Configure rotation on default and choose Next.
19. Review the secret information and choose Store.
20. Copy the ARN of the newly created secret.

Now let’s give Lambda permissions to Secrets Manager.

21. On the Lambda function page, go to the Configurations tab and navigate to the Permissions section.

22. Choose the execution role name to open the IAM page for the role.
23. In the following inline policy, provide the ARN of the secret you created earlier:

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "SecretsManagerRead",
			"Effect": "Allow",
			"Action": ["secretsmanager:GetResourcePolicy",
				"secretsmanager:GetSecretValue",
				"secretsmanager:DescribeSecret",
				"secretsmanager:ListSecrets",
				"secretsmanager:ListSecretVersionIds"
],
			"Resource": "<ARN>"
		}
	]
}

24. Add the inline policy to the role.

Configure QnABot configurations

In this section, we first create some knowledge questions using the Questions feature of QnABot. We then create a response bot that elicits a response from a user when they ask for help. This bot uses document chaining to call another bot, and triggers Lambda to create a ServiceNow ticket.

For more information about using QnABot with generative AI, refer to Deploy generative AI self-service question answering using the QnABot on AWS solution powered by Amazon Lex with Amazon Kendra, and Amazon Bedrock.

Create knowledge question 1

Create a knowledge question for installing software:

1. On the AWS CloudFormation console, navigate to the QnABot stack.
2. On the Outputs tab, and open the link for ContentDesignerURL.
3. Log in to the QnABot Content Designer using admin credentials.
4. Choose Add to add a new question.
5. Select qna.
6. For Item ID, enter software.001.
7. Under Questions/Utterances, enter the following:

   a.	How to install a software 
   b.	How to install developer tools 
   c.	can you give me instructions to install software 
   

8. Under Answer, enter the following answer:

Installing from Self Service does not require any kind of permissions or admin credentials. It will show you software that is available for you, without any additional requests.
1. Click the search icon in the menu at the top. Type Self Service and press Enter.
2. Sign in with your security key credentials.
3. Search for your desired software in the top right corner.
4. Click the Install button.

9. Expand the Advanced section and enter the same text in Markdown Answer.

10. Leave the rest as default, and choose Create to save the question.

Create knowledge question 2

Now you create the second knowledge question.

1. Choose Add to add a new question.
2. Select qna.
3. For Item ID, enter knowledge.001.
4. Under Questions/Utterances, enter Want to learn more about Amazon Lex.
5. Under Answer, enter the following answer:

### Amazon Lex
Here is a video of Amazon Lex Introduction <iframe width="580" height="327" src="https://www.youtube.com/embed/Q2yJf4bn5fQ" title="Conversational AI powered by Amazon Lex | Amazon Web Services" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
Do you want to learn more about it?<br>
Here are some resources<br>
1. [Introduction to Amazon Lex](https://explore.skillbuilder.aws/learn/course/external/view/elearning/249/introduction-to-amazon-lex)
2. [Building better bots using Amazon Connect](https://explore.skillbuilder.aws/learn/course/external/view/elearning/481/building-better-bots-using-amazon-connect)
3. [Amazon Lex V2 getting started- Streaming APIs](https://aws.amazon.com/blogs/machine-learning/delivering-natural-conversational-experiences-using-amazon-lex-streaming-apis/)

6. Expand the Advanced section and enter the same answer under Markdown Answer.

7. Leave the rest as default, and choose Create to save the question.

Create knowledge question 3

Complete the following steps to add another knowledge question:

1. Choose Add to add a new question.
2. Select qna.
3. For Item ID, enter password.reset.
4. Under Questions/Utterances, enter I need to reset my password.
5. Under Answer, enter the following answer:

#### Password Reset Instructions
Please follow below instructions to reset your password
1. Please go to AnyTech's IT web page. 
2. Use the Password Reset Tool on the left hand navigation. 
3. In the Password Reset Tool, provide your new password and save. 
4. Once you change your password, please log out of your laptop and login.
<br><br>
**Note**: If you are logged out of your computer, you can ask your manager to reset the password.

6. Expand the Advanced section and enter the same text for Markdown Answer.
7. Choose Create to save the question.

Create a response bot

Complete the following steps to create the first response bot, which elicits a response:

1. Choose Add to add a new question.
2. Select qna.
3. For Item ID, enter ElicitResponse.001.
4. Under Questions/Utterances, enter Please create a ticket.
5. Under Answer, enter the following answer:

Sure, I can help you with that!! Please give a short description of your problem.

6. Expand the Advanced section and navigate to the Elicit Response section.
7. For Elicit Response: ResponseBot Hook, enter QNAFreeText.
8. For Elicit Response: Response Session Attribute Namespace, enter short_description.

This creates a slot named short_description that captures the response or description for the incident. This slot uses the built-in QNAFreeText, which is used for capturing free text.

9. For Document Chaining: Chaining Rule, enter QID::item.002. This must be in single quotes. Remember this chaining rule to use when creating your document chain.
10. Leave the rest as default.

11. Choose Create to save the question.

Create a document chain

Now we create a document chain in QnABot that will trigger the Lambda function to create a ticket and respond with a ticket number. Document chaining allows you to chain two bots based on the rule you configured. Complete the following steps:

1. Choose Add to add a new question.
2. Select qna.
3. For Item ID, enter item.002. This should match the QID value given in the document chain rule earlier.
4. Under Questions/Utterances, enter servicenow integration.
5. Under Answer, enter the following answer:

There was an error, please contact system administrator

6. In the Advanced section, add the Lambda function ARN for Lambda Hook.

7. Choose Create to save the question.

Test the QnABot

To test the QnABot default client, complete the following steps:

1. Choose the options menu in the Content Designer and choose QnABot Client.

The QnABot client will open in a new browser tab.

2. Log in using the newly created user credentials to begin the test.

If you plan to use the Amazon Lex Web UI on a static page, follow these instructions.

3. Choose the chat icon at the bottom of the page to start the chat.
4. To log in, choose Login on the menu.

You will be routed to the login page.

5. Provide the userId created earlier.
6. For first-time logins, you will be prompted to reset your password.

7. Now we can test the chatbot with example use cases. For our first use case, we want to learn about Amazon and enter the question “I want to learn about Amazon Lex, can you give me some information about it?” QnABot provides a video and some links to resources.

8. In our next, example, we need to install software on our laptop, and ask “Can you give me instructions to install software.” QnABot understands that the user is requesting help installing software and provides answers from the knowledge bank. You can follow those instructions and install the software you need.

9. While installing the software, what if you locked your password due to multiple failed login attempts? To request a password reset, you can ask “I need to reset my password.”

10. You might need additional assistance resetting the password and want to create a ticket. In this case, enter “Please create a ticket.” QnABot asks for a description of the problem; you can enter “reset password.” QnAbot creates a ticket with the description provided and provides the ticket number as part of the response.

11. You can verify the incident ticket was created on the ServiceNow console under Incidents. If the ticket is not shown on the first page, search for the ticket number using the search toolbar.

Clean up

To avoid incurring future charges, delete the resources you created. For instructions to uninstall the QnABot solution plugin, refer to Uninstall the solution.

Conclusion

Integrating QnABot on AWS with ServiceNow provides an end-to-end solution for automated customer support. With QnABot’s conversational AI capabilities to understand customer questions and ServiceNow’s robust incident management features, companies can streamline ticket creation and resolution. You can also extend this solution to show a list of tickets created by the user. For more information about incorporating these techniques into your bots, see QnABot on AWS.

About the Authors

Sujatha Dantuluri is a Senior Solutions Architect in the US federal civilian team at AWS. She has over 20 years of experience supporting commercial and federal government. She works closely with customers in building and architecting mission-critical solutions. She has also contributed to IEEE standards.

Maia Haile is a Solutions Architect at Amazon Web Services based in the Washington, D.C. area. In that role, she helps public sector customers achieve their mission objectives with well-architected solutions on AWS. She has 5 years of experience spanning nonprofit healthcare, media and entertainment, and retail. Her passion is using AI and ML to help public sector customers achieve their business and technical goals.

Read More

In 2021, the pharmaceutical industry generated $550 billion in US revenue. Pharmaceutical companies sell a variety of different, often novel, drugs on the market, where sometimes unintended but serious adverse events can occur.

These events can be reported anywhere, from hospitals or at home, and must be responsibly and efficiently monitored. Traditional manual processing of adverse events is made challenging by the increasing amount of health data and costs. Overall, $384 billion is projected as the cost of pharmacovigilance activities to the overall healthcare industry by 2022. To support overarching pharmacovigilance activities, our pharmaceutical customers want to use the power of machine learning (ML) to automate the adverse event detection from various data sources, such as social media feeds, phone calls, emails, and handwritten notes, and trigger appropriate actions.

In this post, we show how to develop an ML-driven solution using Amazon SageMaker for detecting adverse events using the publicly available Adverse Drug Reaction Dataset on Hugging Face. In this solution, we fine-tune a variety of models on Hugging Face that were pre-trained on medical data and use the BioBERT model, which was pre-trained on the Pubmed dataset and performs the best out of those tried.

We implemented the solution using the AWS Cloud Development Kit (AWS CDK). However, we don’t cover the specifics of building the solution in this post. For more information on the implementation of this solution, refer to Build a system for catching adverse events in real-time using Amazon SageMaker and Amazon QuickSight.

This post delves into several key areas, providing a comprehensive exploration of the following topics:

• The data challenges encountered by AWS Professional Services
• The landscape and application of large language models (LLMs):
  • Transformers, BERT, and GPT
  • Hugging Face
• The fine-tuned LLM solution and its components:
  • Data preparation
  • Model training

Data challenge

Data skew is often a problem when coming up with classification tasks. You would ideally like to have a balanced dataset, and this use case is no exception.

We address this skew with generative AI models (Falcon-7B and Falcon-40B), which were prompted to generate event samples based on five examples from the training set to increase the semantic diversity and increase the sample size of labeled adverse events. It’s advantageous to us to use the Falcon models here because, unlike some LLMs on Hugging Face, Falcon gives you the training dataset they use, so you can be sure that none of your test set examples are contained within the Falcon training set and avoid data contamination.

The other data challenge for healthcare customers are HIPAA compliance requirements. Encryption at rest and in transit has to be incorporated into the solution to meet these requirements.

Transformers, BERT, and GPT

The transformer architecture is a neural network architecture that is used for natural language processing (NLP) tasks. It was first introduced in the paper “Attention Is All You Need” by Vaswani et al. (2017). The transformer architecture is based on the attention mechanism, which allows the model to learn long-range dependencies between words. Transformers, as laid out in the original paper, consist of two main components: the encoder and the decoder. The encoder takes the input sequence as input and produces a sequence of hidden states. The decoder then takes these hidden states as input and produces the output sequence. The attention mechanism is used in both the encoder and the decoder. The attention mechanism allows the model to attend to specific words in the input sequence when generating the output sequence. This allows the model to learn long-range dependencies between words, which is essential for many NLP tasks, such as machine translation and text summarization.

One of the more popular and useful of the transformer architectures, Bidirectional Encoder Representations from Transformers (BERT), is a language representation model that was introduced in 2018. BERT is trained on sequences where some of the words in a sentence are masked, and it has to fill in those words taking into account both the words before and after the masked words. BERT can be fine-tuned for a variety of NLP tasks, including question answering, natural language inference, and sentiment analysis.

The other popular transformer architecture that has taken the world by storm is Generative Pre-trained Transformer (GPT). The first GPT model was introduced in 2018 by OpenAI. It works by being trained to strictly predict the next word in a sequence, only aware of the context before the word. GPT models are trained on a massive dataset of text and code, and they can be fine-tuned for a range of NLP tasks, including text generation, question answering, and summarization.

In general, BERT is better at tasks that require deeper understanding of the context of words, whereas GPT is better suited for tasks that require generating text.

Hugging Face

Hugging Face is an artificial intelligence company that specializes in NLP. It provides a platform with tools and resources that enable developers to build, train, and deploy ML models focused on NLP tasks. One of the key offerings of Hugging Face is its library, Transformers, which includes pre-trained models that can be fine-tuned for various language tasks such as text classification, translation, summarization, and question answering.

Hugging Face integrates seamlessly with SageMaker, which is a fully managed service that enables developers and data scientists to build, train, and deploy ML models at scale. This synergy benefits users by providing a robust and scalable infrastructure to handle NLP tasks with the state-of-the-art models that Hugging Face offers, combined with the powerful and flexible ML services from AWS. You can also access Hugging Face models directly from Amazon SageMaker JumpStart, making it convenient to start with pre-built solutions.

Solution overview

We used the Hugging Face Transformers library to fine-tune transformer models on SageMaker for the task of adverse event classification. The training job is built using the SageMaker PyTorch estimator. SageMaker JumpStart also has some complementary integrations with Hugging Face that makes straightforward to implement. In this section, we describe the major steps involved in data preparation and model training.

Data preparation

We used the Adverse Drug Reaction Data (ade_corpus_v2) within the Hugging Face dataset with an 80/20 training/test split. The required data structure for our model training and inference has two columns:

• One column for text content as model input data.
• Another column for the label class. We have two possible classes for a text: Not_AE and Adverse_Event.

Model training and experimentation

In order to efficiently explore the space of possible Hugging Face models to fine-tune on our combined data of adverse events, we constructed a SageMaker hyperparameter optimization (HPO) job and passed in different Hugging Face models as a hyperparameter, along with other important hyperparameters such as training batch size, sequence length, models, and learning rate. The training jobs used an ml.p3dn.24xlarge instance and took an average of 30 minutes per job with that instance type. Training metrics were captured though the Amazon SageMaker Experiments tool, and each training job ran through 10 epochs.

We specify the following in our code:

• Training batch size – Number of samples that are processed together before the model weights are updated
• Sequence length – Maximum length of the input sequence that BERT can process
• Learning rate – How quickly the model updates its weights during training
• Models – Hugging Face pretrained models

# we use the Hyperparameter Tuner
from sagemaker.tuner import IntegerParameter,ContinuousParameter, CategoricalParameter
tuning_job_name = 'ade-hpo'
# Define exploration boundaries
hyperparameter_ranges = {
 'learning_rate': ContinuousParameter(5e-6,5e-4),
 'max_seq_length': CategoricalParameter(['16', '32', '64', '128', '256']),
 'train_batch_size': CategoricalParameter(['16', '32', '64', '128', '256']),
 'model_name': CategoricalParameter(["emilyalsentzer/Bio_ClinicalBERT", 
                                                            "dmis-lab/biobert-base-cased-v1.2", "monologg/biobert_v1.1_pubmed", "pritamdeka/BioBert-PubMed200kRCT", "saidhr20/pubmed-biobert-text-classification" ])
}

# create Optimizer
Optimizer = sagemaker.tuner.HyperparameterTuner(
    estimator=bert_estimator,
    hyperparameter_ranges=hyperparameter_ranges,
    base_tuning_job_name=tuning_job_name,
    objective_type='Maximize',
    objective_metric_name='f1',
    metric_definitions=[
        {'Name': 'f1',
         'Regex': "f1: ([0-9.]+).*$"}],  
    max_jobs=40,
    max_parallel_jobs=4,
)

Optimizer.fit({'training': inputs_data}, wait=False)

Results

The model that performed the best in our use case was the monologg/biobert_v1.1_pubmed model hosted on Hugging Face, which is a version of the BERT architecture that has been pre-trained on the Pubmed dataset, which consists of 19,717 scientific publications. Pre-training BERT on this dataset gives this model extra expertise when it comes to identifying context around medically related scientific terms. This boosts the model’s performance for the adverse event detection task because it has been pre-trained on medically specific syntax that shows up often in our dataset.

The following table summarizes our evaluation metrics.

Although these are relatively small and incremental improvements over the base BERT model, this nevertheless demonstrates some viable strategies to improve model performance through these methods. Synthetic data generation with Falcon seems to hold a lot of promise and potential for performance improvements, especially as these generative AI models get better over time.

Clean up

To avoid incurring future charges, delete any resources created like the model and model endpoints you created with the following code:

# Delete resources
model_predictor.delete_model()
model_predictor.delete_endpoint()

Conclusion

Many pharmaceutical companies today would like to automate the process of identifying adverse events from their customer interactions in a systematic way in order to help improve customer safety and outcomes. As we showed in this post, the fine-tuned LLM BioBERT with synthetically generated adverse events added to the data classifies the adverse events with high F1 scores and can be used to build a HIPAA-compliant solution for our customers.

As always, AWS welcomes your feedback. Please leave your thoughts and questions in the comments section.

About the authors

Zack Peterson is a data scientist in AWS Professional Services. He has been hands on delivering machine learning solutions to customers for many years and has a master’s degree in Economics.

Dr. Adewale Akinfaderin is a senior data scientist in Healthcare and Life Sciences at AWS. His expertise is in reproducible and end-to-end AI/ML methods, practical implementations, and helping global healthcare customers formulate and develop scalable solutions to interdisciplinary problems. He has two graduate degrees in Physics and a doctorate degree in Engineering.

Ekta Walia Bhullar, PhD, is a senior AI/ML consultant with the AWS Healthcare and Life Sciences (HCLS) Professional Services business unit. She has extensive experience in the application of AI/ML within the healthcare domain, especially in radiology. Outside of work, when not discussing AI in radiology, she likes to run and hike.

Han Man is a Senior Data Science & Machine Learning Manager with AWS Professional Services based in San Diego, CA. He has a PhD in Engineering from Northwestern University and has several years of experience as a management consultant advising clients in manufacturing, financial services, and energy. Today, he is passionately working with key customers from a variety of industry verticals to develop and implement ML and generative AI solutions on AWS.

Read More