Detect signatures on documents or images using the signatures feature in Amazon Textract

by
Amazon AWS

Amazon Textract is a machine learning (ML) service that automatically extracts text, handwriting, and data from any document or image. AnalyzeDocument Signatures is a feature within Amazon Textract that offers the ability to automatically detect signatures on any document. This can reduce the need for human review, custom code, or ML experience.

In this post, we discuss the benefits of the AnalyzeDocument Signatures feature and how the AnalyzeDocument Signatures API helps detect signatures in documents. We also walk through how to use the feature through the Amazon Textract console and provide code examples to use the API and process the response with the Amazon Textract response parser library. Lastly, we share some best practices for using this feature.

Benefits of the Signatures feature

Our customers from insurance, mortgage, legal, and tax industries face the challenge of processing huge volumes of paper-based documents while adhering to regulatory and compliance requirements that require signatures in documents. You may need to ensure that specific forms such as loan applications or claims submitted by your end clients contain signatures before you start processing the application. For certain document processing workflows, you may need to go a step further to extract and compare the signatures for verification.

Historically, customers generally route the documents to a human reviewer to detect signatures. Using human reviewers to detect signatures tends to require a significant amount of time and resources. It can also lead to inefficiencies in the document processing workflow, resulting in longer turnaround times and a poor end-user experience.

The AnalyzeDocument Signatures feature allows you to automatically detect handwritten signatures, electronic signatures, and initials on documents. This can help you build an automated scalable solution with less reliance on costly and time-consuming manual processing. Not only can you use this feature to verify whether the document is signed, but you can also validate if a particular field in the form is signed using the location details of the detected signatures. You can also use location information to redact personally identifiable information (PII) in a document.

How AnalyzeDocument Signatures detects signatures in documents

The AnalyzeDocument API has four feature types: Forms, Tables, Queries, and Signatures. When Amazon Textract processes documents, the results are returned in an array of Block objects. The Signatures feature can be used by itself or in combination with other feature types. When used by itself, the Signatures feature type provides a JSON response that includes the location and confidence scores of the detected signatures and raw text (words and lines) from the documents. The Signatures feature combined with other feature types, such as Forms and Tables, can help draw useful insights. In cases where the feature is used with Forms and Tables, the response shows the signature as part of key value pair or a table cell. For example, the response for the following form contains the key as Signature of Lender and the value as the Block object.

How to use the Signatures feature on the Amazon Textract console

Before we get started with the API and code samples, let’s review the Amazon Textract console. After you upload the document to the Amazon Textract console, select Signature detection in the Configure document section and choose Apply configuration.

The following screenshot shows an example of a paystub on the Signatures tab for the Analyze Document API on the Amazon Textract console.

The feature detects and presents the signature with its corresponding page and confidence score.

Code examples

You can use the Signatures feature to detect signatures on different types of documents, such as checks, loan application forms, claims forms, paystubs, mortgage documents, bank statements, lease agreements, and contracts. In this section, we discuss some of these documents and show how to invoke the AnalyzeDocument API with the Signatures parameter to detect signatures.

The input document can either be in a byte array format or located in an Amazon Simple Storage Service (Amazon S3) bucket. For documents in a byte array format, you can submit image bytes to an Amazon Textract API operation by using the bytes property. Signatures as a feature type is supported by the AnalyzeDocument API for synchronous document processing and StartDocumentAnalysis for asynchronous processing of documents.

In the following example, we detect signatures on an employment verification letter.

We use the following sample Python code:

import boto3
import json

#create a Textract Client
textract = boto3.client('textract')
#Document
documentName = image_filename

response = None
with open(image_filename, 'rb') as document:
    imageBytes = bytearray(document.read())

# Call Textract AnalyzeDocument by passing a document from local disk
response = textract.analyze_document(
    Document={'Bytes': imageBytes},
    FeatureTypes=["FORMS",'SIGNATURES']
    )

Let’s analyze the response we get from the AnalyzeDocument API. The following response has been trimmed to only show the relevant parts. The response has a BlockType of SIGNATURE that shows the confidence score, ID for the block, and bounding box details:

'BlockType': 'SIGNATURE',
   'Confidence': 38.468597412109375,
   'Geometry': {'BoundingBox': {'Width': 0.15083004534244537,
     'Height': 0.019236255437135696,
     'Left': 0.11393339931964874,
     'Top': 0.8885205388069153},
    'Polygon': [{'X': 0.11394496262073517, 'Y': 0.8885205388069153},
     {'X': 0.2647634446620941, 'Y': 0.8887625932693481},
     {'X': 0.264753133058548, 'Y': 0.9077568054199219},
     {'X': 0.11393339931964874, 'Y': 0.907513439655304}]},
   'Id': '609f749c-5e79-4dd4-abcc-ad47c6ebf777'}]

We use the following code to print the ID and location in a tabulated format:

#print detected text
from tabulate import tabulate
d = []
for item in response["Blocks"]:
    if item["BlockType"] == "SIGNATURE":
        d.append([item["Id"],item["Geometry"]])

print(tabulate(d, headers=["Id", "Geometry"],tablefmt="grid",maxcolwidths=[None, 100]))

The following screenshot shows our results.

More details and the complete code is available in the notebook on the GitHub repo.

For documents that have legible signatures in key value formats, we can use the Textract response parser to extract just the signature fields by searching for the key and the corresponding value to those keys:

from trp import Document
doc = Document(response)
d = []

for page in doc.pages:
    # Search fields by key
    print("nSearch Fields:")
    key = "Signature"
    fields = page.form.searchFieldsByKey(key)
    for field in fields:
        d.append([field.key, field.value])        

print(tabulate(d, headers=["Key", "Value"]))

The preceding code returns the following results:

Search Fields:
Key                        		Value
-------------------------  		--------------
8. Signature of Applicant 	Paulo Santos
26. Signature of Employer 	Richard Roe
3. Signature of Lender     	Carlos Salazar

Note that in order to transcribe the signatures in this way, the signatures must be legible.

Best practices for using the Signatures feature

Consider the following best practices when using this feature:

  • For real-time responses, use the synchronous operation of the AnalyzeDocument API. For use cases where you don’t need the response in real time, such as batch processing, we suggest using the asynchronous operation of the API.
  • The Signatures feature works best when there are up to three signatures on a page. When there are more than three signatures on a page, it’s best to split the page into sections and feed each of the sections separately to the API.
  • Use the confidence scores provided with the detected signatures to route the documents for human review when the scores don’t meet your required threshold. The confidence score is not a measure of accuracy, but an estimate of the model’s confidence in its prediction. You should select a confidence score that makes the most sense for your use case.

Summary

In this post, we provided an overview of the Signatures feature of Amazon Textract to automatically detect signatures on documents, such as paystubs, rental lease agreements, and contracts. AnalyzeDocument Signatures reduces the need for human reviewers and helps you reduce costs, save time, and build scalable solutions for document processing.

To get started, log on to the Amazon Textract console to try out the feature. To learn more about Amazon Textract capabilities, refer to Amazon Textract, the Amazon Textract Developer Guide, or Textract Resources.

About the Authors

Maran Chandrasekaran is a Senior Solutions Architect at Amazon Web Services, working with our enterprise customers. Outside of work, he loves to travel and ride his motorcycle in Texas Hill Country.

Shibin Michaelraj is a Sr. Product Manager with the AWS Textract team. He is focused on building AI/ML-based products for AWS customers.

Suprakash Dutta is a Sr. Solutions Architect at Amazon Web Services. He focuses on digital transformation strategy, application modernization and migration, data analytics, and machine learning. He is part of the AI/ML community at AWS and designs intelligent document processing solutions.

Read More