-
Interface Summary Interface Description AccessConrtolLambdaProps AdminRoleProps BillingRoleProps ComprehendS3olabProps CustSupportRoleProps GeneralRoleProps LambdaArnCaptorResourceProps RedactionLambdaProps S3AccessPointNames -
Enum Summary Enum Description IamRoleName
Package io.github.hsiehshujeng.cdk.comprehend.s3olap Description
cdk-comprehend-s3olap
This construct creates the foundation for developers to explore the combination of Amazon S3 Object Lambda and Amazon Comprehend for PII scenarios and it is designed with flexibility, i.e, the developers could tweak arguments via CDK to see how AWS services work and behave.
Table of Contents
Serverless Architecture
Access Control
Data Flow
Ram R. and Austin Q., 2021
Arhictecture
Ram R. and Austin Q., 2021
Redaction
Ram R. and Austin Q., 2021
Ram R. and Austin Q., 2021
Introduction
The architecture was introduced by Ram Ramani and Austin Quam and was posted on the AWS Blog as Protect PII using Amazon S3 Object Lambda to process and modify data during retrieval. I converted the architecture into a CDK constrcut for 4 programming languages. With this construct, you could manage the properties of IAM roles, the Lambda functions with Amazon Comprehend, and few for the constrcut. Before deploying the construct via the CDK, you could either places the text files, i.e., those for the access control case and redaction case, under a directory with a specific name as the following or just deploying directly yet you need to upload the text files onto the S3 buckets manually yourself. It's all your choie.
# For the access control case.
$ cd ${ROOT_DIRECTORY_CDK_APPLICATION}
$ mkdir -p files/access_control
$ curl -o survey-results.txt https://raw.githubusercontent.com/aws-samples/amazon-comprehend-examples/master/s3_object_lambda_pii_protection_blog/access-control/survey-results.txt
$ curl -o innocuous.txt https://raw.githubusercontent.com/aws-samples/amazon-comprehend-examples/master/s3_object_lambda_pii_protection_blog/access-control/innocuous.txt
# For the redaction case.
$ cd ${ROOT_DIRECTORY_CDK_APPLICATION}
$ mkdir -p files/redaction
$ curl -o transcript.txt https://raw.githubusercontent.com/aws-samples/amazon-comprehend-examples/master/s3_object_lambda_pii_protection_blog/redaction/transcript.txt
Example
Typescript
You could also refer to here.
$ cdk --init language typescript $ yarn add cdk-comprehend-s3olap
// Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import software.amazon.awscdk.core.*;
import cdk.comprehend.s3olap.ComprehendS3olab;
public class TypescriptStack extends Stack {
public TypescriptStack(Construct scope, String id) {
this(scope, id, null);
}
public TypescriptStack(Construct scope, String id, StackProps props) {
super(scope, id, props);
Object s3olab = ComprehendS3olab.Builder.create(this, "PiiDemo")
.adminRedactionLambdaConfig(Map.of(
"maskCharacter", " ",
"unsupportedFileHandling", "PASS"))
.billingRedactionLambdaConfig(Map.of(
"maskMode", "REPLACE_WITH_PII_ENTITY_TYPE",
"piiEntityTypes", "AGE,DRIVER_ID,IP_ADDRESS,MAC_ADDRESS,PASSPORT_NUMBER,PASSWORD,SSN"))
.cusrtSupportRedactionLambdaConfig(Map.of(
"maskMode", "REPLACE_WITH_PII_ENTITY_TYPE",
"piiEntityTypes", " BANK_ACCOUNT_NUMBER,BANK_ROUTING,CREDIT_DEBIT_CVV,CREDIT_DEBIT_EXPIRY,CREDIT_DEBIT_NUMBER,SSN"))
.build();
new CfnOutput(this, "OPiiAccessControlLambdaArn", new CfnOutputProps().value(s3olab.getPiiAccessConrtolLambdaArn()));
new CfnOutput(this, "OAdminLambdaArn", new CfnOutputProps().value(s3olab.getAdminLambdaArn()));
new CfnOutput(this, "OBillingLambdaArn", new CfnOutputProps().value(s3olab.getBillingLambdaArn()));
new CfnOutput(this, "OCustomerSupportLambdaArn", new CfnOutputProps().value(s3olab.getCustomerSupportLambdaArn()));
new CfnOutput(this, "OS3ObjectLambdaGeneralArn", new CfnOutputProps().value(s3olab.getS3objectLambdaAccessControlArn()));
new CfnOutput(this, "OS3ObjectLambdaAdminArn", new CfnOutputProps().value(s3olab.getS3objectLambdaAdminArn()));
new CfnOutput(this, "OS3ObjectLambdaBillingArn", new CfnOutputProps().value(s3olab.getS3objectLambdaBillingArn()));
new CfnOutput(this, "OS3ObjectLambdaCustomerSupportArn", new CfnOutputProps().value(s3olab.getCustomerSupportLambdaArn()));
}
}
App app = new App();
new TypescriptStack(app, "TypescriptStack", new StackProps()
.stackName("Comprehend-S3olap"));
Python
You could also refer to here.
# upgrading related Python packages $ python -m ensurepip --upgrade $ python -m pip install --upgrade pip $ python -m pip install --upgrade virtualenv # initialize a CDK Python project $ cdk init --language python # make packages installed locally instead of globally $ source .venv/bin/activate $ # add "cdk-comprehend-s3olap==0.0.4" into `setup.py` $ python -m pip install --upgrade -r requirements.txt
from aws_cdk import core as cdk
from cdk_comprehend_s3olap import ComprehendS3olab
class PythonStack(cdk.Stack):
def __init__(self, scope: cdk.Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
s3olab = ComprehendS3olab(self, "PiiDemo",
admin_redaction_lambda_config={
"mask_character": " ",
"unsupported_file_handling": "PASS"
},
billing_redaction_lambda_config={
"mask_mode": "REPLACE_WITH_PII_ENTITY_TYPE",
"pii_entity_types": "AGE,DRIVER_ID,IP_ADDRESS,MAC_ADDRESS,PASSPORT_NUMBER,PASSWORD,SSN"
},
cusrt_support_redaction_lambda_config={
"mask_mode": "REPLACE_WITH_PII_ENTITY_TYPE",
"pii_entity_types": " BANK_ACCOUNT_NUMBER,BANK_ROUTING,CREDIT_DEBIT_CVV,CREDIT_DEBIT_EXPIRY,CREDIT_DEBIT_NUMBER,SSN"
}
)
cdk.CfnOutput(self, "OPiiAccessControlLambdaArn", value=s3olab.pii_access_conrtol_lambda_arn)
cdk.CfnOutput(self, "OAdminLambdaArn", value=s3olab.admin_lambda_arn)
cdk.CfnOutput(self, "OBillingLambdaArn", value=s3olab.billing_lambda_arn)
cdk.CfnOutput(self, "OCustomerSupportLambdaArn", value=s3olab.customer_support_lambda_arn)
cdk.CfnOutput(self, "OS3ObjectLambdaGeneralArn", value=s3olab.s3object_lambda_access_control_arn)
cdk.CfnOutput(self, "OS3ObjectLambdaAdminArn", value=s3olab.s3object_lambda_admin_arn)
cdk.CfnOutput(self, "OS3ObjectLambdaBillingArn", value=s3olab.s3object_lambda_billing_arn)
cdk.CfnOutput(self, "OS3ObjectLambdaCustomerSupportArn", value=s3olab.customer_support_lambda_arn)
Java
You could also refer to here.
$ cdk init --language java
$ mvn package
```xml
.
.
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<cdk.version>1.112.0</cdk.version>
<constrcut.verion>0.0.4</constrcut.verion>
<junit.version>5.7.1</junit.version>
</properties>
.
.
<dependencies>
<!-- AWS Cloud Development Kit -->
<dependency>
<groupId>software.amazon.awscdk</groupId>
<artifactId>core</artifactId>
<version>${cdk.version}</version>
</dependency>
<dependency>
<groupId>io.github.hsiehshujeng</groupId>
<artifactId>cdk-comprehend-s3olap</artifactId>
<version>${constrcut.verion}</version>
</dependency>
.
.
.
</dependencies>
package com.myorg;
import software.amazon.awscdk.core.CfnOutput;
import software.amazon.awscdk.core.CfnOutputProps;
import software.amazon.awscdk.core.Construct;
import software.amazon.awscdk.core.Stack;
import software.amazon.awscdk.core.StackProps;
import io.github.hsiehshujeng.cdk.comprehend.s3olap.RedactionLambdaProps;
import io.github.hsiehshujeng.cdk.comprehend.s3olap.ComprehendS3olab;
import io.github.hsiehshujeng.cdk.comprehend.s3olap.ComprehendS3olabProps;
public class JavaStack extends Stack {
public JavaStack(final Construct scope, final String id) {
this(scope, id, null);
}
public JavaStack(final Construct scope, final String id, final StackProps props) {
super(scope, id, props);
ComprehendS3olab s3olab = new ComprehendS3olab(this, "PiiDemo", ComprehendS3olabProps.builder()
.adminRedactionLambdaConfig(
RedactionLambdaProps.builder()
.maskCharacter(" ")
.unsupportedFileHandling("PASS").build())
.billingRedactionLambdaConfig(
RedactionLambdaProps.builder()
.maskMode("REPLACE_WITH_PII_ENTITY_TYPE")
.piiEntityTypes("AGE,DRIVER_ID,IP_ADDRESS,MAC_ADDRESS,PASSPORT_NUMBER,PASSWORD,SSN")
.build())
.cusrtSupportRedactionLambdaConfig(
RedactionLambdaProps.builder()
.maskMode("REPLACE_WITH_PII_ENTITY_TYPE")
.piiEntityTypes("BANK_ACCOUNT_NUMBER,BANK_ROUTING,CREDIT_DEBIT_CVV,CREDIT_DEBIT_EXPIRY,CREDIT_DEBIT_NUMBER,SSN")
.build())
.build()
);
new CfnOutput(this, "OPiiAccessControlLambdaArn", CfnOutputProps.builder().value(s3olab.getPiiAccessConrtolLambdaArn()).build());
new CfnOutput(this, "OAdminLambdaArn", CfnOutputProps.builder().value(s3olab.getAdminLambdaArn()).build());
new CfnOutput(this, "OBillingLambdaArn", CfnOutputProps.builder().value(s3olab.getBillingLambdaArn()).build());
new CfnOutput(this, "OCustomerSupportLambdaArn", CfnOutputProps.builder().value(s3olab.getCustomerSupportLambdaArn()).build());
new CfnOutput(this, "OS3ObjectLambdaGeneralArn", CfnOutputProps.builder().value(s3olab.getS3objectLambdaAccessControlArn()).build());
new CfnOutput(this, "OS3ObjectLambdaAdminArn", CfnOutputProps.builder().value(s3olab.getS3objectLambdaAdminArn()).build());
new CfnOutput(this, "OS3ObjectLambdaBillingArn", CfnOutputProps.builder().value(s3olab.getS3objectLambdaBillingArn()).build());
new CfnOutput(this, "OS3ObjectLambdaCustomerSupportArn", CfnOutputProps.builder().value(s3olab.getCustomerSupportLambdaArn()).build());
}
}
C#
You could also refer to here.
$ cdk init --language csharp $ dotnet add src/Csharp package Comprehend.S3olap --version 0.0.7
using Amazon.CDK;
using ScottHsieh.Cdk;
namespace Csharp
{
public class CsharpStack : Stack
{
internal CsharpStack(Construct scope, string id, IStackProps props = null) : base(scope, id, props)
{
var S3olab = new ComprehendS3olab(this, "PiiDemo", new ComprehendS3olabProps
{
AdminRedactionLambdaConfig = new RedactionLambdaProps
{
MaskCharacter = " ",
UnsupportedFileHandling = "PASS"
},
BillingRedactionLambdaConfig = new RedactionLambdaProps
{
MaskMode = "REPLACE_WITH_PII_ENTITY_TYPE",
PiiEntityTypes = "AGE,DRIVER_ID,IP_ADDRESS,MAC_ADDRESS,PASSPORT_NUMBER,PASSWORD,SSN"
},
CusrtSupportRedactionLambdaConfig = new RedactionLambdaProps
{
MaskMode = "REPLACE_WITH_PII_ENTITY_TYPE",
PiiEntityTypes = "BANK_ACCOUNT_NUMBER,BANK_ROUTING,CREDIT_DEBIT_CVV,CREDIT_DEBIT_EXPIRY,CREDIT_DEBIT_NUMBER,SSN"
}
});
new CfnOutput(this, "OPiiAccessControlLambdaArn", new CfnOutputProps { Value = S3olab.PiiAccessConrtolLambdaArn });
new CfnOutput(this, "OAdminLambdaArn", new CfnOutputProps { Value = S3olab.AdminLambdaArn });
new CfnOutput(this, "OBillingLambdaArn", new CfnOutputProps { Value = S3olab.BillingLambdaArn });
new CfnOutput(this, "OCustomerSupportLambdaArn", new CfnOutputProps { Value = S3olab.CustomerSupportLambdaArn });
new CfnOutput(this, "OS3ObjectLambdaGeneralArn", new CfnOutputProps { Value = S3olab.S3objectLambdaAccessControlArn });
new CfnOutput(this, "OS3ObjectLambdaAdminArn", new CfnOutputProps { Value = S3olab.S3objectLambdaAdminArn });
new CfnOutput(this, "OS3ObjectLambdaBillingArn", new CfnOutputProps { Value = S3olab.S3objectLambdaBillingArn });
new CfnOutput(this, "OS3ObjectLambdaCustomerSupportArn", new CfnOutputProps { Value = S3olab.CustomerSupportLambdaArn });
}
}
}
Some Notes
- You should see similar items as the following diagram displays after deploying the constrcut.
- After creating the foundation with success, you could switch roles that the consrtcut creates for you and see how Amazon S3 Object Lambda works.
- You explore Amazon S3 Object Lambda through the Object Lambda access points and open or download the text files.
- Lambda code that incorporates with Amazon Comprehend could be see here.