Merge pull request #1221 from awslabs/bump/2.73.0

chore(release): 2.73.0

CHANGELOG.md

@@ -2,6 +2,14 @@
 
 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
 
+## [2.73.0](https://github.com/awslabs/aws-solutions-constructs/compare/v2.72.0...v2.73.0) (2024-10-21)
+
+Build on CDK v2.161.0
+
+### Features
+
+* **aws-sqs-pipes-stepfunctions:** new construct ([#1220](https://github.com/awslabs/aws-solutions-constructs/issues/1220)) ([c508279](https://github.com/awslabs/aws-solutions-constructs/commit/c508279b6f69a1126e8b58bf6bfc13d246ea2a9e))
+
 ## [2.72.0](https://github.com/awslabs/aws-solutions-constructs/compare/v2.71.0...v2.72.0) (2024-10-06)
 
 Build on CDK v2.161.0

DESIGN_GUIDELINES.md

@@ -332,9 +332,9 @@ Existing Inconsistencies would not be published, that’s for our internal use 
 | deadLetterQueueProps?	| [`sqs.QueueProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_sqs.QueueProps.html)|Optional user provided props to override the default props for the SQS queue.|
 | maxReceiveCount	| `int`	| The number of times a message can be unsuccessfully dequeued before being moved to the dead letter queue. Defaults to `15`. |
 | enableQueuePurging	| `boolean`	| Whether to grant additional permissions to the Lambda function enabling it to purge the SQS queue. Defaults to `false`. | This is only on 2 constructs, docs talk about a Lambda function role.|
-|enableEncryptionWithCustomerManagedKey?|`boolean`|If no key is provided, this flag determines whether the queue is encrypted with a new CMK or an AWS managed key. |This flag is ignored if any of the following are defined: queueProps.encryptionMasterKey, encryptionKey or encryptionKeyProps.|
-|encryptionKey?|[`kms.Key`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_kms.Key.html)|An optional, imported encryption key to encrypt the SQS Queue with.|Sending messages from an AWS service to an encrypted queue [requires a Customer Master key](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-key-management.html#compatibility-with-aws-services). Those constructs require these properties. |
-|encryptionKeyProps?|[`kms.KeyProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_kms.Key.html#construct-props)|Optional user provided properties to override the default properties for the KMS encryption key used to encrypt the SQS queue with.|
+|enableEncryptionWithCustomerManagedKey? or encryptQueueWithCmk?|`boolean`|If no key is provided, this flag determines whether the queue is encrypted with a new CMK or an AWS managed key. |In early constructs, the enableEncryptionKeyWithCustomerManagedKey name was used. Later constructs saw name collisions with this name and services like SNS, so the new queue specific name was adopted. This flag is ignored if any of the following are defined: queueProps.encryptionMasterKey, encryptionKey or encryptionKeyProps.|
+|encryptionKey? or queueEncryptionKey?|[`kms.Key`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_kms.Key.html)|An optional, imported encryption key to encrypt the SQS Queue with.|Sending messages from an AWS service to an encrypted queue [requires a Customer Master key](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-key-management.html#compatibility-with-aws-services). Those constructs require these properties. In early constructs, the enableEncryptionKeyWithCustomerManagedKey name was used. Later constructs saw name collisions with this name and services like SNS, so the new queue specific name was adopted. |
+|encryptionKeyProps? or queueEncryptionKeyProps?|[`kms.KeyProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_kms.Key.html#construct-props)|Optional user provided properties to override the default properties for the KMS encryption key used to encrypt the SQS queue with.|In early constructs, the enableEncryptionKeyWithCustomerManagedKey name was used. Later constructs saw name collisions with this name and services like SNS, so the new queue specific name was adopted. |
 
 
 **Required Construct Properties**

source/lerna.json

@@ -5,5 +5,5 @@
     "patterns/@aws-solutions-constructs/*"
   ],
   "rejectCycles": "true",
-  "version": "2.72.0"
+  "version": "2.73.0"
 }

source/patterns/@aws-solutions-constructs/aws-fargate-s3/lib/index.ts

@@ -240,14 +240,10 @@ export class FargateToS3 extends Construct {
     }
 
     // Add environment variables
-    const bucketArnEnvironmentVariableName = this.SetStringWithDefault(props.bucketArnEnvironmentVariableName, 'S3_BUCKET_ARN');
+    const bucketArnEnvironmentVariableName = defaults.CheckStringWithDefault(props.bucketArnEnvironmentVariableName, 'S3_BUCKET_ARN');
     this.container.addEnvironment(bucketArnEnvironmentVariableName, this.s3BucketInterface.bucketArn);
-    const bucketEnvironmentVariableName = this.SetStringWithDefault(props.bucketEnvironmentVariableName, 'S3_BUCKET_NAME');
+    const bucketEnvironmentVariableName = defaults.CheckStringWithDefault(props.bucketEnvironmentVariableName, 'S3_BUCKET_NAME');
     this.container.addEnvironment(bucketEnvironmentVariableName, this.s3BucketInterface.bucketName);
 
   }
-
-  private SetStringWithDefault(value: string | undefined, defaultValue: string) {
-    return value || defaultValue;
-  }
 }

source/patterns/@aws-solutions-constructs/aws-sqs-pipes-stepfunctions/.eslintignore

@@ -0,0 +1,7 @@
+lib/*.js
+test/*.js
+*.d.ts
+coverage
+test/integ.*.js.snapshot/
+test/cdk-integ.out.integ.*.snapshot
+test/lambda

source/patterns/@aws-solutions-constructs/aws-sqs-pipes-stepfunctions/.gitignore

@@ -0,0 +1,16 @@
+lib/*.js
+test/*.js
+!test/lambda/*
+*.js.map
+*.d.ts
+node_modules
+*.generated.ts
+dist
+.jsii
+
+.LAST_BUILD
+.nyc_output
+coverage
+.nycrc
+.LAST_PACKAGE
+*.snk

source/patterns/@aws-solutions-constructs/aws-sqs-pipes-stepfunctions/.npmignore

@@ -0,0 +1,21 @@
+# Exclude typescript source and config
+*.ts
+tsconfig.json
+coverage
+.nyc_output
+*.tgz
+*.snk
+*.tsbuildinfo
+
+# Include javascript files and typescript declarations
+!*.js
+!*.d.ts
+
+# Exclude jsii outdir
+dist
+
+# Include .jsii
+!.jsii
+
+# Include .jsii
+!.jsii

source/patterns/@aws-solutions-constructs/aws-sqs-pipes-stepfunctions/README.md

@@ -0,0 +1,100 @@
+# aws-sqs-pipes-stepfunctions module
+<!--BEGIN STABILITY BANNER-->
+
+---
+
+![Stability: Experimental](https://img.shields.io/badge/stability-Experimental-important.svg?style=for-the-badge)
+
+---
+<!--END STABILITY BANNER-->
+
+| **Reference Documentation**:| <span style="font-weight: normal">https://docs.aws.amazon.com/solutions/latest/constructs/</span>|
+|:-------------|:-------------|
+<div style="height:8px"></div>
+
+| **Language**     | **Package**        |
+|:-------------|-----------------|
+|![Python Logo](https://docs.aws.amazon.com/cdk/api/latest/img/python32.png) Python|`aws_solutions_constructs.aws_sqs_pipes_stepfunctions`|
+|![Typescript Logo](https://docs.aws.amazon.com/cdk/api/latest/img/typescript32.png) Typescript|`@aws-solutions-constructs/aws-sqs-pipes-stepfunctions`|
+|![Java Logo](https://docs.aws.amazon.com/cdk/api/latest/img/java32.png) Java|`software.amazon.awsconstructs.services.sqspipesstepfunctions`|
+
+## Overview
+This AWS Solutions Construct implements an AWS SQS queue whose messages are passed to an AWS Step Functions state machine by an Amazon Eventbridge pipe.
+
+Here is a minimal deployable pattern definition:
+
+Typescript
+``` typescript
+import { Construct } from 'constructs';
+import { Stack, StackProps } from 'aws-cdk-lib';
+import * as stepfunctions from 'aws-cdk-lib/aws-stepfunctions';
+import { SqsToPipesToStepfunctions, SqsToPipesToStepfunctionsProps } from "@aws-solutions-constructs/aws-sqs-pipes-stepfunctions";
+
+const startState = new stepfunctions.Pass(this, 'StartState');
+
+new SqsToPipesToStepfunctions(this, 'SqsToLambdaToStepfunctionsPattern', {
+  stateMachineProps: {
+    definition: startState
+  }
+});
+```
+
+## Pattern Construct Props
+
+| **Name**     | **Type**        | **Description** |
+|:-------------|:----------------|-----------------|
+|existingQueueObj?|[`sqs.Queue`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_sqs.Queue.html)|An optional, existing SQS queue to be used instead of the default queue. Providing both this and `queueProps` will cause an error.|
+|queueProps?|[`sqs.QueueProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_sqs.QueueProps.html)|Optional user provided properties to override the default properties for the SQS queue.|
+|encryptQueueWithCmk|`boolean`|Whether to encrypt the Queue with a customer managed KMS key (CMK). This is the default behavior, and this property defaults to true - if it is explicitly set to false then the Queue is encrypted with an Amazon managed KMS key. For a completely unencrypted Queue (not recommended), create the Queue separately from the construct and pass it in using the existingQueueObject. Since SNS subscriptions do not currently support SQS queues with AWS managed encryption keys, setting this to false will always result in an error from the underlying CDK - we have still included this property for consistency with topics and to be ready if the services one day support this functionality.|
+|queueEncryptionKeyProps?|[`kms.KeyProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_kms.Key.html#construct-props)|An optional subset of key properties to override the default properties used by constructs (`enableKeyRotation: true`). These properties will be used in constructing the CMK used to encrypt the SQS queue.|
+|existingQueueEncryptionKey?|[`kms.Key`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_kms.Key.html)|An optional CMK that will be used by the construct to encrypt the new SQS queue.|
+|deployDeadLetterQueue?|`boolean`|Whether to create a secondary queue to be used as a dead letter queue. Defaults to true.|
+|deadLetterQueueProps?|[`sqs.QueueProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_sqs.QueueProps.html)|Optional user-provided props to override the default props for the dead letter SQS queue.|
+|maxReceiveCount?|`number`|The number of times a message can be unsuccessfully dequeued before being moved to the dead letter queue. Defaults to 15.|
+|stateMachineProps|[`sfn.StateMachineProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_stepfunctions.StateMachineProps.html)|User provided props for the sfn.StateMachine.|
+|createCloudWatchAlarms?|`boolean`|Whether to create recommended CloudWatch alarms|
+| logGroupProps? | [logs.logGroupProps ](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_logs.LogGroupProps.html)| Optional user provided props to override the default props for for the CloudWatchLogs LogGroup for the state machine. |
+|pipeProps?|[ pipes.CfnPipeProps ](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_pipes.CfnPipeProps.html)|Optional customer provided settings for the EventBridge pipe. source, target, roleArn and enrichment settings are set by the construct and cannot be overriden here. The construct will generate default sourceParameters, targetParameters and logConfiguration (found [here](link)) that can be overriden by populating those values in these props. If the client wants to implement enrichment or a filter, this is where that information can be provided. Any other props can be freely overridden. If a client wants to set values such as batchSize, that can be done here in the sourceParameters property.|
+| enrichmentFunction? | [lambda.Function ](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_lambda.Function.html) | Optional - Lambda function that the construct will configure to be called to enrich the message between source and target. The construct will configure the pipe IAM role to allow invoking the function (but will not affect the IArole assigned to the function). Specifying both this and enrichmentStateMachine is an error. Default - undefined |
+| enrichmentStateMachine? | [sfn.StateMachine ](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_stepfunctions.StateMachine.html) | Optional - Step Functions state machine that the construct will configure to be called to enrich the message between source and target. The construct will configure the pipe IAM role to allow executing the state machine (but will not affect the IAM role assigned to the state machine). Specifying both this and enrichmentStateMachine is an error. Default - undefined |
+|logLevel?|PipesLogLevel|Threshold for what messages the new pipe sends to the log, PipesLogLevel.OFF, PipesLogLevel.ERROR, PipesLogLevel.INFO, PipesLogLevel.TRACE. The default is INFO. Setting the level to OFF will prevent any log group from being created. Providing pipeProps.logConfiguration will controls all aspects of logging and any construct provided log configuration is disabled. If pipeProps.logConfiguration is provided then specifying this or pipeLogProps is an error. |
+|pipeLogProps?|[logs.LogGroupProps]()| Default behavior is for the this construct to create a new CloudWatch Logs log group for the pipe. These props are used to override defaults set by AWS or this construct. If there are concerns about the cost of log storage, this is where a client can specify a shorter retention duration (in days) |
+
+## Pattern Properties
+
+| **Name**     | **Type**        | **Description** |
+|:-------------|:----------------|-----------------|
+|stateMachine|[`sfn.StateMachine`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_stepfunctions.StateMachine.html)|Returns an instance of StateMachine created by the construct.|
+|stateMachineLogGroup|[`logs.ILogGroup`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_logs.ILogGroup.html)|Returns an instance of the ILogGroup created by the construct for StateMachine|
+|cloudwatchAlarms?|[`cloudwatch.Alarm[]`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cloudwatch.Alarm.html)|Returns a list of alarms created by the construct.|
+|sqsQueue|[`sqs.Queue`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_sqs.Queue.html)|Returns an instance of the SQS queue created by the pattern. |
+|deadLetterQueue?|[`sqs.Queue`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_sqs.Queue.html)|Returns an instance of the dead letter queue created by the pattern, if one is deployed.|
+|encryptionKey?|[kms.IKey](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_kms.IKey.html)|Returns an instance of kms.Key used for the SQS queue if key is customer managed.|
+|pipe|[ pipes.CfnPipe](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_pipes.CfnPipe.html)| The L1 pipe construct created by this Solutions Construct. |
+| pipeRole | [iam.Role ](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_iam.Role.html) | The role created that allows the pipe to access both the source and the target. |
+
+## Default settings
+
+Out of the box implementation of the Construct without any override will set the following defaults:
+
+### Amazon SQS Queue
+* Deploy SQS dead-letter queue for the source SQS Queue.
+* Enable server-side encryption for source SQS Queue using AWS Managed KMS Key.
+* Enforce encryption of data in transit
+
+### AWS Step Functions State Machine
+* Deploy Step Functions standard state machine
+* Create CloudWatch log group with /vendedlogs/ prefix in name
+* Deploy best practices CloudWatch Alarms for the Step Functions
+
+### AWS EventBridge Pipe
+* Pipe configured with an SQS queue source and state machine target
+* A least privilege IAM role assigned to the pipe to access the queue and state machine
+* CloudWatch logs set up at the 'INFO' level
+* Encrypted with an AWS managed KMS key
+
+## Architecture
+![Architecture Diagram](architecture.png)
+
+***
+&copy; Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.