EC2 Image Builder

This project created all of the resources for an Image Builder Pipeline with a Tailscale subnet router installation.

On top of subnet router IP forwarding settings, the image is also configured with Tailscale performance best practices.

To see this image in use, check my other repository epomatti/aws-rds-tailscale-vpn.

Setup

Copy the .auto.tfvars file:

cp config/template.tfvars .auto.tfvars

You can find the latest image in SSM. Example for Canonical images used in this project:

aws ssm get-parameters --region us-east-2 --names \
   /aws/service/canonical/ubuntu/server/22.04/stable/current/arm64/hvm/ebs-gp2/ami-id

To create the project resources:

terraform init
terraform apply -auto-approve

Login to the AWS Console and start the build process manually.

A test instance will be automatically created by the workflow. If needed, a launch template is available to manually test the created image on EC2.

Checking the status of the CloudWatch agent:

/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status

Implementation

Recipe components:

• Linux update
• CloudWatch Agent
• Custom component for Tailscale with complete setup (tailscale.yaml)
• Reboot

SSM agent is kept in the image to be used during launch. Vulnerability scan will be performed by the pipeline.

Additional information:

• Image Builder publishes managed or curated images which have higher update rates, but appear to be unavailable via other means. I've created this thread to learn more about it.
• EC2 build instances require internet access. This can be configured via NAT in the VPC, or enabling public IP addresses auto-assign.
• Make sure to make the installation completely noninteractive, for example with DEBIAN_FRONTEND=noninteractive for Debian-based distributions.

Cross-account distribution

Steps documented following the cross-account distribution documentation with help from this blob post.

Requirement: You'll need to create a KMS CMK key and use it in your image recipe.

Source

In the Image Builder, create a distribution setting with the KMS CMK Key referenced and the target accounts.

Add this policy to the KMS CMK Key:

{
   "Sid": "Enable cross-account Image Builder distribution",
   "Effect": "Allow",
   "Principal": {
         "AWS": [
            "arn:aws:iam::ACCOUNT_A:root",
            "arn:aws:iam::ACCOUNT_B:root",
            "arn:aws:iam::ACCOUNT_C:root"
         ]
   },
   "Action": [
         "kms:Encrypt",
         "kms:Decrypt",
         "kms:ReEncrypt*",
         "kms:GenerateDataKey*",
         "kms:DescribeKey",
         "kms:CreateGrant",
         "kms:ListGrants",
         "kms:RevokeGrant"
   ],
   "Resource": "*"
}

Destination

1. Create the EC2ImageBuilderDistributionCrossAccountRole on each destination account. This role will be used by the source account. The trust policy should look like below, replacing the SOURCE_ACCOUNT with the source account id.

   {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::SOURCE_ACCOUNT:root"
            },
            "Action": "sts:AssumeRole",
            "Condition": {}
        }
    ]
   }

2. Add the managed policy Ec2ImageBuilderCrossAccountDistributionAccess to the role.

3. Add this inline policy to the role:

   {
    "Version": "2012-10-17",
    "Statement": [
    	{
    		"Sid": "AllowRoleToPerformKMSOperationsOnBehalfOfTheDestinationAccount",
    		"Effect": "Allow",
    		"Action": [
    			"kms:Encrypt",
    			"kms:Decrypt",
    			"kms:ReEncrypt*",
    			"kms:GenerateDataKey*",
    			"kms:DescribeKey",
    			"kms:CreateGrant",
    			"kms:ListGrants",
    			"kms:RevokeGrant"
    		],
    		"Resource": "*"
    	}
    ]
   }

Other integrations

Additional access might be required for automation, since now services that will use your AMI will require access to the KMS CMK Key in cross-account setup.

{
   "Sid": "",
   "Effect": "Allow",
   "Principal": {
         "AWS": [
            "arn:aws:iam::ACCOUNT_A:role/YourRoleStage",
            "arn:aws:iam::ACCOUNT_B:role/YourRoleProduction"
         ]
   },
   "Action": [
         "kms:Encrypt",
         "kms:Decrypt",
         "kms:ReEncrypt*",
         "kms:GenerateDataKey*",
         "kms:DescribeKey"
   ],
   "Resource": "*"
},
{
   "Sid": "",
   "Effect": "Allow",
   "Principal": {
         "AWS": [
            "arn:aws:iam::ACCOUNT_A:role/YourRoleStage",
            "arn:aws:iam::ACCOUNT_B:role/YourRoleProduction"
         ]
   },
   "Action": "kms:CreateGrant",
   "Resource": "*",
   "Condition": {
         "Bool": {
            "kms:GrantIsForAWSResource": "true"
         }
   }
}

---

Clean-up

Destroy the resources after use:

terraform destroy -auto-approve