-
Notifications
You must be signed in to change notification settings - Fork 80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Run integration tests on full Apple CI matrix #1476
Changes from all commits
f63e1aa
5b60da7
c6730ba
352e782
93b6ea9
594b30c
c1dba6d
403021d
f9b13b9
8bef515
c334391
226797c
857c4db
1c98926
9375001
9a4a6a9
5e2c6bc
a317f60
bb57c3b
22b87d5
0819460
0b9ba4c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -18,20 +18,36 @@ jobs: | |
strategy: | ||
fail-fast: false | ||
matrix: | ||
# This matrix runs tests on Mac, on oldest & newest supported Xcodes | ||
# This matrix runs tests on iOS sim & Mac, on oldest & newest supported Xcodes | ||
runner: | ||
- macos-13-xlarge | ||
- macos-14-xlarge | ||
xcode: | ||
- Xcode_14.1 | ||
- Xcode_15.3 | ||
destination: | ||
- 'platform=iOS Simulator,OS=16.1,name=iPhone 14' | ||
- 'platform=iOS Simulator,OS=17.4,name=iPhone 15' | ||
- 'platform=tvOS Simulator,OS=16.1,name=Apple TV 4K (3rd generation) (at 1080p)' | ||
- 'platform=tvOS Simulator,OS=17.4,name=Apple TV 4K (3rd generation) (at 1080p)' | ||
- 'platform=OS X' | ||
exclude: | ||
# Don't run old macOS with new Xcode | ||
- runner: macos-13-xlarge | ||
xcode: Xcode_15.3 | ||
# Don't run new macOS with old Xcode | ||
- runner: macos-14-xlarge | ||
xcode: Xcode_14.1 | ||
# Don't run old simulators with new Xcode | ||
- destination: 'platform=tvOS Simulator,OS=16.1,name=Apple TV 4K (3rd generation) (at 1080p)' | ||
xcode: Xcode_15.3 | ||
- destination: 'platform=iOS Simulator,OS=16.1,name=iPhone 14' | ||
xcode: Xcode_15.3 | ||
# Don't run new simulators with old Xcode | ||
- destination: 'platform=tvOS Simulator,OS=17.4,name=Apple TV 4K (3rd generation) (at 1080p)' | ||
xcode: Xcode_14.1 | ||
- destination: 'platform=iOS Simulator,OS=17.4,name=iPhone 15' | ||
xcode: Xcode_14.1 | ||
steps: | ||
- name: Configure AWS Credentials for Integration Tests | ||
uses: aws-actions/configure-aws-credentials@v4 | ||
|
@@ -80,12 +96,28 @@ jobs: | |
java-version: 17 | ||
- name: Tools Versions | ||
run: ./scripts/ci_steps/log_tool_versions.sh | ||
- name: Add Credentials to Test Plan | ||
run: | | ||
# JSON-escape the credentials. They are also surrounded with double quotes. | ||
AKID_ESCAPED=`echo -n "$AWS_ACCESS_KEY_ID" | jq -Rsa .` | ||
SECRET_ESCAPED=`echo -n "$AWS_SECRET_ACCESS_KEY" | jq -Rsa .` | ||
REGION_ESCAPED=`echo -n "$AWS_DEFAULT_REGION" | jq -Rsa .` | ||
TOKEN_ESCAPED=`echo -n "$AWS_SESSION_TOKEN" | jq -Rsa .` | ||
# Insert the credentials into the .xctestplan file, write the modified JSON | ||
# to a temp file, then move the temp over the original. | ||
jq ".defaultOptions.environmentVariableEntries += [{\"key\": \"AWS_ACCESS_KEY_ID\", \"value\": $AKID_ESCAPED}, {\"key\": \"AWS_SECRET_ACCESS_KEY\", \"value\": $SECRET_ESCAPED}, {\"key\": \"AWS_DEFAULT_REGION\", \"value\": $REGION_ESCAPED}, {\"key\": \"AWS_SESSION_TOKEN\", \"value\": $TOKEN_ESCAPED}]" XCTestPlans/AWSIntegrationTestsOnCI.xctestplan > testplan.tmp | ||
mv testplan.tmp XCTestPlans/AWSIntegrationTestsOnCI.xctestplan | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Above is the "secret sauce": to get AWS credentials into the Xcode simulators: use the ( There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't believe For a script I wrote to run on Amazon Linux I built-in a bash script method that checks if jq is installed and installs it if not, you may need to do something similar here There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. NVM I see it comes with github actions |
||
- name: Prepare Integration Tests | ||
run: ./scripts/ci_steps/prepare_integration_tests.sh | ||
- name: Build Integration Tests | ||
run: swift build --build-tests | ||
- name: Run Integration Tests | ||
run: swift test | ||
run: | | ||
set -o pipefail && \ | ||
NSUnbufferedIO=YES xcodebuild \ | ||
-scheme AWSIntegrationTestsOnCI \ | ||
-testPlan AWSIntegrationTestsOnCI \ | ||
-destination '${{ matrix.destination }}' \ | ||
test 2>&1 \ | ||
| xcpretty | ||
|
||
linux: | ||
runs-on: ubuntu-latest | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
<?xml version="1.0" encoding="UTF-8"?> | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is basically a scheme with default settings, that delegates all test configuration to the test plan at |
||
<Scheme | ||
LastUpgradeVersion = "1530" | ||
version = "1.7"> | ||
<BuildAction | ||
parallelizeBuildables = "YES" | ||
buildImplicitDependencies = "YES" | ||
buildArchitectures = "Automatic"> | ||
</BuildAction> | ||
<TestAction | ||
buildConfiguration = "Debug" | ||
selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB" | ||
selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB" | ||
shouldUseLaunchSchemeArgsEnv = "YES"> | ||
<TestPlans> | ||
<TestPlanReference | ||
reference = "container:XCTestPlans/AWSIntegrationTestsOnCI.xctestplan" | ||
default = "YES"> | ||
</TestPlanReference> | ||
</TestPlans> | ||
</TestAction> | ||
<LaunchAction | ||
buildConfiguration = "Debug" | ||
selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB" | ||
selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB" | ||
launchStyle = "0" | ||
useCustomWorkingDirectory = "NO" | ||
ignoresPersistentStateOnLaunch = "NO" | ||
debugDocumentVersioning = "YES" | ||
debugServiceExtension = "internal" | ||
allowLocationSimulation = "YES"> | ||
</LaunchAction> | ||
<ProfileAction | ||
buildConfiguration = "Release" | ||
shouldUseLaunchSchemeArgsEnv = "YES" | ||
savedToolIdentifier = "" | ||
useCustomWorkingDirectory = "NO" | ||
debugDocumentVersioning = "YES"> | ||
</ProfileAction> | ||
<AnalyzeAction | ||
buildConfiguration = "Debug"> | ||
</AnalyzeAction> | ||
<ArchiveAction | ||
buildConfiguration = "Release" | ||
revealArchiveInOrganizer = "YES"> | ||
</ArchiveAction> | ||
</Scheme> |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,95 @@ | ||
{ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This file defines the test targets and the configuration to be used when running integration tests. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This file is actually ignored by Git, and you have to "force" Git to commit changes to it with It must remain ignored, so that Git credentials aren't accidentally committed. |
||
"configurations" : [ | ||
{ | ||
"id" : "FF4C1A8F-5D70-4A2A-BFDE-77467500C547", | ||
"name" : "Test Scheme Action", | ||
"options" : { | ||
} | ||
} | ||
], | ||
"defaultOptions" : { | ||
"environmentVariableEntries" : [ | ||
|
||
] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The step defined in the Github workflow above will fill this empty JSON array with environment variable settings for the test run's AWS credentials. |
||
}, | ||
"testTargets" : [ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This section of the file contains all the test targets to be run as part of integration tests. Note that when we add a new integration test target to the repo, we will have to add it here to make it run on CI. |
||
{ | ||
"target" : { | ||
"containerPath" : "container:", | ||
"identifier" : "AWSS3IntegrationTests", | ||
"name" : "AWSS3IntegrationTests" | ||
} | ||
}, | ||
{ | ||
"target" : { | ||
"containerPath" : "container:", | ||
"identifier" : "AWSCloudFrontKeyValueStoreIntegrationTests", | ||
"name" : "AWSCloudFrontKeyValueStoreIntegrationTests" | ||
} | ||
}, | ||
{ | ||
"target" : { | ||
"containerPath" : "container:", | ||
"identifier" : "AWSEC2IntegrationTests", | ||
"name" : "AWSEC2IntegrationTests" | ||
} | ||
}, | ||
{ | ||
"target" : { | ||
"containerPath" : "container:", | ||
"identifier" : "AWSECSIntegrationTests", | ||
"name" : "AWSECSIntegrationTests" | ||
} | ||
}, | ||
{ | ||
"target" : { | ||
"containerPath" : "container:", | ||
"identifier" : "AWSEventBridgeIntegrationTests", | ||
"name" : "AWSEventBridgeIntegrationTests" | ||
} | ||
}, | ||
{ | ||
"target" : { | ||
"containerPath" : "container:", | ||
"identifier" : "AWSKinesisIntegrationTests", | ||
"name" : "AWSKinesisIntegrationTests" | ||
} | ||
}, | ||
{ | ||
"target" : { | ||
"containerPath" : "container:", | ||
"identifier" : "AWSMediaConvertIntegrationTests", | ||
"name" : "AWSMediaConvertIntegrationTests" | ||
} | ||
}, | ||
{ | ||
"target" : { | ||
"containerPath" : "container:", | ||
"identifier" : "AWSRoute53IntegrationTests", | ||
"name" : "AWSRoute53IntegrationTests" | ||
} | ||
}, | ||
{ | ||
"target" : { | ||
"containerPath" : "container:", | ||
"identifier" : "AWSSQSIntegrationTests", | ||
"name" : "AWSSQSIntegrationTests" | ||
} | ||
}, | ||
{ | ||
"target" : { | ||
"containerPath" : "container:", | ||
"identifier" : "AWSSTSIntegrationTests", | ||
"name" : "AWSSTSIntegrationTests" | ||
} | ||
}, | ||
{ | ||
"target" : { | ||
"containerPath" : "container:", | ||
"identifier" : "AWSTranscribeStreamingIntegrationTests", | ||
"name" : "AWSTranscribeStreamingIntegrationTests" | ||
} | ||
} | ||
], | ||
"version" : 1 | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The matrix used here for Apple platforms is copied verbatim out of https://github.com/awslabs/aws-sdk-swift/blob/main/.github/workflows/continuous-integration.yml .