Skip to content

Commit

Permalink
chore: deploy test resources w/ new test packages (#433)
Browse files Browse the repository at this point in the history
* chore: deploy test resources

* pr-fix: missing assignment keyword

* pr-fix: correct rg assignment

* pr-fix: key vault name assignment

* pr-fix: add location param + correct existing service principal condition

* pr-fix: add depends on

* pr-fix: use correct way to retrieve secret version

* pr-fix: add logging for sp-related commands

* pr-fix: use solely security-related resources

* pr-fix: add logging for secret version

* pr-fix: add secret version other way

* pr-fix: use property outputs

* pr-fix: add logging for deploy outputs

* pr-fix: correct new output variable assignment

* pr-fix: mark as string

* pr-add: integrate tests and key vault

* pr-fix: hashicorp template path

* pr-fix: add devops resources

* pr-fix: add test variables

* pr-fix: az identity vulnerability

* pr-fix: correct secret value

* pr-fix: run secret retrieval as a pre-job

* pr-fix: get own key vault secrets

* pr-fix: correct parameters

* pr-fix: install module az.keyvault

* pr-fix: use az keyvault

* pr-fix: use client id as var

* pr-fix: remove app insights reference

* pr-fix: add infra smoke tests

* pr-fix: use az cli task

* pr-fix: argument syntax

* pr-fix: add enabled assertion

* pr-fix: correct running

* pr-fix: add az module

* pr-fix: use new arguments syntax

* pr-fix: use higher version of pester

* pr-fix: use at least 5.3.0

* pr-fix: use env variables

* pr-fix: correct test result

* pr-fix: import module

* pr-fix: remove param

* pr-fix: enable test result

* pr-fix: correct env vars

* pr-fix: use other env vars

* pr-fix: use correct secret version extraction

* pr-fix: use pester container for external data

* pr-fix: broaden test assertion + trim secret version setup

* pr-fix: simplify config value retrieval

* pr-fix: remove any spaces from version and secret

* pr-fix: use direct setting of variable

* pr-fix: clean tests

* pr-fix: remove tried smoke tests

* Update Arcus.Security.Providers.AzureKeyVault.csproj

* Update Arcus.Security.Providers.AzureKeyVault.csproj

* pr-fix: use most recent test fixtures

* pr-fix: use correct unauthorized secret names

* pr-fix: remove remote resource group

* Update deploy-test-resources.yml

* Update deploy-test-resources.yml

* Update deploy-test-resources.yml
  • Loading branch information
stijnmoreels authored Aug 26, 2024
1 parent f5e02f2 commit e0b4f97
Show file tree
Hide file tree
Showing 26 changed files with 433 additions and 513 deletions.
13 changes: 5 additions & 8 deletions build/ci-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ parameters:
- name: 'Package.Version.ManualTrigger'
type: string
default: 'preview'
- name: azureServiceConnection
displayName: 'Azure service connection'
type: string
default: 'Azure Codit-Arcus Service Principal'

resources:
repositories:
Expand All @@ -27,9 +31,6 @@ resources:
endpoint: arcus-azure

variables:
# 'Arcus_ServicePrincipal_AccessKey' is added as secret on build in Azure DevOps
- group: 'Arcus Security - Integration Testing'
- group: 'Arcus - GitHub Package Registry'
- group: 'Build Configuration'
- template: ./variables/build.yml
- template: ./variables/test.yml
Expand Down Expand Up @@ -106,14 +107,10 @@ stages:
inputs:
artifact: 'Build'
path: '$(Build.SourcesDirectory)'
- template: 'templates/download-hashicorp-vault.yml'
parameters:
targetFolder: '$(Build.SourcesDirectory)'
version: $(HashiCorp.Vault.Version)
vaultBinVariableName: 'Arcus.HashiCorp.VaultBin'
- template: templates/run-integration-tests.yml
parameters:
dockerProjectName: '$(Project).Tests.Runtimes.AzureFunctions'
azureServiceConnection: '${{ parameters.azureServiceConnection }}'

- stage: ReleaseToMyget
displayName: 'Release to MyGet'
Expand Down
54 changes: 54 additions & 0 deletions build/deploy-test-resources.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Arcus Security - Deploy test resources

trigger: none
pr: none

parameters:
- name: azureServiceConnection
displayName: 'Azure service connection'
type: string
default: 'Azure Codit-Arcus Service Principal'
- name: resourceGroupName
displayName: 'Resource group name'
default: arcus-security-dev-we-rg

variables:
- template: ./variables/build.yml
- template: ./variables/test.yml

resources:
repositories:
- repository: templates
type: github
name: arcus-azure/azure-devops-templates
endpoint: arcus-azure

stages:
- stage: Deploy
jobs:
- job: DeployBicep
displayName: 'Deploy test resources'
pool:
vmImage: '$(Vm.Image)'
steps:
- task: AzureCLI@2
inputs:
azureSubscription: '${{ parameters.azureServiceConnection }}'
addSpnToEnvironment: true
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
$secretName = $env:ARCUS_SECURITY_KEYVAULT_TESTSECRETNAME
$secretValue = [System.Guid]::NewGuid().ToString()
$objectId = (az ad sp show --id $env:servicePrincipalId | ConvertFrom-Json).id
az deployment sub create `
--location westeurope `
--template-file ./build/templates/deploy-test-resources.bicep `
--parameters location=westeurope `
--parameters resourceGroupName=${{ parameters.resourceGroupName }} `
--parameters keyVaultName=$env:ARCUS_SECURITY_KEYVAULT_NAME `
--parameters secretName=$secretName `
--parameters secretValue=$secretValue `
--parameters servicePrincipal_objectId=$objectId `
| ConvertFrom-Json
7 changes: 0 additions & 7 deletions build/nuget-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,6 @@ resources:
endpoint: arcus-azure

variables:
- group: 'Arcus Security - Integration Testing'
- group: 'Arcus - GitHub Package Registry'
- group: 'Build Configuration'
- template: ./variables/build.yml
- template: ./variables/test.yml
Expand Down Expand Up @@ -92,11 +90,6 @@ stages:
inputs:
artifact: 'Build'
path: '$(Build.SourcesDirectory)'
- template: 'templates/download-hashicorp-vault.yml'
parameters:
targetFolder: '$(Build.SourcesDirectory)'
version: $(HashiCorp.Vault.Version)
vaultBinVariableName: 'Arcus.HashiCorp.VaultBin'
- template: templates/run-integration-tests.yml
parameters:
dockerProjectName: '$(Project).Tests.Runtimes.AzureFunctions'
Expand Down
56 changes: 56 additions & 0 deletions build/templates/deploy-test-resources.bicep
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// Define the location for the deployment of the components.
param location string

// Define the name of the resource group where the components will be deployed.
param resourceGroupName string

// Define the name of the Key vault.
param keyVaultName string

// Define the name of the secret that will be added to the Key vault.
param secretName string

// Define the secret value that will be by default added to the Key vault.
@secure()
param secretValue string

// Define the Service Principal ID that needs access full access to the deployed resource group.
param servicePrincipal_objectId string

targetScope='subscription'

module resourceGroup 'br/public:avm/res/resources/resource-group:0.2.3' = {
name: 'resourceGroupDeployment'
params: {
name: resourceGroupName
location: location
}
}

resource rg 'Microsoft.Resources/resourceGroups@2021-04-01' existing = {
name: resourceGroupName
}

module vault 'br/public:avm/res/key-vault/vault:0.6.1' = {
name: 'vaultDeployment'
dependsOn: [
resourceGroup
]
scope: rg
params: {
name: keyVaultName
location: location
roleAssignments: [
{
principalId: servicePrincipal_objectId
roleDefinitionIdOrName: 'Key Vault Secrets officer'
}
]
secrets: [
{
name: secretName
value: secretValue
}
]
}
}
33 changes: 33 additions & 0 deletions build/templates/run-integration-tests.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
parameters:
dockerProjectName: ''
azureServiceConnection: ''

steps:
- bash: |
Expand All @@ -9,6 +10,38 @@ steps:
fi
env:
PROJECT_NAME: ${{ parameters.dockerProjectName }}
- task: AzureCLI@2
displayName: 'Import secrets from Azure Key Vault'
inputs:
azureSubscription: '${{ parameters.azureServiceConnection }}'
addSpnToEnvironment: true
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
Set-PSRepository -Name PSGallery -InstallationPolicy Trusted
Install-Module -Name Arcus.Scripting.DevOps -AllowClobber
Set-AzDevOpsVariable 'Arcus.Security.TenantId' -Value $env:tenantId -AsSecret
Set-AzDevOpsVariable 'Arcus.Security.ServicePrincipal.ClientId' -Value $env:servicePrincipalId -AsSecret
Set-AzDevOpsVariable 'Arcus.Security.ServicePrincipal.ClientSecret' -Value $env:servicePrincipalKey -AsSecret
$unauthorizedClientId = az keyvault secret show --name $env:ARCUS_GENERAL_UNAUTHORIZED_SERVICEPRINCIPAL_CLIENTID_SECRETNAME --vault-name $env:ARCUS_GENERAL_KEYVAULT_NAME | ConvertFrom-Json
$unauthorizedClientSecret = az keyvault secret show --name $env:ARCUS_GENERAL_UNAUTHORIZED_SERVICEPRINCIPAL_CLIENTSECRET_SECRETNAME --vault-name $env:ARCUS_GENERAL_KEYVAULT_NAME | ConvertFrom-Json
Set-AzDevOpsVariable 'Arcus.Security.Unauthorized.ServicePrincipal.ClientId' -Value $unauthorizedClientId.value -AsSecret
Set-AzDevOpsVariable 'Arcus.Security.Unauthorized.ServicePrincipal.ClientSecret' -Value $unauthorizedClientSecret.value -AsSecret
$testSecret = az keyvault secret show --name $env:ARCUS_SECURITY_KEYVAULT_TESTSECRETNAME --vault-name $env:ARCUS_SECURITY_KEYVAULT_NAME | ConvertFrom-Json
$testSecretVersion = $testSecret.id.Split('/') | Select-Object -Last 1
Write-Host "Test secret '$($testSecret.name)' version is '$testSecretVersion'"
$testSecretValue = $testSecret.value
Set-AzDevOpsVariable -AsSecret -Name 'Arcus.Security.KeyVault.TestSecretValue' -Value $testSecretValue
Set-AzDevOpsVariable -AsSecret -Name 'Arcus.Security.KeyVault.TestSecretVersion' -Value $testSecretVersion
- template: 'download-hashicorp-vault.yml'
parameters:
targetFolder: '$(Build.SourcesDirectory)'
version: $(HashiCorp.Vault.Version)
vaultBinVariableName: 'Arcus.HashiCorp.VaultBin'
- task: UseDotNet@2
displayName: 'Import .NET Core SDK ($(DotNet.Sdk.VersionBC))'
inputs:
Expand Down
11 changes: 7 additions & 4 deletions build/variables/test.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
variables:
Arcus.KeyVault.TestKeyName: "ArcusTestSecret"
Arcus.KeyVault.TestKeyVersion: "8bde7a16366849e28b7abe26732e12e3"
HashiCorp.Vault.Version: 1.5.0
Arcus.AzureFunctions.HttpPort: "5000"
Arcus.Security.KeyVault.Name: 'arcus-security-kv'
Arcus.Security.KeyVault.TestSecretName: 'ArcusTestSecret'
Arcus.General.KeyVault.Name: 'arcus-kv'
Arcus.General.Unauthorized.ServicePrincipal.ClientId.SecretName: 'Arcus-Unauthorized-ServicePrincipal-ClientId'
Arcus.General.Unauthorized.ServicePrincipal.ClientSecret.SecretName: 'Arcus-Unauthorized-ServicePrincipal-ClientSecret'
Arcus.AzureFunctions.HttpPort: '5000'
HashiCorp.Vault.Version: 1.5.0
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
</ItemGroup>

<ItemGroup>
<PackageReference Include="Azure.Identity" Version="1.10.2" />
<PackageReference Include="Azure.Identity" Version="1.11.4" />
<PackageReference Include="Azure.Security.KeyVault.Secrets" Version="4.5.0" />
<PackageReference Include="Microsoft.Extensions.Configuration" Version="6.0.0" />
<PackageReference Include="Polly" Version="7.2.1" />
Expand All @@ -34,4 +34,4 @@
<ProjectReference Include="..\Arcus.Security.Core\Arcus.Security.Core.csproj" />
</ItemGroup>

</Project>
</Project>
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>netstandard2.1</TargetFramework>
<TargetFrameworks>net6.0;net8.0</TargetFrameworks>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Guard.Net" Version="1.2.0" />
<PackageReference Include="Arcus.Testing.Logging" Version="0.5.0" />
<PackageReference Include="Arcus.Testing.Security.Providers.InMemory" Version="0.5.0" />
<PackageReference Include="Guard.Net" Version="3.0.0" />
<PackageReference Include="Arcus.Testing.Logging.Xunit" Version="1.0.3" />
<PackageReference Include="Arcus.Testing.Security.Providers.InMemory" Version="1.0.3" />
<PackageReference Include="Serilog" Version="2.10.0" />
</ItemGroup>

Expand Down
45 changes: 0 additions & 45 deletions src/Arcus.Security.Tests.Core/Stubs/SpyLogger.cs

This file was deleted.

36 changes: 0 additions & 36 deletions src/Arcus.Security.Tests.Core/Stubs/TestLoggerProvider.cs

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

<ItemGroup>
<FrameworkReference Include="Microsoft.AspNetCore.App" />
<PackageReference Include="Arcus.Testing.Core" Version="1.0.3" />
<PackageReference Include="Arcus.Observability.Telemetry.Serilog.Sinks.ApplicationInsights" Version="3.0.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.7.1" />
<PackageReference Include="Serilog.Extensions.Hosting" Version="3.1.0" />
Expand Down
Loading

0 comments on commit e0b4f97

Please sign in to comment.