Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Re-enable Terraform Tests #4564

Open
ellismg opened this issue Nov 15, 2024 · 1 comment
Open

ci: Re-enable Terraform Tests #4564

ellismg opened this issue Nov 15, 2024 · 1 comment
Assignees
Milestone

Comments

@ellismg
Copy link
Member

ellismg commented Nov 15, 2024

Our terraform tests in CI are currently disabled (in the case of the inner-loop build-cli leg) or failing (in the case of template tests) since moving to the TME environment since we need to use OIDC to connect.

This bug tracks re-enabling these disabled tests.

@ellismg ellismg added this to the Nov 2024 milestone Nov 15, 2024
@ellismg ellismg self-assigned this Nov 15, 2024
ellismg added a commit to ellismg/azure-dev that referenced this issue Nov 15, 2024
We should be able to re-enable the tests now. The one exception is the
`azd login` test that validated login with a long lived client secret,
since we no longer have a service principal with a long lived client
secret.

We'll have to make due without the end to end coverage here - but in
practice we have good coverage at the unit test level of much of the
code here, so we should be okay.

Contributes To Azure#4564
@ellismg
Copy link
Member Author

ellismg commented Nov 15, 2024

Trying to re-enable the remote state terraform tests failed with this error coming back from TF:

=== FAIL: cli/azd/test/functional Test_CLI_InfraCreateAndDeleteResourceTerraformRemote (113.42s)
    cli_test.go:833: DIR: /tmp/Test_CLI_InfraCreateAndDeleteResourceTerraformRemote560961891/001
    cli_test.go:841: AZURE_ENV_NAME: azdtest-lb61dc9
2024/11/15 19:05:09 Run exec: 'az group create --name rs-azdtest-lb61dc9 --location eastus2' , exit code: 0
2024/11/15 19:05:34 Run exec: 'az storage account create --resource-group rs-azdtest-lb61dc9 --name azdtestlb61dc9 --sku Standard_LRS --encryption-services blob' , exit code: 0
2024/11/15 19:05:36 Run exec: 'az storage account keys list --resource-group rs-azdtest-lb61dc9 --account-name azdtestlb61dc9 --query [0].value -o tsv' , exit code: 0
2024/11/15 19:05:37 Run exec: 'az storage container create --name tfstate --account-name azdtestlb61dc9 --account-key <redacted>' , exit code: 0
    cli.go:241: 17ms [stdout] 
    cli.go:241: 19ms [stdout] Initializing an app to run on Azure (azd init)
    cli.go:241: 19ms [stdout] 
    cli.go:241: 37ms [stdout] Enter a new environment name: 
    cli.go:241: 42ms [stdout] SUCCESS: Initialized environment azdtest-lb61dc9.
    cli_test.go:903: Starting infra create
    cli.go:241: 20ms [stdout] 
    cli.go:241: 24ms [stdout] Provisioning Azure resources (azd provision)
    cli.go:241: 27ms [stdout] Provisioning Azure resources can take some time.
    cli.go:241: 27ms [stdout] 
    cli.go:241: 1.489s [stdout] Select an Azure Subscription to use (or hit enter to use the default  1. Azure SDK Test Resources - TME (4d042dc6-fe17-4698-a23f-ec6a8d1e98f4)): Reading subscription and location from environment...
    cli.go:241: 4.676s [stdout] Subscription: Azure SDK Test Resources - TME (4d042dc6-fe17-4698-a23f-ec6a8d1e98f4)
    cli.go:241: 4.677s [stdout] Location: East US 2
    cli.go:241: 4.677s [stdout] 
    cli.go:241: 4.677s [stdout] Locating plan file...
    cli.go:241: 4.678s [stdout] Generating terraform backend config file...
    cli.go:241: 5.475s [stdout] Initializing the backend...
    cli.go:241: 6.184s [stderr] ╷
    cli.go:241: 6.197s [stderr] │ Error: Error building ARM Config: Authenticating using the Azure CLI is only supported as a User (not a Service Principal).
    cli.go:241: 6.198s [stderr] │ 
    cli.go:241: 6.201s [stdout] 
    cli.go:241: 6.207s [stdout] ERROR: deployment failed: error deploying infrastructure: terraform init failed:  , err: failed running terraform init:  (exit code: 1)
    cli.go:241: 6.214s [stderr] │ To authenticate to Azure using a Service Principal, you can use the separate 'Authenticate using a Service Principal'
    cli.go:241: 6.214s [stdout] TraceID: 957c3d18ecfa564f6d182e0f475dae9d
    cli.go:241: 6.222s [stderr] │ auth method - instructions for which can be found here: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret
    cli.go:241: 6.223s [stderr] │ 
    cli.go:241: 6.228s [stderr] │ Alternatively you can authenticate using the Azure CLI by using a User Account.
    cli.go:241: 6.229s [stderr] │ 
    cli.go:241: 6.23s [stderr] │ 
    cli.go:241: 6.231s [stderr] ╵
    cli_test.go:905: 
        	Error Trace:	/mnt/vss/_work/1/s/cli/azd/test/functional/cli_test.go:905
        	Error:      	Received unexpected error:
        	            	command 'azd provision --cwd /tmp/Test_CLI_InfraCreateAndDeleteResourceTerraformRemote560961891/001 in /tmp/Test_CLI_InfraCreateAndDeleteResourceTerraformRemote560961891/001' had non-zero exit code: exit status 1
        	Test:       	Test_CLI_InfraCreateAndDeleteResourceTerraformRemote
2024/11/15 19:07:01 Run exec: 'az group delete --name rs-azdtest-lb61dc9 --yes' , exit code: 0

I think this is an auth problem with the remote state provider, not the azure RM provider, since the other terraform test passed without issue. Keeping this skipped against the bug for now.

ellismg added a commit to ellismg/azure-dev that referenced this issue Nov 19, 2024
We should be able to re-enable the tests now. The one exception is the
`azd login` test that validated login with a long lived client secret,
since we no longer have a service principal with a long lived client
secret.

We'll have to make due without the end to end coverage here - but in
practice we have good coverage at the unit test level of much of the
code here, so we should be okay.

Contributes To Azure#4564
ellismg added a commit that referenced this issue Nov 21, 2024
We should be able to re-enable the tests now. The one exception is the
`azd login` test that validated login with a long lived client secret,
since we no longer have a service principal with a long lived client
secret.

We'll have to make due without the end to end coverage here - but in
practice we have good coverage at the unit test level of much of the
code here, so we should be okay.

Contributes To #4564
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant