Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot schedule pod unrelated to akv2k8s - certificate signed by unknown authority #709

Open
lyubomirk opened this issue May 7, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@lyubomirk
Copy link

Note: Make sure to check out known issues (https://akv2k8s.io/troubleshooting/known-issues/) before submitting

Components and versions
Select which component(s) the bug relates to with [X].

[ ] Controller, version: x.x.x (docker image tag)
[x] Env-Injector (webhook), version: 1.6.0 (docker image tag)
[ ] Other

Describe the bug
I have an AKS cluster and a kubernetes namespace, with the label azure-key-vault-env-injection: enabled. I use env injection for only one of my workloads. The others don't reference akv2k8s at all. Still, the ones that are NOT referencing keyvault secrets sometimes fail to schedule with the following error message:

Error creating: Internal error occurred: failed calling webhook "pods.env-injector.admission.spv.no": failed to call webhook: Post "https://akv2k8s-envinjector.akv2k8s.svc:443/pods?timeout=10s": tls: failed to verify certificate: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "svc-cat-ca")

If I restart the injector pods it sometimes solves the issue temporarily, but at some point, I get the above error again.

To Reproduce
Steps to reproduce the behavior:

  1. Install akv2k8s using the latest helm chart (2.6.0) and the default values file.
  2. Add label azure-key-vault-env-injection: enabled to desired namespace
  3. Schedule a pod to the namespace using a deployment (none of the workloads should reference any of the akv2k8s resources)

Expected behavior
The pod should schedule without issues

Additional context
I haven't seen this issue in the older versions of the helm chart (chart version 2.1.0 - Image versions 1.3.0)

@lyubomirk lyubomirk added the bug Something isn't working label May 7, 2024
@lyubomirk
Copy link
Author

lyubomirk commented May 10, 2024

The issue appeared again. I receive the following error message in the envinjector:
image

When I look up the IP, it is Azure's konnectiviy-agent. Restarting the pods does not help this time.
There are events that the sync of the secrets is successful, but the pod that uses the secret does not get admitted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant