Fail with a .finished event with proper message and status if launching the container fails

[christian-kreuzberger-dtx]

I have a docker container that fails to start, where the following log/error message shows up in Kubernetes events:

  Warning  Failed     4m8s   kubelet            Error: failed to create containerd task: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "/entrypoint.sh": permission denied: unknown

This is due to me having forgotten to do a chmod +x entrypoint.sh, which is quite a common problem with Docker containers.

Another problem could be that your configuration points to a non-executable or non-existing file, e.g.:

    tasks:
      - name: "foobar files"
        files:
          - /
        image: "alpine/helm:3.7.2"
        cmd:
          - "bash"

Note: bash is not a part of alpine ;)

While this is obviously a user error, it's super hard to troubleshoot right now.

In Keptn Bridge we only get this message:

11:31 job-executor-service finished with result fail

Job job-executor-service-job-855e19ed-baed-48b3-973e-0e62-1 failed: job job-executor-service-job-855e19ed-baed-48b3-973e-0e62-1 failed. Reason: BackoffLimitExceeded, Message: Job has reached the specified backoff limit

Definition of Done

• Proper .finished event with error message is created
• Keptn Bridge shows the proper error message, e.g., could not start container docker.io/myorg/mycontainer

[pchila]

On a technical note: accessing the init container logs (in case of failures at init stage) requires a container name (otherwise we risk skipping some logs)

If there is an error during init phase (can be checked via the status of the pod) it would be necessary to access the logs of the specific init-container (name can be retrieved from code or (even better) from the PodSpec itself.

Refs:
https://kubernetes.io/docs/tasks/debug-application-cluster/debug-init-containers/#accessing-logs-from-init-containers
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#read-log-pod-v1-core