Features: dynamic install vs bake into shared image

[kf6kjg]

So the current design of features in the devcontainer ecosystem is that they are tacked onto the base Docker container after the container is instantiated - what I'm calling "dynamic install" for lack of a better term. For the teams I work with this has provided a problem: several of the features we need take a very long time to install, making a devcontainer very slow to build - the first time and any time after that.

Instead I created a workaround that speeds up the process at the sacrifice of getting the latest versions every rebuild:

1. The devcontainer.json file references either a central image built as described below, or a local Dockerfile or docker-compose.yaml with the same architecture. The former provides the fastest process, but the latter provides local customizability.
2. The Dockerfile is based off of a devcontainer image that provides the majority of the needed tools, and then bakes the features into itself as layers:

ARG VARIANT=18-bullseye

FROM mcr.microsoft.com/vscode/devcontainers/javascript-node:0-${VARIANT}
ARG AZURE_CORE_TOOLS_VERSION=4
ARG TERRAFORM_VERSION=1.3
ARG TERRAGRUNT_VERSION=none
ARG TFLINT_VERSION=0.46.1

RUN true \
    && mkdir -p /tmp/azure-cli; \
    cd /tmp/azure-cli \
    && curl -sfL https://raw.githubusercontent.com/devcontainers/features/main/src/azure-cli/install.sh -o install.sh 2>/dev/null \
    && chmod +x install.sh \
    && ./install.sh \
    && cd / \
    && rm -rf /tmp/azure-cli \
    \
    && rm -rf /var/lib/apt/lists/* \
    ;
RUN true \
    && mkdir -p /tmp/terraform; \
    cd /tmp/terraform \
    && curl -sfL https://raw.githubusercontent.com/devcontainers/features/main/src/terraform/install.sh -o install.sh 2>/dev/null \
    && chmod +x install.sh \
    && VERSION=${TERRAFORM_VERSION} TERRAGRUNT=${TERRAGRUNT_VERSION} TFLINT=${TFLINT_VERSION} ./install.sh \
    && cd / \
    && rm -rf /tmp/terraform \
    \
    && rm -rf /var/lib/apt/lists/* \
    ;

# etc...

The sacrifice can have its benefit: a company can maintain a centrally defined well-vetted suite of devcontainer images that each individual repository just taps into. This provides opportunity for security checks as well as helping make sure that all developers, CI/CD systems, etc. are all utilizing the same tooling.

However this has had some problems of late. Recently the user env vars spec proposal was made REMOTE_USER aware causing my setup above to totally fail.

I get that I'm using the tooling in an unexpected manner. I'd like to open up discussion about how we could make baking features into the docker images easier to accomplish and maintain - I don't expect to be able to speed up the installs of these features, they are simply massive.

[kf6kjg]

Why rebuild? I may have executed a command that destroyed the usefulness of the container. Rebuilding is fast when all I have to do is remake it from the already cached Docker layers. Slow if I have to do that AND download and install features from the internet.

[bamurtaugh]

Thanks for sharing this!

@craiglpeters @joshspicer I'd love to get your thoughts here, as this feels somewhat related to prebuild and Feature installation discussions we've had.

[kf6kjg]

Heard anything back? I'm still very interested in this and am still using my workaround.

[samruddhikhandale]

Hi 👋

The Feature scripts are installed immediately after the commands from the Dockerfile are executed (or after the image is pulled). We say that a container is ready post completion of both these things. Also, the Feature scripts are run with Docker's Run commands, and the docker image is ready only after the completion of Dokerfile cmds and Feature installations.

So the current design of features in the devcontainer ecosystem is that they are tacked onto the base Docker container after the container is instantiated

Hence, I believe that ^ statement is incorrect. Feel free to correct me if I have misunderstood your question/scenario.

The devcontainer.json file references either a central image built as described below,

Why do you need to run the scripts within the Dockerfile, adding them to the Features object has the same outcome. See https://github.com/devcontainers/images which builds and publishes images using Dockerfiles and Features.

[kf6kjg]

Your sequence reads correctly, but a detail is omitted. The Dockerfile is compiled into layers packed together as an image. Once the image is done then a container is instantiated from it, launched, and the Features get installed. Once those are done, then the container is ready as you said.

Now for my detail. I want the features to be baked into the image as layers. I did this via the demonstrated hacks, but I'd like to discuss if there's a nicer way of supporting the pattern I've demonstrated.

The result? I can push the resulting image to a registry and all company users now are downloading vetted known versions of even the Features. This can't be done with the system as-is. If needed I can explain further.