Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy Auth to Fly.io PROD #172

Closed
4 tasks done
nelsonic opened this issue Dec 30, 2021 · 37 comments
Closed
4 tasks done

Deploy Auth to Fly.io PROD #172

nelsonic opened this issue Dec 30, 2021 · 37 comments
Assignees
Labels
awaiting-review An issue or pull request that needs to be reviewed chore a tedious but necessary task often paying technical debt enhancement New feature or enhancement of existing functionality priority-2 Second highest priority, should be worked on as soon as the Priority-1 issues are finished T2h Time Estimate 2 Hours technical A technical issue that requires understanding of the code, infrastructure or dependencies

Comments

@nelsonic
Copy link
Member

nelsonic commented Dec 30, 2021

Our hits (side) project https://hits.dwyl.com/ has been running on Fly.io for the past 3 months without any issues.
See: dwyl/learn-devops#76 and dwyl/hits#132

Todo

On prod we want additional security/restrictions e.g. Restricting who can see /people #171

@nelsonic nelsonic added enhancement New feature or enhancement of existing functionality priority-2 Second highest priority, should be worked on as soon as the Priority-1 issues are finished chore a tedious but necessary task often paying technical debt technical A technical issue that requires understanding of the code, infrastructure or dependencies T2h Time Estimate 2 Hours labels Dec 30, 2021
@nelsonic nelsonic self-assigned this Dec 30, 2021
@nelsonic
Copy link
Member Author

nelsonic commented Jan 2, 2022

Picking this up now. Dunno if I will finish today ... but let's see how far we can get.

@nelsonic
Copy link
Member Author

nelsonic commented Jan 2, 2022

Trying the "Speedrun" https://fly.io/docs/speedrun/
image

flyctl auth signup

Output:

WARN no config file found at /Users/n/.fly/config.yml

Attempt to launch new app:

flyctl launch

Error:

Preparing system for Elixir builds
Installing application dependencies
Running Docker release generator
Error failed running /usr/local/bin/mix phx.gen.release --docker: exit status 1

@nelsonic
Copy link
Member Author

nelsonic commented Jan 3, 2022

flyctl deploy
==> Verifying app config
--> Verified app config
==> Building image
Remote builder fly-builder-rough-river-3816 ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 20.4s (0/1)
[+] Building 2.9s (18/23)
 => [internal] load remote build context                                                0.0s
 => copy /context /                                                                     0.9s
 => [internal] load metadata for docker.io/library/alpine:3.13.3                        1.6s
 => [internal] load metadata for docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.  1.7s
 => CANCELED [app 1/5] FROM docker.io/library/alpine:3.13.3@sha256:826f70e0ac33e99a72c  0.1s
 => => resolve docker.io/library/alpine:3.13.3@sha256:826f70e0ac33e99a72cf20fb0571245a  0.0s
 => => sha256:826f70e0ac33e99a72cf20fb0571245a8fee52d68cb26d8bc58e53bf 1.64kB / 1.64kB  0.0s
 => => sha256:4266485e304a825d82c375d3584121b53c802e3540d6b520b212a9f0784d 528B / 528B  0.0s
 => => sha256:302aba9ce190db9e247d710f4794cc303b169035de2048e76b82c9ed 1.47kB / 1.47kB  0.0s
 => => sha256:9aae54b2144e5b2b00c610f8805128f4f86822e1e52d3714c463744a431f 0B / 2.81MB  0.1s
 => CANCELED [build  1/14] FROM docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.1  0.1s
 => => resolve docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.13.3@sha256:4eefe1  0.0s
 => => sha256:65927483596c7e170b1758d4924b443065128ba7e3bc2e9c6ca92633 2.37kB / 2.37kB  0.0s
 => => sha256:4eefe11eb82f277cd82ba5d792b82c0e82f7d41e3a021676b5f1eb107f57 743B / 743B  0.0s
 => => sha256:1b127a5d4cc5cacac0fc4770a4ac75e9e5d1c6fe3ddc701268bc6f31 1.16kB / 1.16kB  0.0s
 => CACHED [build  2/14] RUN apk add --no-cache build-base npm                          0.0s
 => CACHED [build  3/14] WORKDIR /app                                                   0.0s
 => CACHED [build  4/14] RUN mix local.hex --force &&     mix local.rebar --force       0.0s
 => CACHED [build  5/14] COPY mix.exs mix.lock ./                                       0.0s
 => CACHED [build  6/14] COPY config config                                             0.0s
 => CACHED [build  7/14] RUN mix deps.get --only prod &&     mix deps.compile           0.0s
 => CACHED [build  8/14] COPY priv priv                                                 0.0s
 => CACHED [build  9/14] COPY assets assets                                             0.0s
 => CACHED [build 10/14] RUN mix assets.deploy                                          0.0s
 => CACHED [build 11/14] RUN mix phx.digest                                             0.0s
 => CACHED [build 12/14] COPY lib lib                                                   0.0s
 => ERROR [build 13/14] COPY rel rel                                                    0.0s
------
 > [build 13/14] COPY rel rel:
------
Error failed to fetch an image or build from source: error building: failed to compute cache key: "/rel" not found: not found

1 similar comment
@nelsonic
Copy link
Member Author

nelsonic commented Jan 3, 2022

flyctl deploy
==> Verifying app config
--> Verified app config
==> Building image
Remote builder fly-builder-rough-river-3816 ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 20.4s (0/1)
[+] Building 2.9s (18/23)
 => [internal] load remote build context                                                0.0s
 => copy /context /                                                                     0.9s
 => [internal] load metadata for docker.io/library/alpine:3.13.3                        1.6s
 => [internal] load metadata for docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.  1.7s
 => CANCELED [app 1/5] FROM docker.io/library/alpine:3.13.3@sha256:826f70e0ac33e99a72c  0.1s
 => => resolve docker.io/library/alpine:3.13.3@sha256:826f70e0ac33e99a72cf20fb0571245a  0.0s
 => => sha256:826f70e0ac33e99a72cf20fb0571245a8fee52d68cb26d8bc58e53bf 1.64kB / 1.64kB  0.0s
 => => sha256:4266485e304a825d82c375d3584121b53c802e3540d6b520b212a9f0784d 528B / 528B  0.0s
 => => sha256:302aba9ce190db9e247d710f4794cc303b169035de2048e76b82c9ed 1.47kB / 1.47kB  0.0s
 => => sha256:9aae54b2144e5b2b00c610f8805128f4f86822e1e52d3714c463744a431f 0B / 2.81MB  0.1s
 => CANCELED [build  1/14] FROM docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.1  0.1s
 => => resolve docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.13.3@sha256:4eefe1  0.0s
 => => sha256:65927483596c7e170b1758d4924b443065128ba7e3bc2e9c6ca92633 2.37kB / 2.37kB  0.0s
 => => sha256:4eefe11eb82f277cd82ba5d792b82c0e82f7d41e3a021676b5f1eb107f57 743B / 743B  0.0s
 => => sha256:1b127a5d4cc5cacac0fc4770a4ac75e9e5d1c6fe3ddc701268bc6f31 1.16kB / 1.16kB  0.0s
 => CACHED [build  2/14] RUN apk add --no-cache build-base npm                          0.0s
 => CACHED [build  3/14] WORKDIR /app                                                   0.0s
 => CACHED [build  4/14] RUN mix local.hex --force &&     mix local.rebar --force       0.0s
 => CACHED [build  5/14] COPY mix.exs mix.lock ./                                       0.0s
 => CACHED [build  6/14] COPY config config                                             0.0s
 => CACHED [build  7/14] RUN mix deps.get --only prod &&     mix deps.compile           0.0s
 => CACHED [build  8/14] COPY priv priv                                                 0.0s
 => CACHED [build  9/14] COPY assets assets                                             0.0s
 => CACHED [build 10/14] RUN mix assets.deploy                                          0.0s
 => CACHED [build 11/14] RUN mix phx.digest                                             0.0s
 => CACHED [build 12/14] COPY lib lib                                                   0.0s
 => ERROR [build 13/14] COPY rel rel                                                    0.0s
------
 > [build 13/14] COPY rel rel:
------
Error failed to fetch an image or build from source: error building: failed to compute cache key: "/rel" not found: not found

@nelsonic
Copy link
Member Author

nelsonic commented Jan 3, 2022

 => ERROR [build 11/15] RUN mix assets.deploy                                           1.5s
------
 > [build 11/15] RUN mix assets.deploy:
#19 0.991
#19 0.991 02:31:50.361 [debug] Downloading esbuild from https://registry.npmjs.org/esbuild-linux-64/-/esbuild-linux-64-0.12.18.tgz
#19 1.407
#19 1.407   ../priv/static/assets/app.js   82.3kb
#19 1.407   ../priv/static/assets/app.css   667b
#19 1.407
#19 1.407 ⚡ Done in 16ms
#19 1.433 ** (Mix) You're trying to run :auth on Elixir v1.12.1 but it has declared in its mix.exs file it supports only Elixir ~> 1.12.3
------
Error failed to fetch an image or build from source: error building: executor failed running [/bin/sh -c mix assets.deploy]: exit code: 1

@nelsonic
Copy link
Member Author

nelsonic commented Jan 3, 2022

flyctl deploy
==> Verifying app config
--> Verified app config
==> Building image
Remote builder fly-builder-rough-river-3816 ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 158.9s (0/1)
[+] Building 77.3s (23/24)
 => [internal] load remote build context                                                0.0s
 => copy /context /                                                                     0.8s
 => [internal] load metadata for docker.io/library/alpine:3.13.3                        1.1s
 => [internal] load metadata for docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.  1.2s
 => [app 1/5] FROM docker.io/library/alpine:3.13.3@sha256:826f70e0ac33e99a72cf20fb0571  0.0s
 => [build  1/15] FROM docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.13.3@sha25  0.0s
 => CACHED [app 2/5] RUN apk add --no-cache libstdc++ openssl ncurses-libs              0.0s
 => CACHED [app 3/5] WORKDIR /app                                                       0.0s
 => CACHED [app 4/5] RUN chown nobody:nobody /app                                       0.0s
 => CACHED [build  2/15] RUN apk add --no-cache build-base npm                          0.0s
 => CACHED [build  3/15] WORKDIR /app                                                   0.0s
 => CACHED [build  4/15] RUN mix local.hex --force &&     mix local.rebar --force       0.0s
 => [build  5/15] COPY mix.exs mix.lock ./                                              0.2s
 => [build  6/15] COPY config config                                                    0.0s
 => [build  7/15] RUN mix deps.get --only prod                                          1.9s
 => [build  8/15] RUN mix deps.compile                                                 68.0s
 => [build  9/15] COPY priv priv                                                        0.0s
 => [build 10/15] COPY assets assets                                                    0.0s
 => [build 11/15] RUN mix assets.deploy                                                 2.1s
 => [build 12/15] RUN mix phx.digest                                                    1.3s
 => [build 13/15] COPY lib lib                                                          0.0s
 => [build 14/15] COPY rel rel                                                          0.0s
 => ERROR [build 15/15] RUN mix do compile, release                                     1.6s
------
 > [build 15/15] RUN mix do compile, release:
#23 1.077 Compiling 39 files (.ex)
#23 1.574
#23 1.574 == Compilation error in file lib/auth_web/router.ex ==
#23 1.574 ** (RuntimeError) No AUTH_API_KEY set, find out how at: https://git.io/JJ6sS
#23 1.574     (auth_plug 1.4.7) lib/helpers.ex:40: AuthPlug.Helpers.check_environment_vars/0
#23 1.574     (auth_plug 1.4.7) lib/auth_plug.ex:33: AuthPlug.init/1
#23 1.574     (plug 1.12.1) lib/plug/builder.ex:318: Plug.Builder.init_module_plug/4
#23 1.574     (plug 1.12.1) lib/plug/builder.ex:302: anonymous fn/5 in Plug.Builder.compile/3
#23 1.574     (elixir 1.12.1) lib/enum.ex:2356: Enum."-reduce/3-lists^foldl/2-0-"/3
#23 1.574     (plug 1.12.1) lib/plug/builder.ex:300: Plug.Builder.compile/3
#23 1.574     lib/auth_web/router.ex:35: (module)
------
Error failed to fetch an image or build from source: error building: executor failed running [/bin/sh -c mix do compile, release]: exit code: 1

More tomorrow.

@nelsonic
Copy link
Member Author

nelsonic commented Jan 3, 2022

one more try:

--> Pushing image done
image: registry.fly.io/authprod:deployment-1641177885
image size: 25 MB
==> Creating release
--> release v2 created

--> You can detach the terminal anytime without stopping the deployment
==> Release command detected: /app/bin/auth eval Auth.Release.migrate

--> This release will not be available until the release command succeeds.
	 Starting instance
	 Configuring virtual machine
	 Pulling container image
	 Unpacking image
	 Preparing kernel init
	 Configuring firecracker
	 Preparing to run: `/app/bin/auth eval Auth.Release.migrate` as nobody
	     (elixir 1.12.1) lib/application.ex:656: Application.fetch_env!/2
	     lib/auth/release.ex:11: Auth.Release.migrate/0
	     (stdlib 3.15) erl_eval.erl:685: :erl_eval.do_apply/6
	     (elixir 1.12.1) lib/code.ex:656: Code.eval_string_with_error_handling/3
	 Main child exited normally with code: 1
	 Starting clean up.
Error Release command failed, deployment aborted

Looks like we need to dig into the release migrate ... ⏳

@nelsonic
Copy link
Member Author

nelsonic commented Jan 3, 2022

$ flyctl deploy
==> Verifying app config
--> Verified app config
==> Building image
Remote builder fly-builder-rough-river-3816 ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 42.1s (0/1)
[+] Building 77.8s (25/25) FINISHED
 => [internal] load remote build context                                                0.0s
 => copy /context /                                                                     0.9s
 => [internal] load metadata for docker.io/library/alpine:3.13.3                        0.6s
 => [internal] load metadata for docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.  0.5s
 => [build  1/15] FROM docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.13.3@sha25  0.0s
 => [app 1/5] FROM docker.io/library/alpine:3.13.3@sha256:826f70e0ac33e99a72cf20fb0571  0.0s
 => CACHED [build  2/15] RUN apk add --no-cache build-base npm                          0.0s
 => CACHED [build  3/15] WORKDIR /app                                                   0.0s
 => CACHED [build  4/15] RUN mix local.hex --force &&     mix local.rebar --force       0.0s
 => CACHED [build  5/15] COPY mix.exs mix.lock ./                                       0.0s
 => CACHED [build  6/15] COPY config config                                             0.0s
 => [build  7/15] RUN mix deps.get --only prod                                          2.4s
 => [build  8/15] RUN mix deps.compile                                                 66.0s
 => [build  9/15] COPY priv priv                                                        0.0s
 => [build 10/15] COPY assets assets                                                    0.0s
 => [build 11/15] RUN mix assets.deploy                                                 2.0s
 => [build 12/15] RUN mix phx.digest                                                    1.2s
 => [build 13/15] COPY lib lib                                                          0.0s
 => [build 14/15] COPY rel rel                                                          0.0s
 => [build 15/15] RUN mix do compile, release                                           3.9s
 => CACHED [app 2/5] RUN apk add --no-cache libstdc++ openssl ncurses-libs              0.0s
 => CACHED [app 3/5] WORKDIR /app                                                       0.0s
 => CACHED [app 4/5] RUN chown nobody:nobody /app                                       0.0s
 => [app 5/5] COPY --from=build --chown=nobody:nobody /app/_build/prod/rel/auth ./      0.2s
 => exporting to image                                                                  0.2s
 => => exporting layers                                                                 0.2s
 => => writing image sha256:aae5a44ed9c0ac38b3f35fd3c4d9d017e7b4c2eebb92b8e70b399f4c5f  0.0s
 => => naming to registry.fly.io/authprod:deployment-1641179338                         0.0s
--> Building image done
==> Pushing image to fly
The push refers to repository [registry.fly.io/authprod]
6cbf8b60876e: Pushed
cd2caf626aac: Layer already exists
4b4a216473bc: Layer already exists
40ec9b5ef120: Layer already exists
0f7b3ff8b310: Layer already exists
deployment-1641179338: digest: sha256:2ab5c6e028205eadba4bd03141c34eac81bfe28d33c6f3585ef29bfd38163f5a size: 1365
--> Pushing image done
image: registry.fly.io/authprod:deployment-1641179338
image size: 25 MB
==> Creating release
--> release v2 created

--> You can detach the terminal anytime without stopping the deployment
==> Release command detected: /app/bin/auth eval Auth.Release.migrate

--> This release will not be available until the release command succeeds.
Error Release command failed, deployment aborted

unclear why it failed this time ... 🤷‍♂️

@nelsonic
Copy link
Member Author

nelsonic commented Jan 3, 2022

 flyctl deploy
==> Verifying app config
--> Verified app config
==> Building image
Remote builder fly-builder-rough-river-3816 ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 29.7s (0/1)
[+] Building 79.8s (25/25) FINISHED
 => [internal] load remote build context                                                       0.0s
 => copy /context /                                                                            0.9s
 => [internal] load metadata for docker.io/library/alpine:3.13.3                               1.1s
 => [internal] load metadata for docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.13.3     1.0s
 => [build  1/15] FROM docker.io/hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.13.3@sha256:4eefe  0.0s
 => [app 1/5] FROM docker.io/library/alpine:3.13.3@sha256:826f70e0ac33e99a72cf20fb0571245a8fe  0.0s
 => CACHED [build  2/15] RUN apk add --no-cache build-base npm                                 0.0s
 => CACHED [build  3/15] WORKDIR /app                                                          0.0s
 => CACHED [build  4/15] RUN mix local.hex --force &&     mix local.rebar --force              0.0s
 => CACHED [build  5/15] COPY mix.exs mix.lock ./                                              0.0s
 => CACHED [build  6/15] COPY config config                                                    0.0s
 => [build  7/15] RUN mix deps.get --only prod                                                 2.3s
 => [build  8/15] RUN mix deps.compile                                                        67.3s
 => [build  9/15] COPY priv priv                                                               0.0s
 => [build 10/15] COPY assets assets                                                           0.0s
 => [build 11/15] RUN mix assets.deploy                                                        2.0s
 => [build 12/15] RUN mix phx.digest                                                           1.2s
 => [build 13/15] COPY lib lib                                                                 0.0s
 => [build 14/15] COPY rel rel                                                                 0.0s
 => [build 15/15] RUN mix do compile, release                                                  4.1s
 => CACHED [app 2/5] RUN apk add --no-cache libstdc++ openssl ncurses-libs                     0.0s
 => CACHED [app 3/5] WORKDIR /app                                                              0.0s
 => CACHED [app 4/5] RUN chown nobody:nobody /app                                              0.0s
 => [app 5/5] COPY --from=build --chown=nobody:nobody /app/_build/prod/rel/auth ./             0.2s
 => exporting to image                                                                         0.2s
 => => exporting layers                                                                        0.2s
 => => writing image sha256:82ab0c35e45795d7e3eb7b1e69af9fda70b847f299f26ed37551bbed90038712   0.0s
 => => naming to registry.fly.io/authprod:deployment-1641181864                                0.0s
--> Building image done
==> Pushing image to fly
The push refers to repository [registry.fly.io/authprod]
406ebe2b07e7: Pushed
cd2caf626aac: Layer already exists
4b4a216473bc: Layer already exists
40ec9b5ef120: Layer already exists
0f7b3ff8b310: Layer already exists
deployment-1641181864: digest: sha256:b46f40202a23bdad0d1c83c353096beb4b74af73cfedcac3f22e073a13c6c8e8 size: 1365
--> Pushing image done
image: registry.fly.io/authprod:deployment-1641181864
image size: 25 MB
==> Creating release
--> release v2 created

--> You can detach the terminal anytime without stopping the deployment
==> Release command detected: /app/bin/auth eval Auth.Release.migrate

--> This release will not be available until the release command succeeds.
	 Starting instance
	 Configuring virtual machine
	 Pulling container image
	 Preparing kernel init
	 Starting virtual machine
	 Starting init (commit: 7943db6)...
	 ** (EXIT from #PID<0.93.0>) an exception was raised:
	 you can add it to `extra_applications` in your `mix.exs`:
	   def application do
	   end
	         (postgrex 0.15.13) lib/postgrex.ex:632: Postgrex.ensure_deps_started!/1
	         (stdlib 3.15) gen_server.erl:390: :gen_server.init_it/6
	         (stdlib 3.15) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
	 Rain child exited normally with code: 1
	 Staped child process with pid: 561 and signal: SIGUSR1, core dumped? false
	 Starting clean up.
Error Release command failed, deployment aborted

@nelsonic
Copy link
Member Author

nelsonic commented Feb 2, 2022

Thanks @SimonLab added that line of config last night:
7ab46b5#diff-fcb6e9bf2546587075d6850070c1b4d26f061f0c199e6ea3ac4d8fd76d9fa8f6

Rather than continuing to bash my head against the brick wall with trying to get the init function to run in the release.ex,
I'm going to create a /status page on the auth app that will let the person deploying auth check the status of their deployment. See: #174 (comment)

@nelsonic
Copy link
Member Author

nelsonic commented Feb 7, 2022

PR #173 contains all these enhancements. :shipit:

https://auth.dwyl.com/ works as expected and it's fast!

image

@nelsonic nelsonic assigned SimonLab and unassigned nelsonic Feb 7, 2022
@nelsonic nelsonic added the awaiting-review An issue or pull request that needs to be reviewed label Feb 7, 2022
nelsonic added a commit that referenced this issue Feb 7, 2022
SimonLab added a commit that referenced this issue Feb 8, 2022
@nelsonic
Copy link
Member Author

This is working on Fly.io: https://auth.dwyl.com 🚀
image

Closing. ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting-review An issue or pull request that needs to be reviewed chore a tedious but necessary task often paying technical debt enhancement New feature or enhancement of existing functionality priority-2 Second highest priority, should be worked on as soon as the Priority-1 issues are finished T2h Time Estimate 2 Hours technical A technical issue that requires understanding of the code, infrastructure or dependencies
Projects
None yet
Development

No branches or pull requests

3 participants