Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup Fly.io #184

Merged
merged 10 commits into from
Jun 21, 2024
Merged

Setup Fly.io #184

merged 10 commits into from
Jun 21, 2024

Conversation

codestronger
Copy link
Contributor

@codestronger codestronger commented Jun 3, 2024

Also posted a message on Teams that gives some context re: this PR, so probably helpful to check that out before diving into this code.

codestronger and others added 4 commits May 28, 2024 17:28
Add operations guide and reference.
Note that the latest google-java-format is 1.22.0. Left it at 1.16.0 as the update would require minor formatting changes to many existing files, but something to keep in mind.
Copy link
Member

@nonprofittechy nonprofittechy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just starting the review, but I really appreciate the detailed operations manual, decision log, etc. this is great, and thank you

Copy link
Collaborator

@BryceStevenWilley BryceStevenWilley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review in progress; I've read everything that's not the operational model. I'll get to that soon.

Everything else looks good so far though!

fly.yml.example Outdated Show resolved Hide resolved
pom.xml Show resolved Hide resolved

## Decision

Integrate Papertrail through the existing slf4j/logback stack as a log appender option, using TLS-encrypted syslog over TCP. If the Papertrail environment variables are set (see env.example for details), the logs will also be forwarded to Papertrail. Papertrail will serve as a central point of access for viewing the logs. The Papertrail UI makes it easy to filter and search through the logs. Being able to control log access independent of access to more admin operations within Fly.io makes for better security. Separating out the logging concern allows you to ship logs from other environments besides Fly.io as well. For example, the logs from Lightsail instances can be fed into the same dashboard. The current staging and production Fly.io instances will all forward to a single Papertrail dashboard and identified by the system and configurable application name.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all seems great. @nonprofittechy , you might want to make sure that a docassemble integration with papertrail wouldn't be too hard, so eventually all of the logs could reside in the same place.

Rename DISABLE_QUARTZ_SCHEDULE_FOR_CODE_UPDATE to DISABLE_SCHEDULE_FOR_CODE_UPDATE.
Treat immediate and daily Tyler EFM code update schedules as non-exclusive.
…fault. Wonder if we should auto re-formatting?
docs/operations_guide/index.md Outdated Show resolved Hide resolved
docs/operations_guide/index.md Outdated Show resolved Hide resolved
docs/operations_guide/index.md Outdated Show resolved Hide resolved
Noted that CourtPolicy cached in memory and would be need to updated if scaling out.
Added link to Tyler.
Add section in the docs about the psql settings page in Supabase. This is useful for restoring a Postgres backup from the Docker postgres.
Copy link
Collaborator

@BryceStevenWilley BryceStevenWilley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry it's taken so long to approve this, but finally got through the ops guide. It all looks very good!

I know we ran into some issues trying it out, the fixes for those issues are still in progress in #185. I'll try to fix that so CI can pass before this merges. Is there a last date you can work on this @codestronger? I can make sure I get that in with enough time for you to finish up.

fly_create_api_key.sh Outdated Show resolved Hide resolved
fly_create_prod_api_key.sh Outdated Show resolved Hide resolved
docs/operations_guide/index.md Outdated Show resolved Hide resolved
docs/operations_guide/index.md Outdated Show resolved Hide resolved
docs/operations_guide/index.md Outdated Show resolved Hide resolved
docs/operations_guide/index.md Outdated Show resolved Hide resolved
docs/operations_guide/index.md Outdated Show resolved Hide resolved
docs/operations_guide/index.md Outdated Show resolved Hide resolved
fly deploy -c fly.production.toml
```

One thing to keep in mind is that the manual deployment builds the Docker image from the contents of your local filesystem. This is crucial for .pfx certificate files, as they are git-ignored but not docker-ignored to allow for the normal local testing. The automatic continuous deployment is a bit safer if you do not want to bake certs into your docker image since that will build from the contents in GitHub, which already has gone through the git ignore filtering step.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something I'm curious about is how the docker images are stored / transferred to fly. The current system is definitely set up as is because the images never leave the prod machine, so baking in certs isn't exactly a security issue. Does fly save previous built versions of the images? Are they accessible by anyone, or just us?

There might be some finagling that we could do to have a docker image build for fly without certs, even if it's built locally. Likely a later issue though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should rewrite this to tone down the language. It's a bit alarmist upon re-reading it. Fly.io stores the docker images in their own registry, that's scoped to our app. There's a layer of auth there, although it's not on the same level as the typical secrets manager. We had a discussion about this topic a while back and the decision was to err on the side of caution. However, anyone with access to the registry would likely also have access to the machines, so it's not that big a deal to bake in secrets even though that's not a best practice.

@codestronger
Copy link
Contributor Author

Sorry it's taken so long to approve this, but finally got through the ops guide. It all looks very good!

I know we ran into some issues trying it out, the fixes for those issues are still in progress in #185. I'll try to fix that so CI can pass before this merges. Is there a last date you can work on this @codestronger? I can make sure I get that in with enough time for you to finish up.

I think the official wrap is June 21, but I can continue to work with you on it after that date. I'd like take the time to get things right and not rush it. Thanks again for all the help and guidance!

Enabled continuous deployment to efsp-staging from GitHub.
Copy link

@samglover samglover left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll defer to @BryceStevenWilley about whether this is ready to merge.

@codestronger codestronger merged commit a55f80a into main Jun 21, 2024
3 checks passed
@codestronger codestronger deleted the setup-fly-io branch June 21, 2024 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants