Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: translate TaskRunner #1

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Conversation

therealvio
Copy link
Contributor

@therealvio therealvio commented Nov 1, 2024

Purpose 🎯

These changes transfer the original TaskRunner code from one of our services into this repo. The selected service represented our "favoured" application of the TaskRunner pattern that we want to decompose into a Buildkite plugin. This involved translating from Typescript to Golang. This change will act as the starting point for further iteration as we move towards making it how we need it to be.

Context 🧠

  • Originating Task CRSE-4960
  • Part of wider effort to create the ecs-task-runner Buildkite plugin CSRE-4944

src/internal/aws/ecs.go Outdated Show resolved Hide resolved
@therealvio therealvio force-pushed the feat/translate-task-runner branch 2 times, most recently from 9e0d7ac to 4154d59 Compare November 27, 2024 00:58
Copy link
Member

@liamstevens liamstevens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general this looks good. Let's align the versions of Go we are using, though.

Additionally, I think it's worth considering fuzzing for our tests in order to give us some additional confidence. We can assume some structure around our inputs given the requirements imposed by AWS.

src/go.mod Show resolved Hide resolved
@therealvio
Copy link
Contributor Author

Additionally, I think it's worth considering fuzzing for our tests in order to give us some additional confidence. We can assume some structure around our inputs given the requirements imposed by AWS.

Is this something we want to maybe include in the related jira ticket as a checklist item so we can break it off into a dedicated card?

@therealvio therealvio force-pushed the feat/translate-task-runner branch 3 times, most recently from aa6397b to ad783ba Compare November 27, 2024 05:52
@therealvio therealvio marked this pull request as ready for review November 28, 2024 02:38
@therealvio therealvio requested a review from a team November 28, 2024 02:38
Comment on lines 77 to 84
waiterClient := ecs.NewTasksStoppedWaiter(ecsClient, func(o *ecs.TasksStoppedWaiterOptions) {
o.MinDelay = time.Second
o.MaxDelay = 10 * time.Second
})
result, err := awsinternal.WaitForCompletion(ctx, waiterClient, taskArn)
if err != nil {
// TODO: Do we wanna wanna go from fatal, to print, and provide an opportunity to share logs from the task if there any?
// That, or we include the log sharing logic within this condition
log.Printf("error waiting for task completion: %v", err)
log.Fatalf("failure information: %v", result.Failures[0])
}
Copy link
Contributor Author

@therealvio therealvio Nov 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the murmur implementation, this was done by checking the result of the waiter. Our waiter doesn't include this information. Instead, we're working with the Golang SDK by taking the error and result values and using them in the error handling, rather than waiting near the end of the execution (like in the original implementation).

We're failing fast, and that's probably a better pattern than carrying on needlessly.

@therealvio therealvio force-pushed the feat/translate-task-runner branch 3 times, most recently from e20d05e to 8600d08 Compare November 28, 2024 05:56
liamstevens
liamstevens previously approved these changes Nov 29, 2024
Copy link
Member

@liamstevens liamstevens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall - happy for this to be merged.

Minor nit on the Go versions.

src/go.mod Show resolved Hide resolved
This change is a naive golang translation of the `TaskRunner` code from
Murmur (refer to notes). The aim of this commit is to get the initial
code into the repo and start building on it from there.

Notes:
- TaskRunner permalink: https://github.com/cultureamp/murmur/blob/260f8936d59aaa6074c6fc6f31b935e011bda9f6/ops/migrations-runner/lib/TaskRunner.ts
- migrations-runner permalink: https://github.com/cultureamp/murmur/tree/078d0154b1215f226df5d3d081dfd131a8ae898a/ops/migrations-runner
The value to have deterministic import orders in our code doesn't seem
that useful considering that imports will automatically be cleaned up
and sorted thanks to the native tooling. This should be good enough
without another opinionated tool complaining that the order isn't _that_
specific way. It's low-impact, and not worth having 2 toolings fight
over this.
Copy link

github-actions bot commented Dec 3, 2024

Package Line Rate Health
ecs-task-runner/aws 81%
ecs-task-runner/buildkite 0%
ecs-task-runner 0%
ecs-task-runner/plugin 4%
Summary 33% (84 / 251)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants