Skip to content

Latest commit

 

History

History
85 lines (66 loc) · 3.18 KB

README.md

File metadata and controls

85 lines (66 loc) · 3.18 KB
Raw

drone-autoscaler

This app scales a Drone CI agent cluster up or down based on build volume.

It was created because the autoscaler provided by Drone has several limitations:

  1. It doesn't support AWS's Autoscaling Groups (ASGs). Instead, it creates standalone ec2 instances. drone-autoscaler is designed to simply manipulate the desired capacity of the ASG that manages your Drone Agents.
  2. It installs & configures an agent on a newly created machine. drone-autoscaler assumes that any new machine in the agent ASG is already provisioned.
  3. It waits for upto 60 minutes for running builds to finish on an agent marked for termination. drone-autoscaler only terminates nodes that aren't running any builds.

Usage

Setup

This app assumes that your Drone agent cluster is managed by an AWS Autoscaling Group. New machinesi in this group should already be configured to run an agent upon startup and those being stopped/terminated should stop the agent gracefully.

The app needs access to IAM credentials with the following permissions:

ec2:DescribeTags

ec2:DescribeInstances
ec2:DescribeInstanceTypes
ec2:DescribeInstanceStatus

autoscaling:TerminateInstanceInAutoScalingGroup

autoscaling:SetInstanceProtection
autoscaling:SetDesiredCapacity

autoscaling:DetachInstances

autoscaling:DescribeScalingProcessTypes
autoscaling:DescribeScalingActivities
autoscaling:DescribeAutoScalingInstances
autoscaling:DescribeAutoScalingGroups"

Configuration

The app's behaviour can be configured using various parameters

Environment variable Required
DRONE_AGENT_MAX_BUILDS Yes
DRONE_AGENT_AUTOSCALING_GROUP Yes
DRONE_SERVER_HOST Yes
DRONE_SERVER_AUTH_TOKEN Yes
SCALER_PROBE_INTERVAL No
SCALER_LOG_FORMAT No
SCALER_DEBUG No
SCALER_DRY No
DRONE_AGENT_MIN_RETIREMENT_AGE No
DRONE_AGENT_MIN_COUNT No
DRONE_SERVER_PROTO No
DRONE_BUILD_PENDING_MAX_DURATION No
DRONE_BUILD_RUNNING_MAX_DURATION No

See config.go for parameter descriptions

Note that the autoscaler cannot scale beyond the maximum machine count set in your agent autoscaling group.

Running

  1. Download a pre-compiled binary from the releases page or build it from code using make dist.
  2. Set the required configuration parameters via environment variables.
  3. Run the acquired binary.

Developing

The recommended way to run the app in development mode is to use the following configuration:

SCALER_LOG_FORMAT=text
SCALER_DEBUG=true

DRONE_BUILD_PENDING_MAX_DURATION="4h"
DRONE_BUILD_RUNNING_MAX_DURATION="1h"

# Outputs what the app plans to do, without making any changes
# to the actual infrastructure. This allows you to point the
# app to the actual agent ASG in development mode without
# worrying about accidental destruction of nodes. 
SCALER_DRY=true

# If AWS profile is configured, use these params to load creds
# and all profile configuration.
AWS_PROFILE=my_profile
AWS_SDK_LOAD_CONFIG=true

Run make fmt to format the Go code. To run tests, use make test.

To create a new release, bump the app version in main and run make dist.