Skip to content

This repository contains backup tools and scripts for automating the backup of HashiCorp Vault data, S3 buckets, and GitHub organizations. It includes Bash scripts and Dockerfiles to facilitate creating backups, uploading them to AWS S3, and validating Vault backups to ensure data integrity.

Notifications You must be signed in to change notification settings

GlueOps/backup-tools

Repository files navigation

Requirements for ALL backup jobs

  • AWS S3 Credentials. Regardless of whether you backup google team drives or github repos, you will need these environment variables to be set:
export AWS_DEFAULT_REGION=us-west-2 # must be the same region as the bucket
export S3_BUCKET_NAME="bucket-name"
export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXXX 
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXX

GitHub Organization backups

Following variables must be set

export GITHUB_ORG_TO_BACKUP="GlueOps" # Set this to the organization you want to backup. The GITHUB_TOKEN must have read access to all the repos in this organization.
export GITHUB_TOKEN="" # GitH needs to have read access to all repositories within the organization. We use the fine grained access tokens (beta feature)
  • To create a GITHUB_TOKEN use the newer fine grained tokens:
image
  • Example backup
docker build . -t backup && docker run -it backup
# Export ALL the variables required as mentioned in this README.md and then run:
backup-github

Google Drive Shared Drive Backups

note this only works for shared team drives. we do not have anything to backup personal drives

Following variables must be set

export RCLONE_DRIVE_SERVICE_ACCOUNT_CREDENTIALS='<<json-without \n (newlines)>>' # Get this from the IAM user in the rclone google cloud service account project and remove all newlines \n
export RCLONE_DRIVE_TEAM_DRIVE="XXXXXXXXXXXXXX" # team drive id ex. `0ZZH9DD53YuyEaYU7sqb`
  • Example to run a download of the team drive to local
docker build . -t backup && docker run -it backup
# Export ALL the variables required as mentioned in this README.md and then run:
./gdrive-backup.sh

About

This repository contains backup tools and scripts for automating the backup of HashiCorp Vault data, S3 buckets, and GitHub organizations. It includes Bash scripts and Dockerfiles to facilitate creating backups, uploading them to AWS S3, and validating Vault backups to ensure data integrity.

Topics

Resources

Stars

Watchers

Forks

Packages