Skip to content

AWS Lambda + OpenAPI + DynamoDB + Terraform + GitHub Actions

Notifications You must be signed in to change notification settings

radhixcode/devops-terraform

Repository files navigation

Setup

Run on local machine

  • Install pip3 and install needed dependency packages
pip3 install botocore boto3 dijkstar
  • Install docker and run docker command to install dynamoDB locally then run the container before running the bash script.
docker run -p 8000:8000 amazon/dynamodb-local  
  • Run bash script create-local-db.sh to create the dynamoDB tables airport and vehicle and populate them
./create-local-db.sh
  • Check dynamoDB tables
aws dynamodb list-tables --endpoint-url http://localhost:8000
aws dynamodb describe-table --table-name airport --endpoint-url http://localhost:8000
  • Inside lambda folder run bash script run.sh to run the lambda
./run.sh

Terraform with AWS

  • Install openapi-merger for combining OpenAPI specifications (Inside openapi folder)
npm install openapi-merger -g 
openapi-merger -i main.yaml -o deploy-api.yaml
  • Install Terraform with brew
brew tap hashicorp/tap
brew install hashicorp/tap/terraform
  • Terraform commands. Run inside terraform folder
terraform init (initializes the current directory)
terraform init -upgrade (upgrade to latest acceptable provider version)
terraform validate (command validates the configuration files in a directory)
terraform fmt (format your configuration files into a canonical format and style)
terraform plan (dry run to see what Terraform will do)
terraform refresh (refreshes the state file)
terraform output (views Terraform outputs)
terraform apply (applies the Terraform code and builds stuff)
terraform graph (creates a DOT-formatted graph)
terraform destroy --target aws_lambda_function.lambda (destroys indivifual resource)
terraform destroy  (destroys what has been built by Terraform)
  • Populate AWS DynamoDB after terraform apply
./update-db.sh
  • AWS lambda sample test JSON
{
  "resource": "/vehicle/{people}/{distance}",
  "path": "/vehicle/2/100",
  "httpMethod": "GET",
  "queryStringParameters": {},
  "multiValueQueryStringParameters": {},
  "pathParameters": {
    "people": "2",
    "distance": "100"
  },
  "stageVariables": "None",
  "body": "None",
  "isBase64Encoded": "FALSE"
}

Github actions

  • .github/workflow/terraform.yaml is the workflow file
  • Create new IAM user for github actions and get the credentials.
  • Add new New repository secrets (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, BUCKET_TF_STATE) in Actions secrets of the Github repository.

Current Shortcomings

  • Github actions: Can do terraform apply locally for the AWS but github action terrafom apply didn't trigger on merge/push to the main branch.
  • API gateway: Manual steps involved.
    AWS Labmda is working (tested with above sample JSON). But to test API gateway Method execution, Integration Request function name needed to be manually updated for each GET.

Img 1 Img 2

Execute the 'Deploy API' button for the root resource ('/') from the AWS API gateway console.

Img 3

  • Public URL: The public URL throws Internal server error or Endpoint request timed out for some time and then it works (Need x-api-key which we can get this from Amazon API gateway -> API keys)
curl --location --request GET 'https://8v5w5mv4ze.execute-api.us-east-1.amazonaws.com/holiday-stage/vehicle/2/100' \
--header 'Content-Type: application/json' \
--header 'x-api-key: <x-api-key here>'

Img 3

Reference

Logo

Holiday Agency

Overview

A holiday agency would like to suggest the lowest travel cost for holiday journeys to their customers.
A return journey consists of the following parts:

  1. Journey to the airport:

    • Taxi: £0.40/mile - (allows up to 4 people per taxi - e.g. require 2 taxis if 5 people travel together)
    • Car: £0.20/mile - (allows up to 4 people per car), additional £3 parking fee
  2. An outbound and an inbound flight journey:

    • Flight: £0.10/passenger/mile

This flight agency keeps a table with its airports and their connections:

Airport ID Airport Name Latitude Longitude Connections and distance
IST Istanbul Airport 41.262222 28.727778 ATH: 100mi
AMS: 1200mi
SVO: 800mi
VIE: 750mi
CDG Charles de Gaulle Airport 49.009722 2.547778 ORY: 10mi
LHR: 100mi
AMS: 120mi
FRA: 130mi
ZRH: 120mi
FCO: 500mi
LIS: 750mi
MAD: 600mi
LHR Heathrow Airport 51.4775 -0.461389 LGW: 10mi
CDG: 50mi
AMS: 100mi
FRA: 230mi
MAD: 500mi
LIS: 350mi
AMS Schiphol Airport 52.308056 4.764167 LHR: 100mi
FRA: 90mi
CDG: 110mi
OSL: 500mi
SVO Sheremetyevo International Airport 55.972778 37.414722 VKO: 10mi
DME: 15mi
LED: 200mi
ATH: 1300mi
FRA Frankfurt Airport 50.033333 8.570556 VIE: 300mi
MUC: 90mi
ZRH: 100mi
CDG: 105mi
LHR: 120mi
AMS: 100mi
DME: 1500mi
MAD Adolfo Suárez Madrid-Barajas Airport 40.472222 -3.560833 LIS: 145mi
FCO: 220mi
LHR: 500mi
BCN: 120mi
DME Moscow Domodedovo Airport 55.408611 37.906111 ATH: 1105mi
OSL: 980mi
VKO: 15mi
SVO: 20mi
LED: 210mi
ATH: 1000mi
BCN Josep Tarradellas Barcelona-El Prat Airport 41.296944 2.078333 MAD: 50mi
FCO: 140mi
VKO Vnukovo International Airport 55.596111 37.2675 SVO: 5mi
DME: 8mi
MUC Munich Airport 48.353889 11.786111 VIE: 90mi
ZRH: 80mi
FRA: 100mi
FCO: 200mi
LED Pulkovo Airport 59.800278 30.2625 VKO: 190mi
SVO: 170mi
OSL: 300mi
LIS: 2500mi
ORY Orly Airport 48.723333 2.379444 CDG: 5mi
AMS: 150mi
LGW Gatwick Airport 51.148056 -0.190278 LHR: 10mi
ORY: 125mi
FCO Leonardo da Vinci-Fiumicino Airport 41.800278 12.238889 ATH: 200mi
BCN: 190mi
IST: 310mi
AMS: 495mi
LIS Lisbon Airport 38.774167 -9.134167 MAD: 100mi
BCN: 300mi
LHR: 300mi
SVO: 2100mi
OSL Oslo Airport - Gardermoen 60.202778 11.083889 LED: 320mi
LHR: 390mi
FRA: 720mi
AMS: 690mi
VKO: 980mi
ZRH Zurich Airport 47.464722 8.549167 FCO: 150mi
MUC: 85mi
VIE: 190mi
FRA: 100mi
CDG: 180mi
AMS: 310mi
ATH Athens International Airport 37.936389 23.947222 IST: 150mi
FCO: 200mi
MAD: 500mi
VIE Vienna International Airport 48.110833 16.570833 IST: 350mi
VKO: 890mi
OSL: 880mi
MUC: 95mi
FCO: 200mi

While the route A->B can exist, it doesn't necessarily mean that B->A exists. Additionally, the mileage may be different for each direction.

As a side-note, some of the mileages might not be at all accurate but imagine that someone make some fairly large one-way portals in the sky that can shorten the distance between two airports.

The current logic allows for the following endpoints:

  • GET: /airports - returns a list of airports
  • GET: /airports/{airportId} - return details of a single airport
  • GET: /airports/{airportId}/to/{airportToId} - returns the cheapest quote for a journey from airportId to airportToId
  • GET: /vehicle/{numberOfPeople}/{distance} - returns the cheapest vehicle to use (and the cost) for the given numberOfPeople and distance in miles