Skip to content

DevOps Pipeline

Kuan Fan edited this page Oct 11, 2019 · 15 revisions

page owner: TFRS DevOps Specialist

Purpose

DevOps.com describes the Continuous Delivery Pipeline with these three characteristics: [1]

  • enables organizations to deliver new features to users as fast and efficiently as possible
  • to create a repeatable, reliable and incrementally improving process for taking software from concept to customer
  • to enable a constant flow of changes into production via an automated software production line

TFRS Pipeline Implementation

Stages

Unit Test

TFRS Unit Test is the first stage in the pipeline. It includes the isolated tests for Python Api code and runs in a Python pod managed by Kubernetes. When runs the unit tests, a code coverage report is created as well. The report shows how many percentages of the code are covered by the unit test cases. The number is about 91% at the moment of updating this wiki which is quite high.

Quality Check

Sonarqube is used by Quality Check stage to give TFRS ability to run continuous inspection of code quality. It performs automatic reviews with static code analysis to detect bugs, code smells, security vulnerabilities and etc.

Security Vulnerabilities Test

The OWASP Zed Attack Proxy (ZAP) automatically finds security vulnerabilities in web applications and a Zap report is created in Jenkins after the scan completes.

Build

At this stage, the TFRS build process is triggered to checkout the application source code from GitHub to the base Openshift Python and starts to build the TFRS application image. The image is then put to TFRS image stream to be used by dev, test and prod environment.

Deploy to Dev

The Dev deployment process is triggered at this stage. The application is deployed on Dev environment according to the deployment config.

Functional Test on Dev

This is an integration test stage, a set of predefined functional test cases are run to validation the application deployed on Dev environment. All the functions such as Authentication, Authorization, user/role management and all TFRS business functions are validated during this test. A report is generated and put in Jenkins to show the results of the testing. All functional tests have to pass in order to proceed to next stage.

Deploy to Test

Once all above stages pass, the application is deployed on Test environment.

Deploy to Prod

The last stage deploys the application on Prod environment.

Completed Pipeline Enhancements

  • Jenkins upgrade to allow multiple pipelines creation
  • Pipeline triggering every night during quiet time
  • Extract Unit Test from build stage to independent stage
  • Run functional test in Browser Stack
  • verified deployment to make sure test running against correct release
  • advanced image tagging to simplify tagging process and save image storage
  • Integrate unit test and code coverage report to Jenkins for each build

Future Pipeline Enhancements

  • additional Functional Testing
  • checking compliance with Privacy Impact
  • verifying completion of Data Classification
  • verifying generation of schema documentation