Terraform Azure Data Scientist Associate

The main purpose of this repository is to terraform all the resources needed for Exam DP-100: Designing and Implementing a Data Science Solution on Azure.

The end goal is to easily deploy all the resources needed for the self-paced learning modules. I have created resources following the instructions in the lab exercises to reflect using the Azure Portal UI. Please note your your mileage may (or might) vary, as these resources were deployed using my personal account which has no restrictions.

Development

Dependencies

• terraform
• terragrunt
• pre-commit
• terraform-docs this is required for terraform_docs hooks

Prerequisites

1. Have a Azure Portal account.
2. You will need to create a Service Principal with a Client Secret follow instructions.
3. Other permissions needed are Azure Key Vault and Azure Storage for the Terraform client.

Information on HashiCorp BSL License Change

Due to the HashiCorp BSL license change, restricting Terraform to the latest open source version (1.5.7). Will create a new project template using OpenTofu. To learn more, see the official OpenTofu website and project status.

Usage

1. Navigate to the environment you would like to deploy,
2. Plan your changes with terragrunt plan to see what changes will be made,
3. If you're happy with the changes terragrunt apply.

IMPORTANT

Please note that .tfstate files are stored locally on your machine as no backend has been specified. If you would like to properly version control your state files, it is possible to use an azure storage account to store these files. This will ensure anyone else other than you running a plan or apply will be using the same state file.

Pre-Commit hooks

Git hook scripts are very helpful for identifying simple issues before pushing any changes. Hooks will run on every commit automatically pointing out issues in the code e.g. trailing whitespace.

To help with the maintenance of these hooks, pre-commit is used, along with pre-commit-hooks.

Please following these instructions to install pre-commit locally and ensure that you have run pre-commit install to install the hooks for this project.

Additionally, once installed, the hooks can be updated to the latest available version with pre-commit autoupdate.

Documentation Generation

Code formatting and documentation for variables and outputs is generated using pre-commit-terraform hooks that in turn uses terraform-docs that will insert/update documentation. The following markers have been added to the README.md:

<!-- {BEGINNING|END} OF PRE-COMMIT-TERRAFORM DOCS HOOK --->

Requirements

Name	Version
terraform	>= 1.0.0, <= 1.5.7
azurerm	~> 3.105.0

Providers

Name	Version
azurerm	3.105.0

Modules

Name	Source	Version
ml_workspace	./modules/machine_learning_workspace	n/a
ml_workspace_storage	./modules/storage_account	n/a
tfstate_storage	./modules/storage_account	n/a

Resources

Name	Type
azurerm_resource_group.resource_group	resource
azurerm_client_config.current	data source

Inputs

Name	Description	Type	Default	Required
arm_client_id	The Client ID which should be used. This can also be sourced from the ARM_CLIENT_ID Environment Variable.	string	n/a	yes
arm_client_secret	The Client Secret which should be used. This can also be sourced from the ARM_CLIENT_SECRET Environment Variable.	string	n/a	yes
arm_subscription_id	The Subscription ID which should be used. This can also be sourced from the ARM_SUBSCRIPTION_ID Environment Variable.	string	n/a	yes
arm_tenant_id	The Tenant ID which should be used. This can also be sourced from the ARM_TENANT_ID Environment Variable.	string	n/a	yes
cloud_enviornment	The Cloud Environment which should be used. Possible values are public, usgovernment, german, and china. Defaults to public. This can also be sourced from the ARM_ENVIRONMENT Environment Variable.	string	"public"	no
create_tfstate_store	Create a storage account to remotely store tfstate for the project. NOTE: requires local .tfstate to be migrated to remote, after storeage account has been created.	bool	false	no
env_prefix	The prefix added to resources in the environment.	string	n/a	yes
environment	The name of the environment to help identify resources.	string	n/a	yes
location	The Azure Region where the Resource Group should exist. Changing this forces a new Resource Group to be created.	string	"West Europe"	no
personal_ip_address	Add your client IP address to the storage account	string	n/a	yes
tags	Tags to be added to resources created.	map(string)	{}	no

Outputs

Name	Description
service_principal_client_id	The principal being used to apply terraform changes for this subscription.
tenant_id	The tenant ID used for this subscription.