All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, adheres to Semantic Versioning, and is generated by Changie.
- Added
github
andjson
logging options to change output - Added option to set global and site-level configurations. These will be merged into the component variables during generation
- Added command to validate terraform configurations
- Added git-fallback option to cloud update command. Usage;
mach-composer update --cloud --git-fallback
- Continue with update proces even when commits / changelogs cannot be fetched between old and new version
- Updated to latest SDK version
- Added a ticker when running go routines to show activity
- chore(deps): bump the github-actions group with 2 updates
- chore(deps): bump the go group with 2 updates
- support for nix package
- Added
create-component
flag to register-component-version command
- Load referenced files before validating config
- Added all the terraform azurem backend options
- chore(deps): bump the go group with 7 updates
- Added option to set reusable component templates in configuration
- Updated goreleaser configuration
- Added option to register-component-version to set branch
- Added additional logging around $LATEST when using mach-composer update with cloud
- chore(deps): bump the go group across 1 directory with 14 updates
- chore(deps): bump golang.org/x/net from 0.22.0 to 0.23.0
- Escape backslashes in module source path for Windows
- Fixed state file locations
- Added sops file to hash
- Use cloud endpoint to fetch commits during update
- Moved hash storage out of terraform
- Added option to quiet log outputs
- Improved source handling so we can use multiple types of sources
- Changed --force naming to the more descriptive --ignore-change-detection
- Re-added chocolaty for beta building
- Deprecated warning for missing commits in diff to info
- Made terraform init parallelization configurable, and run in same order as other batches
- Added depends_on to hash outputs
- Skip directory hash generation on terraform code when git source
- Removed lock file usage
- Added check on terraform files to detect changes
- Added option to force run commands that would normally be skipped if change detection is not triggered
- Made hash check more permissive for site components
- Set correct backend for aws remote state
- Made module outputs sensitive
- Added support for v2 version of SDK
- Upgraded dependencies to latest minor versions
- Moved commands to internal to enable code generation
- Updated docs to latest state
- Added command to output graph in dot language
- Reworked state handling to allow for separate state per component
- Added parallelism to runners
- Added change detection when processing nodes
- Added option to run mach-composer plan without lock on state file
- Updated release pipelines
- Updated dependencies and added support for go 1.21.x
- Fix resolving plugin executables on Windows
- Fixed usage of root directory when loading referenced files
- Fix for variables_file containing a directory
- Added gcp docker container
- Fixed iterating until beginning if commit not found in paths.
- Added git tags to update output
- Fixed lockfile interpolation of $LATEST
- Fixed tag interpolation during update check
- reworked ref resolution to also deal with include
- Add state generation directly into CLI
- Fixed handling of mach-composer update
- Fixed issue with unknown paths field
- Added filtering on file paths to determine updates in update functionality
- Removed built in plugins
- Fixed race condition in git repo downloads
- Added deprecation warning for endpoints
- Fixed slice interpolation in variable processing
- Fixed goreleaser config to point to correct domains
- Allow running mach-composer update with referenced components file
- updated version of commercetools plugin
- Don't render empty depends_on or providers block in modules
- Refactor matching if commits are relevant for path filter
- Improve the component version registration flow
- Don't error when a previous version is not found in the git repository
- Add
--dry-run
and--git-filter-paths
tocloud register-component-version
command. The--git-filter-paths
argument can be used to limit the scope of the changes when working within monorepo's
- Update commercetools plugin to latest version
- Updated dependencies
- Improve the auto-completion with support for the config files, sites and components (#139)
- Add support for resolving a component version $LATEST. When the version is defined as $LATEST resolve the version by querying mach composer cloud. This is only enabled when the config file has cloud support enabled by defining the organization/project
- Remove the
-lockfile=readonly
argument to terraform apply. This was introduced in 2.7.0 but didn't work correctly. - Add fallback to check in git when using
mach-composer update --cloud
when a component isn't found.
- Default to branch
main
when no branch is defined for a component when retrieving the last version.
- Add
--cloud
argument to themach-composer update
command to fetch the latest version registered instead of retrieving the latest commit from the Git repository. - Add
mach-composer cloud update-component
to update an existing component to for example rename the key. - Update built-in plugins to the following versions: mach-composer-plugin-commercetools v0.1.7 mach-composer-plugin-sentry v0.1.3
- Revert the
depends_on
change since it causes data sources to be read too late resulting in unexpected changes. See https://itnext.io/beware-of-depends-on-for-modules-it-might-bite-you-da4741caac70
- Set
depends_on
on components (modules) when output variables are used. This turned out to be neccessary when the variables were used in data sources.
- Update both built-in plugins to the following versions: mach-composer-plugin-aws v0.1.0 mach-composer-plugin-azure v0.1.0 mach-composer-plugin-commercetools v0.1.5 mach-composer-plugin-contentful v0.1.0 mach-composer-plugin-sdk v0.0.6 mach-composer-plugin-sentry v0.1.2
- Update the commercetools built-in plugin to v0.1.1.
- Fix invalid rendering of store variables in the generated terraform files.
- Fix explicitly disabling the creation of frontend credentials.
- Fix setting store_secrets and store_variablds in the commercetools block
- Fix authentication flow with Mach Composer Cloud and store the refreshed tokens correctly.
-
Add optional support to automatically download MACH composer plugins when defined in the config file. For example::
mach_composer: version: 1 plugins: aws: source: mach-composer/aws version: 0.1.0 sentry: source: mach-composer/sentry version: 0.1.2
Will download the plugins to your local plugin directory if these are not found.
- Update the Commercetools plugin to version 0.1.0. This version is compatible with more recent commercetools terraform providers.
- Update the Sentry plugin to version 0.1.1. This allows using encrypted secrets for the auth token
- Update the Sentry plugin to version 0.1.0
- Properly close plugins when mach-composer is terminated (e.g. via ctrl-c)
- Only load built-in plugins when they are not found on the $PATH
- Add
--no-color
flag tomach-composer show-plan
to use in GitHub actions
- Print a deprecation warning when using
mach-composer init
. This command is replaced bymach-composer terraform init
. - Generate a mach-composer lock file in the output directory to optimize rebuilds and prevent unnecessary terraform actions.
- Add a new command
mach-composer show-plan
which shows the generated plans by terraform.
Disable automatic distribution to chocolatey, it seems to not work together properly with automated releases.
Patch release to fix windows distribution via chocolatey
This release introduces support for plugins in mach-composer. All previous supported plugins (also called integrations) are still bundled with the exectutable but will in the future be distributed separately.
With this release it is also possible to use build custom plugins. A plugin
should be created with the mach-composer-plugin-sdk and the resulting executable
should be available on your $PATH as mach-composer-plugin-<plugin>
Other changes include:
- New
schema
subcommand which outputs a json schema based on the given config file (to know which plugins to use). - Improved error and warning output (inspired by Terraform)
- Improved variable handling, only allow SOPS for encrypted variables when the variables are used.
- Fix issue with parsing aws_account_id by required it to be a string, fixes invalid yaml guessing when the account_id starts with a zero.
- Interpolate variable references in the global config block
- Fix an encoding bug when passing sops encrypted variables to the generated terraform file.
- Fix another config inheritance issue in the sentry plugin
- Fix serialization issue for variables when the key was not a string. We now force keys of maps to be strings.
- Fix config inheritance issue in the sentry plugin (global -> component)
- Validate the AWS Account ID as a string, fixes an issue if the account starts with 0.
- Set default sentry terraform provider version
- Update build process to include arm docker images
- Major rewrite to move to a plugin based system. The plugins are for now all built-in and part of the mach composer release but will in the future be moved to external plugins.
- Use
go-git
to retrieve the git log instead of executinggit
directly. This fixes (#188) - Remove panic() calls in the codebase and handle errors properly by printing them to stdout and setting correct exit code
- Re-implement the
$(include())
syntax for include a separate components file for the 2.0 versione (#115) - Fix rendering of variables in the generated terraform output (#194)
- Fix a yaml parsing bug which resulted in not being able to disable management of commercetools stores. (#193)
- Escape backslashes for strings in HCL (#133)
- Fix windows version due to wrong handling of the path separator (#182)
- Restore the
sites
subcommand to output the sites in the config (#178) - Restore the
components
subcommand to output the components in the config (#179) - Support only passing one config file and use
main.yml
as default for now (#177)
- Fixed inconsistencies between 1.2 and 2.x:
- Add
branch
option to component definitions to be able to perform amach-composer update
and stay within a certain branch (during development) - Fixed commercetools
stores
definitions - Fixed bug where custom AWS and Azure provider version definitions weren't picked up
- Add
- Fixed inconsistencies between 1.2 and 2.x:
- Upgrade Terraform providers in golang version of the MACH composer to match the 1.2 release:
- Upgraded commercetools provider to 0.30.0
- Upgraded Amplience provider to 0.3.7
- Upgraded Azure provider to 2.99.0
- Add
variables_file
option to themach_composer
configuration block to define a variable file - Fix auto add cloud integration (aws or azure) when
integration
list is left empty - Add ability to define a custom provider version including the version operator
- Upgrade Terraform providers in golang version of the MACH composer to match the 1.2 release:
- Deprecate
commercetools.frontend
block, will be removed in a later release.
- Don't crash when running
mach-composer apply
without--auto-approve
- Add back support to update sops encrypted config files
- Properly implement the
--check
flag onupdate
command
- Pass environment variables to terraform command
- Add aws-cli to the Docker container
Rewrite of the Python codebase to Go. Goal is to make it easier to distribute mach-composer in a cross-platform way.
A number of features which were minimal used are removed.
- The
mach bootstrap
command is no longer present. It was a simple wrapper around Python cookiecutter. This can still be used separately - The
mach sites
andmach components
commands since they were unused. - The
--with-sp-login
is removed. This flags used to runaz login
. If this is needed it needs to be run before mach-composer is run. - The
--ignore-version
flag is removed. The version in the config file now indicates a schema version. Only version 1 is supported and updates within this schema version should always be backwards compatible.
general
- Add
mach init
command - Skip non-MACH configuration files when processing all yaml files in a directory.
This allows you to run things likemach apply
ormach update
without having to specify the-f main.yml
option if you only have one valid MACH configuration file in your directory. Fixes #150 - Ignore missing variables when running
mach sites
andmach components
- Add
--destroy
flag to theplan
andapply
commands - Add
variables_file
option to themach_composer
configuration block to define a variable file - Show commit author in
mach update
output - Upgraded commercetools provider to
0.30.0
- Upgraded Amplience provider to
0.3.7
- Upgraded Azure provider to
2.99.0
AWS
- Upgraded Terraform AWS provider to
3.74.1
- Add support for default tags on provider level
aws: account_id: 123456789 region: eu-central-1 default_tags: environment: test owner: john
General
- Variable support:
${var.}
to be used with the--var-file
command line option${component.}
to use component output values${env.}
to include environment variables in the configuration file
AWS
- Upgraded Terraform AWS provider to
3.66.0
- Add AWS specific endpoint options;
enable_cdn
creates a CDN in front of an endpointthrottling_burst_limit
andthrottling_rate_limit
controls throttling on the API gateway
Azure
- Upgraded Terraform Azure provider to
2.86.0
- Add extra Frontdoor frontend_endpoint options:
- session affinity
- waf policy support
- Add extra Frontdoor routing options to components such as:
- Custom routing paths
- Health probe settings
- Custom host address and ports
- Caching options
- Include
frontend_endpoint
in ignore list whensuppress_changes
is used - Add Frontdoor
ssl_key_vault
option to supply your own SSL certificate for your endpoints - Add Azure specific endpoint options:
internal_name
Overwrites the frontend endpoint namewaf_policy_id
Defines the Web Application Firewall policy ID for the endpointsession_affinity_enabled
Whether to allow session affinitysession_affinity_ttl_seconds
The TTL to use in seconds for session affinity
- Add new
service_plans
optionper_site_scaling
- Fix: set correct root-level DNS record (
@
) when endpoint URL is the same as the zone
Commercetools
- Upgraded Terraform commercetools provider to
0.25.3
- Add
tax_categories
to allow more complex tax setups. Does not work in conjunction withtaxes
For Azure
- Each component that has an
endpoint
defined needs to have an Terraform output defined for that endpoint. For example:
Read more about the configuration options.output "azure_endpoint_main" { value = { address = azurerm_function_app.main.default_hostname } }
- Remove endpoints restrictions: Azure components can now use multiple endpoints.
- Changes have been made in the Frontdoor configuration in the underlying Terraform Azure provider.
If you are using endpoints with a custom domain, you'll need to import the newazurerm_frontdoor_custom_https_configuration
into your Terraform state.
More on how to work with the Terraform state in our troubleshooting guide.
New platforms
- Add Amplience support
- Add Apollo Federation support
- Add Sentry DSN management options
General
- Add
mach_composer
configuration block to configure required MACH composer version - SOPS support: SOPS-encrypted configuration files will get decrypted before being parsed further
- Add
--ignore-version
to disable the MACH composer version check - Improved development workflow:
- Improved git log parsing
- Add
mach bootstrap
commands:mach bootstrap config
for creating a new MACH configurationmach bootstrap component
for creating a new MACH component
- Add
--site
option to thegenerate
,plan
andapply
commands - Add
--component
option to theplan
andapply
commands - Add
--reuse
flag to theplan
andapply
commands to suppress aterraform init
call - Add support for relative paths to components
- Add extra component definition settings
artifacts
to facilitate local deployments
- Improved dependencies between components and MACH-managed commercetools configurations
- Add option to override Terraform provider versions
- Add support for multiple API endpoints:
base_url
replaced withendpoints
has_public_api
replaced withendpoints
- Supports a
default
endpoint that doesn't require custom domain settings
- Add support for including yaml files using the
${include(...)}
components: ${include(components.yml)} components: ${include(git::https://github.com/labd/mach-configs.git@9f42fe2//components.yml)}
commercetools
- Move
currencies
,languages
,countries
,messages_enabled
toproject_settings
configuration block - Add support for commercetools Store-specific variables and secrets on
components included in new variable:
ct_stores
- Add
managed
setting to commercetools store. Set to false it will indicate the store should not be managed by MACH composer - Add support for commercetools shipping zones
- Make commercetools frontend API client scopes configurable with new
frontend
configuration block
AWS
- AWS: Set
auto-deploy
on API gateway stage - AWS: Add new component variable
tags
Azure
- Add configuration options for Azure service plans
- Upgraded Terraform to
0.14.5
- Upgraded Terraform commercetools provider to
0.25.3
- Upgraded Terraform AWS provider to
3.28.0
- Upgraded Terraform Azure provider to
2.47.0
- Azure: Remove
project_key
fromvar.tags
and addEnvironment
andSite
- Azure: Add
--with-sp-login
option tomach plan
command - Azure: Remove function app sync bash command: this is now the responsibility of the component
Generic
- config: Rename
general_config
toglobal
- config:
base_url
has been replaced by theendpoints
settings:
becomessites: - identifier: mach-site-eu base_url: https://api.eu-tst.mach-example.net
When you name the endpoint that replacessites: - identifier: mach-site-eu endpoints: main: https://api.eu-tst.mach-example.net
base_url
"main", it will have the least effect on your existing Terraform state.
When endpoints are defined on a component, the component needs to define endpoint Terraform variables (AWS and Azure) - config: commercetools
create_frontend_credentials
is replaced with newfrontend
block:default is stillcommercetools: frontend: create_credentials: false
true
- config Default scopes for commercetools frontend API client changed:
- If you want to maintain previous scope set, define the following in the
frontend
block:
commercetools: frontend: permission_scopes: [manage_my_profile, manage_my_orders, view_states, manage_my_shopping_lists, view_products, manage_my_payments, create_anonymous_token, view_project_settings]
- Old scope set didn't include store-specific
manage_my_profile:project:store
scope. If you're using the old set as described above, MACH will need to re-create the store-specific API clients in order to add the extra scope. For migration options, see next point - In case the scope needs to be updated but (production) frontend
implementations are already using the current API client credentials, a
way to migrate is to;
- Remove the old API client resource with
terraform state rm commercetools_api_client.frontend_credentials
- Repeat step for the store-specific API clients in your Terraform state
- Perform
mach apply
to create the new API clients with updated scope - Your commercetools project will now contain API clients with the same name. Once the frontend implementation is migrated, the older one can safely be removed.
- Remove the old API client resource with
- If you want to maintain previous scope set, define the following in the
- component: Components with a
commercetools
integration require a new variablect_stores
:variable "ct_stores" { type = map(object({ key = string variables = any secrets = any })) default = {} }
- component: The folowing deprecated values in the
var.variables
are removed:See 0.5.0 release notesvar.variables["CT_PROJECT_KEY"] var.variables["CT_API_URL"] var.variables["CT_AUTH_URL"]
- component: The
var.environment_variables
won't be set by MACH anymore. Usevar.variables
for this
AWS
- config: The AWS
route53_zone_name
setting has been removed in favour of multiple endpoint support - config: The
deploy_role
setting has been renamed todeploy_role_name
- component: Introduced new variable
tags
:variable "tags" { type = map(string) description = "Tags to be used on resources." }
- component: Add
aws_endpoint_*
variable when theendpoints
configuration option is used. More information on defining and using endpoints in AWS.
Azure
-
config: The
front_door
configuration block has been renamed tofrontdoor
-
config: The Azure frontdoor settings
dns_zone
andssl_key_*
settings have been removed;
Certificates are now managed by Frontdoor and dns_zone is auto-detected. -
config: The Azure frontdoor settings
resource_group
has been renamed todns_resource_group
-
config: Moved component
short_name
to newazure
configuration block -
state: The Terraform
azurerm_dns_cname_record
resources have been renamed; they now take the name of the associated endpoint key. For the smoothest transition, rename them in your Terraform state:terraform state mv azurerm_dns_cname_record.<project-key> azurerm_dns_cname_record.<endpoint-key>
-
component: Prefixed all Azure-specific variables with
azure_
-
component: The
FRONTDOOR_ID
value is removed from thevar.variables
of a component. Replaced withvar.azure_endpoint_*
. More information on defining and using endpoints in Azure. -
component:
app_service_plan_id
has been replaced withazure_app_service_plan
containing both anid
andname
so the azurerm_app_service_plan data source can be used in a component.It will only be set when
service_plan
is configured in the component definition or site configurationvariable "azure_app_service_plan" { type = object({ id = string name = string resource_group_name = string }) }
-
component: Replaced
resource_group_name
andresource_group_location
withazure_resource_group
:variable "azure_resource_group" { type = object({ name = string location = string }) }
- Removed
aws
block in general_config - Add
branch
option to component definitions to be able to perform amach update
and stay within a certain branch (during development)
- Add new CLI options:
mach-composer components
to list all componentsmach-composer sites
to list all sites
- Improved
update
command:- Supports updating (or checking for updates) on all components based on their git history
- This can now also be used to manually update a single component;
mach-composer update my-component v1.0.4
- Add
--commit
argument to automatically create a git commit message
- Add new AWS configuration option
route53_zone_name
- Remove unused
api_gateway
attribute on AWS config - Remove restriction from
environment
value; can now be any. Fixes #9
- Require
ct_api_url
andct_auth_url
for components withcommercetools
integration
In a component, the use of the following variables have been deprecated;
var.variables["CT_PROJECT_KEY"]
var.variables["CT_API_URL"]
var.variables["CT_AUTH_URL"]
Instead you should use:
var.ct_project_key
var.ct_api_url
var.ct_auth_url
- Make AWS role definitions optional so MACH can run without an 'assume role' context
- Add 'encrypt' option to AWS state backend
- Correctly depend component modules to the commercetools project settings resource
- Extend Azure regions mapping
- Fixed TypeError when using
resource_group
on site Azure configuration
- Add Contentful support
is_software_component
has been replaced by theintegrations
settings
components:
- name: my-product-types
source: git::ssh://git@github.com/example/product-types/my-product-types.git
version: 81cd828
is_software_component: false
becomes
components:
- name: my-product-types
source: git::ssh://git@github.com/example/product-types/my-product-types.git
version: 81cd828
integrations: ["commercetools"]
or integrations: []
if no integrations are needed at all.
- Add option to specify custom resource group per site
- All
resource_group_name
attributes is renamed toresource_group
- The
storage_account_name
attribute is renamed tostorage_account
- Fixed Azure config merge: not all generic settings where merged with site-specific ones
- Only validate short_name length check for Azure implementations
- Setup Frontdoor per 'public api' component regardless of global Frontdoor settings
- Fixed rendering of STORE environment variables in components
- Updated Terraform version to 0.13.4
- Fix
--auto-approve
option onmach apply
command
- Add AWS support
- Add new required attribute
cloud
in general config
- Initial release of rewrite to Python (previously Go)
Initial version