-
-
Notifications
You must be signed in to change notification settings - Fork 115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
exposing terraform info back to nixos configurations #413
Comments
i had this same problem and after fighting forever with flakes, i touched upon this solution for now: resource "local_file" "arm-nixincludes" {
filename = "${path.module}/arm/tofuout.nix"
file_permission = "0644"
content = <<EOT
{ networking.hostName = "${oci_core_instance.arm.display_name}"; }
EOT
}
module "nixos-anywhere" {
source = "github.com/nix-community/nixos-anywhere/terraform/all-in-one"
depends_on = [
local_file.arm-nixincludes
]
... another solution i thought about sitting on the toilet: allow the put this down to the long list of problems caused by flakes' behaviour with git, and not having an impure escape hatch |
This PR adds a Terraform input variable I so far named `content`. This allows passing in a string from Terraform to expose to nixos-anywhere's build step to make available as a file (default: `./nixos-vars.json`) within a NixOS configuration, as suggested by @Mic92 at nix-community#414. Note the file is staged even if added to gitignore, making this less suited for development in case the file includes ephemeral content. As a result, I would consider this approach complementary rather than as superseding nix-community#414. Example usage: ```nix let servers = ...; variable = ...; data = ...; resource = ...; in { inherit variable data resource; module = lib.mapAttrs (server_name: _server_config: let in { # pin module version by nix flake inputs source = "github.com/numtide/nixos-anywhere?ref=${inputs.nixos-anywhere.sourceInfo.rev}/terraform/all-in-one"; ... content = lib.tfRef "jsonencode(${lib.strings.toJSON { # all variables # TF_VARS = lib.mapAttrs (k: _: lib.tfRef "jsonencode(var.${k})") variable; # non-sensitive variables TF_VARS = lib.mapAttrs (k: _: lib.tfRef "var.${k}") (lib.filterAttrs (_k: v: !(v ? sensitive && v.sensitive)) variable); TF_DATA = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "data.${type}.${k}") instances) data; TF_RESOURCES = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "resource.${type}.${k}") instances) resource; TF_SERVER = lib.tfRef "resource.hcloud_server.${server_name}"; SERVER_NAME = server_name; }})"; }) servers; } ``` You can then verify their contents from your `nixosConfigurations` like: ```nix lib.nixosSystem { inherit system; modules = [ { environment.etc."nixos-vars.json".source = ./nixos-vars.json; } ]; }; ``` -> `ls /etc/` -> `cat /etc/nixos-vars.json` (in practice one would instead probably parse these back to nix values to grab specific properties from, e.g. like: `lib.strings.fromJSON ./nixos-vars.json`)
This PR adds a Terraform input variable I so far named `content`. This allows passing in a string from Terraform to expose to nixos-anywhere's build step to make available as a file (default: `./nixos-vars.json`) within a NixOS configuration, as suggested by @Mic92 at nix-community#414. Note the file is staged even if added to gitignore, making this less suited for development in case the file includes ephemeral content. As a result, I would consider this approach complementary rather than as superseding nix-community#414. Example usage: ```nix let servers = ...; variable = ...; data = ...; resource = ...; in { inherit variable data resource; module = lib.mapAttrs (server_name: _server_config: let in { # pin module version by nix flake inputs source = "github.com/numtide/nixos-anywhere?ref=${inputs.nixos-anywhere.sourceInfo.rev}/terraform/all-in-one"; ... content = lib.tfRef "jsonencode(${lib.strings.toJSON { # all variables # TF_VARS = lib.mapAttrs (k: _: lib.tfRef "jsonencode(var.${k})") variable; # non-sensitive variables TF_VARS = lib.mapAttrs (k: _: lib.tfRef "var.${k}") (lib.filterAttrs (_k: v: !(v ? sensitive && v.sensitive)) variable); TF_DATA = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "data.${type}.${k}") instances) data; TF_RESOURCES = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "resource.${type}.${k}") instances) resource; TF_SERVER = lib.tfRef "resource.hcloud_server.${server_name}"; SERVER_NAME = server_name; }})"; }) servers; } ``` You can then verify their contents from your `nixosConfigurations` like: ```nix lib.nixosSystem { inherit system; modules = [ { environment.etc."nixos-vars.json".source = ./nixos-vars.json; } ]; }; ``` -> `ls /etc/` -> `cat /etc/nixos-vars.json` (in practice one would instead probably parse these back to nix values to grab specific properties from, e.g. like: `lib.strings.fromJSON ./nixos-vars.json`)
This PR adds a Terraform input variable I so far named `content`. This allows passing in a string from Terraform to expose to nixos-anywhere's build step to make available as a file (default: `./nixos-vars.json`) within a NixOS configuration, as suggested by @Mic92 at nix-community#414. Note the file is staged even if added to gitignore, making this less suited for development in case the file includes ephemeral content. As a result, I would consider this approach complementary rather than as superseding nix-community#414. Example usage: ```nix let servers = ...; variable = ...; data = ...; resource = ...; in { inherit variable data resource; module = lib.mapAttrs (server_name: _server_config: let in { # pin module version by nix flake inputs source = "github.com/numtide/nixos-anywhere?ref=${inputs.nixos-anywhere.sourceInfo.rev}/terraform/all-in-one"; ... content = lib.tfRef "jsonencode(${lib.strings.toJSON { # all variables # TF_VARS = lib.mapAttrs (k: _: lib.tfRef "jsonencode(var.${k})") variable; # non-sensitive variables TF_VARS = lib.mapAttrs (k: _: lib.tfRef "var.${k}") (lib.filterAttrs (_k: v: !(v ? sensitive && v.sensitive)) variable); TF_DATA = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "data.${type}.${k}") instances) data; TF_RESOURCES = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "resource.${type}.${k}") instances) resource; TF_SERVER = lib.tfRef "resource.hcloud_server.${server_name}"; SERVER_NAME = server_name; }})"; }) servers; } ``` You can then verify their contents from your `nixosConfigurations` like: ```nix lib.nixosSystem { inherit system; modules = [ { environment.etc."nixos-vars.json".source = ./nixos-vars.json; } ]; }; ``` -> `ls /etc/` -> `cat /etc/nixos-vars.json` (in practice one would instead probably parse these back to nix values to grab specific properties from, e.g. like: `lib.strings.fromJSON ./nixos-vars.json`)
This PR adds a Terraform input variable I so far named `content`. This allows passing in a string from Terraform to expose to nixos-anywhere's build step to make available as a file (default: `./nixos-vars.json`) within a NixOS configuration, as suggested by @Mic92 at nix-community#414. Note the file is staged even if added to gitignore, making this less suited for development in case the file includes ephemeral content. As a result, I would consider this approach complementary rather than as superseding nix-community#414. Example usage: ```nix let servers = ...; variable = ...; data = ...; resource = ...; in { inherit variable data resource; module = lib.mapAttrs (server_name: _server_config: let in { # pin module version by nix flake inputs source = "github.com/numtide/nixos-anywhere?ref=${inputs.nixos-anywhere.sourceInfo.rev}/terraform/all-in-one"; ... content = lib.tfRef "jsonencode(${lib.strings.toJSON { # all variables # TF_VARS = lib.mapAttrs (k: _: lib.tfRef "jsonencode(var.${k})") variable; # non-sensitive variables TF_VARS = lib.mapAttrs (k: _: lib.tfRef "var.${k}") (lib.filterAttrs (_k: v: !(v ? sensitive && v.sensitive)) variable); TF_DATA = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "data.${type}.${k}") instances) data; TF_RESOURCES = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "resource.${type}.${k}") instances) resource; TF_SERVER = lib.tfRef "resource.hcloud_server.${server_name}"; SERVER_NAME = server_name; }})"; }) servers; } ``` You can then verify their contents from your `nixosConfigurations` like: ```nix lib.nixosSystem { inherit system; modules = [ { environment.etc."nixos-vars.json".source = ./nixos-vars.json; } ]; }; ``` -> `ls /etc/` -> `cat /etc/nixos-vars.json` (in practice one would instead probably parse these back to nix values to grab specific properties from, e.g. like: `lib.strings.fromJSON ./nixos-vars.json`)
This PR adds a Terraform input variable I so far named `content`. This allows passing in a string from Terraform to expose to nixos-anywhere's build step to make available as a file (default: `./nixos-vars.json`) within a NixOS configuration, as suggested by @Mic92 at nix-community#414. Note the file is staged even if added to gitignore, making this less suited for development in case the file includes ephemeral content. As a result, I would consider this approach complementary rather than as superseding nix-community#414. Example usage: ```nix let servers = ...; variable = ...; data = ...; resource = ...; in { inherit variable data resource; module = lib.mapAttrs (server_name: _server_config: let in { # pin module version by nix flake inputs source = "github.com/numtide/nixos-anywhere?ref=${inputs.nixos-anywhere.sourceInfo.rev}/terraform/all-in-one"; ... content = lib.tfRef "jsonencode(${lib.strings.toJSON { # all variables # TF_VARS = lib.mapAttrs (k: _: lib.tfRef "jsonencode(var.${k})") variable; # non-sensitive variables TF_VARS = lib.mapAttrs (k: _: lib.tfRef "var.${k}") (lib.filterAttrs (_k: v: !(v ? sensitive && v.sensitive)) variable); TF_DATA = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "data.${type}.${k}") instances) data; TF_RESOURCES = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "resource.${type}.${k}") instances) resource; TF_SERVER = lib.tfRef "resource.hcloud_server.${server_name}"; SERVER_NAME = server_name; }})"; }) servers; } ``` You can then verify their contents from your `nixosConfigurations` like: ```nix lib.nixosSystem { inherit system; modules = [ { environment.etc."nixos-vars.json".source = ./nixos-vars.json; } ]; }; ``` -> `ls /etc/` -> `cat /etc/nixos-vars.json` (in practice one would instead probably parse these back to nix values to grab specific properties from, e.g. like: `lib.strings.fromJSON ./nixos-vars.json`)
This PR adds a Terraform input variable named `content`. This allows passing in a string from Terraform to expose to NixOS's run-time to make available as a file (default: `/etc/nixos-vars.json`) as suggested by @Mic92 at nix-community#414. This third iteration wraps the original `lib.nixosSystem` call to allow passing info without either use of `--impure` or having to stage to Git. Note the file is staged even if added to gitignore, making this less suited for development in case the file includes ephemeral content. As a result, I would consider this approach complementary rather than as superseding nix-community#414. Example usage: ```nix let servers = ...; variable = ...; data = ...; resource = ...; in { inherit variable data resource; module = lib.mapAttrs (server_name: _server_config: let in { # pin module version by nix flake inputs source = "github.com/numtide/nixos-anywhere?ref=${inputs.nixos-anywhere.sourceInfo.rev}/terraform/all-in-one"; ... content = lib.tfRef "jsonencode(${lib.strings.toJSON { # all variables # TF_VARS = lib.mapAttrs (k: _: lib.tfRef "jsonencode(var.${k})") variable; # non-sensitive variables TF_VARS = lib.mapAttrs (k: _: lib.tfRef "var.${k}") (lib.filterAttrs (_k: v: !(v ? sensitive && v.sensitive)) variable); TF_DATA = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "data.${type}.${k}") instances) data; TF_RESOURCES = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "resource.${type}.${k}") instances) resource; TF_SERVER = lib.tfRef "resource.hcloud_server.${server_name}"; SERVER_NAME = server_name; }})"; }) servers; } ``` You can then verify their contents from your `nixosConfigurations` like: `cat /etc/nixos-vars.json` However, so far I did not yet manage to reach my goal: - On this attempt I have so far just exposed the content to run-time, rather than to the intended NixOS build-time. To address this, I consider looking into injecting thru `specialArgs`, tho I am not sure yet this would work, and feel open to suggestions. - When putting the content file into `.gitignore`, the build currently errors on same NAR hash mismatch on the file.
This PR adds a Terraform input variable named `content`. This allows passing in a string from Terraform to expose to NixOS's run-time to make available as a file (default: `/etc/nixos-vars.json`) as suggested by @Mic92 at nix-community#414. This third implementation wraps the original `lib.nixosSystem` call to allow passing info without either use of `--impure` or having to stage to Git. Example usage: ```nix let servers = ...; variable = ...; data = ...; resource = ...; in { inherit variable data resource; module = lib.mapAttrs (server_name: _server_config: let in { # pin module version by nix flake inputs source = "github.com/numtide/nixos-anywhere?ref=${inputs.nixos-anywhere.sourceInfo.rev}/terraform/all-in-one"; ... special_args = lib.tfRef "jsonencode(${lib.strings.toJSON { tf = { inherit server_name; # all variables # var = lib.mapAttrs (k: _: lib.tfRef "jsonencode(var.${k})") variable; # non-sensitive variables var = lib.mapAttrs (k: _: lib.tfRef "var.${k}") (lib.filterAttrs (_k: v: !(v ? sensitive && v.sensitive)) variable); data = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "data.${type}.${k}") instances) data; resource = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "resource.${type}.${k}") instances) resource; server = lib.tfRef "resource.hcloud_server.${server_name}"; }; }})"; }) servers; } ``` You can then use these in your `nixosConfigurations` thru the `tf` argument. Status: - [ ] resolve occasional NAR hash mismatches - [ ] Test putting the content file into `.gitignore`.
This PR adds a Terraform input variable named `content`. This allows passing in a string from Terraform to expose to NixOS's run-time to make available as a file (default: `/etc/nixos-vars.json`) as suggested by @Mic92 at nix-community#414. This third implementation wraps the original `lib.nixosSystem` call to allow passing info without either use of `--impure` or having to stage to Git. Example usage: ```nix let servers = ...; variable = ...; data = ...; resource = ...; in { inherit variable data resource; module = lib.mapAttrs (server_name: _server_config: let in { # pin module version by nix flake inputs source = "github.com/numtide/nixos-anywhere?ref=${inputs.nixos-anywhere.sourceInfo.rev}/terraform/all-in-one"; ... special_args = lib.tfRef "jsonencode(${lib.strings.toJSON { tf = { inherit server_name; # all variables # var = lib.mapAttrs (k: _: lib.tfRef "jsonencode(var.${k})") variable; # non-sensitive variables var = lib.mapAttrs (k: _: lib.tfRef "var.${k}") (lib.filterAttrs (_k: v: !(v ? sensitive && v.sensitive)) variable); data = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "data.${type}.${k}") instances) data; resource = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "resource.${type}.${k}") instances) resource; server = lib.tfRef "resource.hcloud_server.${server_name}"; }; }})"; }) servers; } ``` You can then use these in your `nixosConfigurations` thru the `tf` argument. Status: - [ ] Resolve occasional NAR hash mismatches - [ ] Test putting the content file into `.gitignore`. - [ ] Let the user pass in a TF map rather than string.
This PR adds a Terraform input variable named `special_args`. This allows passing in a string from Terraform to expose to NixOS's `specialArgs` at build-time. This implementation extends the original `lib.nixosSystem` call to allow passing info without either use of `--impure` or having to stage to Git, thanks to @Mic92's suggestion at nix-community#414. Example usage: ```nix let servers = ...; variable = ...; data = ...; resource = ...; in { inherit variable data resource; module = lib.mapAttrs (server_name: _server_config: let in { # pin module version by nix flake inputs source = "github.com/numtide/nixos-anywhere?ref=${inputs.nixos-anywhere.sourceInfo.rev}/terraform/all-in-one"; ... special_args = lib.tfRef "jsonencode(${lib.strings.toJSON { tf = { inherit server_name; # all variables # var = lib.mapAttrs (k: _: lib.tfRef "var.${k}") variable; # non-sensitive variables var = lib.mapAttrs (k: _: lib.tfRef "var.${k}") (lib.filterAttrs (_k: v: !(v ? sensitive && v.sensitive)) variable); data = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "data.${type}.${k}") instances) data; resource = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "resource.${type}.${k}") instances) resource; server = lib.tfRef "resource.hcloud_server.${server_name}"; }; }})"; }) servers; } ``` You can then use these in your `nixosConfigurations`, in this example thru the `tf` argument.
This PR adds a Terraform input variable named `special_args`. This allows passing in a string from Terraform to expose to NixOS's `specialArgs` at build-time. This implementation extends the original `lib.nixosSystem` call to allow passing info without either use of `--impure` or having to stage to Git, thanks to @Mic92's suggestion at nix-community#414. Example usage: ```nix let servers = ...; variable = ...; data = ...; resource = ...; in { inherit variable data resource; module = lib.mapAttrs (server_name: _server_config: let in { # pin module version by nix flake inputs source = "github.com/numtide/nixos-anywhere?ref=${inputs.nixos-anywhere.sourceInfo.rev}/terraform/all-in-one"; ... special_args = lib.tfRef "jsonencode(${lib.strings.toJSON { tf = { inherit server_name; # all variables # var = lib.mapAttrs (k: _: lib.tfRef "var.${k}") variable; # non-sensitive variables var = lib.mapAttrs (k: _: lib.tfRef "var.${k}") (lib.filterAttrs (_k: v: !(v ? sensitive && v.sensitive)) variable); data = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "data.${type}.${k}") instances) data; resource = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "resource.${type}.${k}") instances) resource; server = lib.tfRef "resource.hcloud_server.${server_name}"; }; }})"; }) servers; } ``` You can then use these in your `nixosConfigurations`, in this example thru the `tf` argument.
This PR adds a Terraform input variable named `special_args`. This allows passing in a JSON string from Terraform to expose to NixOS's `specialArgs` at build-time. This implementation extends the original `lib.nixosSystem` call to allow passing info without either use of `--impure` or having to stage to Git, thanks to @Mic92's suggestion at nix-community#414. Example usage: ```nix let servers = ...; variable = ...; data = ...; resource = ...; in { inherit variable data resource; module = lib.mapAttrs (server_name: _server_config: let in { # pin module version by nix flake inputs source = "github.com/numtide/nixos-anywhere?ref=${inputs.nixos-anywhere.sourceInfo.rev}/terraform/all-in-one"; ... special_args = lib.tfRef "jsonencode(${lib.strings.toJSON { tf = { inherit server_name; # all variables # var = lib.mapAttrs (k: _: lib.tfRef "var.${k}") variable; # non-sensitive variables var = lib.mapAttrs (k: _: lib.tfRef "var.${k}") (lib.filterAttrs (_k: v: !(v ? sensitive && v.sensitive)) variable); data = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "data.${type}.${k}") instances) data; resource = lib.mapAttrs (type: instances: lib.mapAttrs (k: _: tfRef "resource.${type}.${k}") instances) resource; server = lib.tfRef "resource.hcloud_server.${server_name}"; }; }})"; }) servers; } ``` You can then use these in your `nixosConfigurations`, in this example thru the `tf` argument.
in order to prevent duplication and hard-coding, it would be nice if we had a way to pass info from TF to the nixos configurations.
before nixos-anywhere i used teraflops, which exposed a static TF
show -json
dump thru aspecialArgs
parameterresources
.now, they can use such a static dump as they deploy the nix bits after terraform. as nixos-anywhere deploys nixos during terraform execution, here such info would instead need to be passed from within the terraform.
this could be facilitated in different ways, e.g. opinionated like teraflops (exposing such info in a given format), versus more generically by just passing thru more nix arguments, while leaving the details to the user.
there would also be considerations there between exposing info in a pure vs impure way:
The text was updated successfully, but these errors were encountered: