-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
(wip) some patches to get things working with puppet-unbound #1
base: add-hiera-overrides
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
# frozen_string_literal: true | ||
|
||
module PuppetStrings | ||
module Hiera | ||
require_relative 'hiera/hierarchy_data_path' | ||
require_relative 'hiera/data' | ||
|
||
def self.load_config | ||
PuppetStrings::Hiera::Data.new('hiera.yaml') | ||
end | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,132 @@ | ||
# frozen_string_literal: true | ||
|
||
module PuppetStrings::Hiera | ||
class Data | ||
attr_reader :config_path, :interpolated_paths, :uninterpolated_paths, :defaults | ||
|
||
def initialize(config_path) | ||
@config_path = config_path | ||
@interpolated_paths = [] | ||
# This will probably always be ['common.yaml'] jut make it an array just incase | ||
@uninterpolated_paths = [] | ||
|
||
load_config | ||
end | ||
|
||
def files | ||
@files ||= begin | ||
result = {} | ||
|
||
interpolated_paths.each do |dp| | ||
dp.matches.each do |file, interpolations| | ||
unless result.key?(file) | ||
result[file] = interpolations | ||
end | ||
end | ||
end | ||
|
||
result | ||
end | ||
end | ||
|
||
# @return [Hash[String, Hash[String, Any]]] | ||
# Full variable (class::var) -> filename: value | ||
def overrides | ||
@overrides ||= begin | ||
overrides = {} | ||
files.each_key do |file| | ||
data = YAML.load(File.read(file)) | ||
data.each do |key, value| | ||
overrides[key] ||= {} | ||
overrides[key][file] = value | ||
end | ||
end | ||
overrides | ||
end | ||
end | ||
|
||
# @return [Hash[String, Hash[String, Any]]] | ||
# Full variable (class::var) -> filename: value | ||
def defaults | ||
@defaults ||= begin | ||
defaults = {} | ||
uninterpolated_paths.each do |file| | ||
data = YAML.load(File.read(file)) | ||
data.each do |key, value| | ||
defaults[key] = value.nil? ? 'undef' : value.inspect | ||
end | ||
end | ||
defaults | ||
end | ||
end | ||
|
||
# @return [Hash[String, Hash[String, Any]]] | ||
# variable -> filename: value | ||
def overrides_for_class(class_name) | ||
filter_mappings(class_name, overrides) | ||
end | ||
|
||
# @return [Hash[String, Hash[String, Any]]] | ||
# variable -> filename: value | ||
def defaults_for_class(class_name) | ||
filter_mappings(class_name, defaults) | ||
end | ||
|
||
def to_s | ||
config_path | ||
end | ||
|
||
private | ||
|
||
# @return [Hash[String, Hash[String, Any]]] | ||
# variable -> filename: value | ||
def filter_mappings(class_name, mappings) | ||
result = {} | ||
mappings.each do |key, value| | ||
mapped_class_name, _, variable = key.rpartition('::') | ||
if mapped_class_name == class_name | ||
result[variable] = value | ||
end | ||
end | ||
result | ||
end | ||
|
||
# TODO: this should be a class method not an instance method | ||
def load_config | ||
return unless File.exist?(config_path) | ||
|
||
config = YAML.load(File.read(config_path)) | ||
|
||
unless config['version'] == 5 | ||
log.warn("Unsupported version '#{config['version']}'") | ||
return | ||
end | ||
|
||
hierarchy = config['hierarchy'] | ||
return unless hierarchy | ||
|
||
hierarchy.each do |level| | ||
data_hash = level['data_hash'] || config['defaults']['data_hash'] | ||
next unless data_hash == 'yaml_data' | ||
|
||
datadir = level['datadir'] || config['defaults']['datadir'] | ||
|
||
if level['path'] | ||
if level['path'] =~ /%{[^}]+}/ | ||
interpolated_paths << PuppetStrings::Hiera::HierarchyDataPath.new(datadir, level['path']) | ||
else | ||
uninterpolated_paths << File.join(datadir, level['path']) | ||
end | ||
elsif level['paths'] | ||
level['paths'].each do |path| | ||
if path =~ /%{[^}]+}/ | ||
interpolated_paths << PuppetStrings::Hiera::HierarchyDataPath.new(datadir, path) | ||
else | ||
uninterpolated_paths << File.join(datadir, path) | ||
end | ||
end | ||
end | ||
end | ||
end | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
# frozen_string_literal: true | ||
|
||
module PuppetStrings::Hiera | ||
class HierarchyDataPath | ||
attr_reader :datadir, :path, :regex, :mapping | ||
|
||
def initialize(datadir, path) | ||
@datadir = datadir | ||
@path = path | ||
@regex, @mapping = HierarchyDataPath.path2regex(path) | ||
end | ||
|
||
def matches | ||
result = {} | ||
|
||
Dir.glob(File.join(datadir, '**', '*.yaml')).each do |entry| | ||
next unless File.file?(entry) | ||
|
||
regex.match(entry) do |match| | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I had the chdir there so the globbed paths were relative to There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. hmm im not sure thats the best idea. 1) im not convinced there is a problem to solve (do you have some examples, even contrived ones) 2) if there is a problem at best we are just reducing the risk of triggering it by using chdir There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The alternative is to normalize it again. For example (using pathname): data_dir = Pathname.new(datadir)
data_dir.glob('**').each do |entry|
next unless x.file?
regex.match(entry.relative_path_from(data_dir)) do |match|
# ...
end
end Note that I also chose to glob everything so that we don't have to think about extensions (yaml vs yml). The regex matching will take care of that. |
||
interpolations = {} | ||
|
||
mapping.each do |name, interpolation| | ||
interpolations[interpolation] = match.named_captures[name] | ||
end | ||
|
||
result[entry] = interpolations | ||
end | ||
end | ||
|
||
result | ||
end | ||
|
||
def self.path2regex(path) | ||
mapping = {} | ||
|
||
intermediate_result = path | ||
|
||
# First pass - intermediate replacements | ||
path.scan(/%{[^}]+}/).each_with_index do |interpolation, i| | ||
replacement = "X_INTERPOLATION_#{i}_X" | ||
mapping[replacement] = interpolation[2..-2] | ||
intermediate_result = intermediate_result.sub(interpolation, replacement) | ||
end | ||
|
||
# Second pass - escape any special chars | ||
escaped = Regexp.escape(intermediate_result) | ||
|
||
# Third pass - replacement intermediates with regex | ||
mapping.each_key do |replacement| | ||
escaped = escaped.sub(replacement, "(?<#{replacement}>.+)") | ||
end | ||
|
||
[Regexp.new(escaped), mapping] | ||
end | ||
|
||
def to_s | ||
File.join(datadir, path) | ||
end | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -81,6 +81,19 @@ Options: | |
<% if defaults && defaults[param[:name]] -%> | ||
Default value: `<%= value_string(defaults[param[:name]]) %>` | ||
|
||
<% end -%> | ||
<% if !hiera_overrides.empty? && hiera_overrides[param[:name]] -%> | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I didn't have the empty check because in Ruby a missing key in a hash returns nil so it wouldn't be triggered. I think this change is redundant. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
<details> | ||
<summary>Hiera overrides in a detailed table</summary> | ||
|
||
| Filename | Interpolations | Value | | ||
|----------|----------------|-------| | ||
<% hiera_overrides[param[:name]].each do |filename, interpolations, value| -%> | ||
| `<%= filename %>` | <%= interpolations.map { |i, v| "`#{i}`: `#{v}`" }.join("<br>") %> | `<%= value %>` | | ||
<% end -%> | ||
|
||
</details> | ||
|
||
<% end -%> | ||
<% end -%> | ||
<% end -%> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think this achieves "Graceful fallback if legacy Hiera data is used" although see the todo above, currently we load the config for every instance