Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Feature/simon hoenscheid 603 restructure repository management #605

Closed
wants to merge 12 commits into from
Closed

WIP: Feature/simon hoenscheid 603 restructure repository management #605

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
6eeb0b0
added new data structure
SimonHoenscheid Sep 27, 2020
fcf3f4c
exemple repo handling for debian 10
SimonHoenscheid Sep 27, 2020
f25392b
document class
SimonHoenscheid Sep 27, 2020
1345bab
added newlines
SimonHoenscheid Sep 27, 2020
7b3bcd4
added arch linux hiera structure
SimonHoenscheid Sep 28, 2020
c918c71
* added documentation, * added requested changes from PR
SimonHoenscheid Sep 28, 2020
9c58a9d
updated .travis.yml
SimonHoenscheid Sep 28, 2020
b60982f
fix syntax test
SimonHoenscheid Sep 28, 2020
6d451aa
updated documentatation to puppet strings, sorted params, updated logic
SimonHoenscheid Sep 28, 2020
7e4a9b0
added classes for repo mgmt, updated metadata.json
SimonHoenscheid Sep 28, 2020
e7a10a6
Update hiera.yaml
SimonHoenscheid Sep 29, 2020
fb9c572
fix metadata.json, make condition easier to understand
SimonHoenscheid Oct 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ jobs:
branches:
only:
- master
- module_rewrite
- /^v\d/
notifications:
email: false
Expand Down
1 change: 1 addition & 0 deletions data/Archlinux/rolling.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/CentOS/6.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/CentOS/7.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/CentOS/8.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
25 changes: 25 additions & 0 deletions data/Debian/10.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
php::external_repo: false
php::php_version: '7.3'
php::os_supported_php_versions:
- '7.3'

php::external_repo_supported_php_versions:
- '5.6'
- '7.0'
- '7.1'
- '7.2'
- '7.3'
- '7.4'

php::external_repo_details:
'source_php_sury':
comment: 'the debian sury repo sury.org'
location: 'https://packages.sury.org/php/'
repos: 'main'
include:
src: true
deb: true
key:
id: '15058500A0235D97F5D10063B188E2B695BD4743'
source: 'https://packages.sury.org/php/apt.gpg'
1 change: 1 addition & 0 deletions data/Debian/9.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/Fedora/31.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/Fedora/32.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/Fedora/33.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/FreeBSD/11.3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/FreeBSD/11.4.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/FreeBSD/12.1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/OpenSuSE/15.1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/OpenSuSE/15.2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/OpenSuSE/15.3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/RedHat/6.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/RedHat/7.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/RedHat/8.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/SLES/11.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/SLES/12.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/SLES/15.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/Ubuntu/16.04.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/Ubuntu/18.04.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions data/Ubuntu/20.04.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
2 changes: 0 additions & 2 deletions data/default.yaml → data/common.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
---

lookup_options:
php::fpm::pools:
merge: first

11 changes: 9 additions & 2 deletions hiera.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
---
version: 5
defaults:
datadir: 'data'
data_hash: 'yaml_data'

hierarchy:
- name: default.yaml
- name: 'My Hierachie'
SimonHoenscheid marked this conversation as resolved.
Show resolved Hide resolved
paths:
- 'default.yaml'
- '%{facts.os.name}/%{facts.os.distro.release.major}.yaml'
- '%{facts.os.name}/%{facts.os.release.full}.yaml'
- '%{facts.os.name}/%{facts.os.release.major}.yaml'
- 'common.yaml'
46 changes: 46 additions & 0 deletions manifests/init.pp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,43 @@
# If set to false, a restart will be executed instead of a reload.
# This default will be changed in a future release.
#
#########
######### REWRITE
#
# @param [Boolean] manage_external_repo
# manage an external package repository or not
# @option external_repo [Boolean] :false
# default
# @option external_repo [Boolean] :true
#
# @param [Hash] external_repo_details
# The PHP versions the additional repositories support
# @option external_repo_details [Hash] :{}
# default: empty Hash, default for OS from Hiera Data in module
#
# @param [Array] external_repo_supported_php_versions
# The PHP versions the additional repositories support
# @option external_repo_supported_php_versions [Array] :[]
# default: empty array, default for OS from Hiera Data in module
#
# @param [Boolean] manage_internal_repo
# manage an internal package repository, parameter is to control
# app streams in CENTOS/RHEL/Fedora, Repos on FreeBSD or Archlinux
# @option external_repo [Boolean] :false
# true
# @option external_repo [Boolean] :true
#
# @param [Array] os_supported_php_versions
# The PHP versions the OS supports by default, without any additional repositories,
# on RHEL/CentOS and Fedora this includes app streams, on FreeBSD or Archlinux it manages Repos
# @option os_supported_php_versions [Array] :[]
# default: empty array, default for OS from Hiera Data in module
#
# @param [String] php_version
# Set the PHP version to install
# @option php_version [String] :'7.4'
# defaults: latest stable PHP Version available on php.net, default for OS from Hiera Data in module
#
class php (
String $ensure = $php::params::ensure,
Boolean $manage_repos = $php::params::manage_repos,
Expand Down Expand Up @@ -169,6 +206,15 @@
String $log_group = $php::params::fpm_group,
Boolean $pool_purge = $php::params::pool_purge,
Boolean $reload_fpm_on_config_changes = true,
# added for refactoring
Boolean $manage_external_repo = false,
Array $external_repo_supported_php_versions = [],
Hash $external_repo_details = {},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • should maybe "custom repo detail" or "repo config" or "repo settings" or simply "repository" (is it real relevant if it is a external or internal repository? isn't it "here we have custom repository config" (i can be the default debian repo where i only change a key hash)

  • should be a custom type like

type Profile::Apt::Source = Struct[{
    Optional[location] =>     Optional[String[1]],
    Optional[comment] =>      String[1],
    Optional[ensure] =>       Optional[Enum['file', 'present', 'absent']],
    Optional[release] =>      Optional[String],
    Optional[repos] =>        String[1],
    Optional[include] =>      Optional[Struct[{
                                Optional[src] => Boolean,
                                Optional[deb] => Boolean,
                              }]],
    Optional[key] =>          Optional[Struct[{
                                Optional[id] => String[1],
                                Optional[source] => String[1],
                              }]],
    Optional[pin] =>          Optional[Struct[{
                                Optional[explanation] => String[1],
                                Optional[packages] => String[1],
                                Optional[origin] => String[1],
                                Optional[priority] => Integer,
                              }]],
    Optional[architecture] =>     Optional[String],
    Optional[allow_unsigned] =>     Boolean,
    Optional[notify_update] =>     Boolean,

    # Optional[comment]  => String[1],
    # location => 'https://packages.gitlab.com/runner/gitlab-runner/debian/',
    # release  => 'stretch',
    # repos    => 'main',
    # pin => {
    #   explanation => 'Prefer GitLab provided packages over the Debian native ones',
    #   packages => 'gitlab-runner',
    #   origin => 'packages.gitlab.com',
    #   priority => 1001,},
    # key      => {
    #   'id'  => '1A4C919DB987D435939638B914219A96E15E78F4',
    #   'source' => 'https://packages.gitlab.com/runner/gitlab-runner/gpgkey',
    # },
    # include  => {
    #   'src' => true,
    #   'deb' => true,
    # },
}]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this work for every repo type?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"yes" we would have to write a type for each repo stucture we want to support.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add this type to the module

Boolean $manage_internal_repo = true,
Array $os_supported_php_versions = [],
String[1] $php_version = '7.4', # TODO: validate a version string here?


) inherits php::params {
$real_fpm_package = pick($fpm_package, "${package_prefix}${php::params::fpm_package_suffix}")

Expand Down
75 changes: 48 additions & 27 deletions manifests/repo.pp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,32 +1,53 @@
# Configure package repository
# @api private
# @summary Configure package repository
#
class php::repo {
$msg_no_repo = "No repo available for ${facts['os']['family']}/${facts['os']['name']}"
# @param [Boolean] manage_external_repo
# manage an external package repository or not
# @option external_repo [Boolean] :false
# default
# @option external_repo [Boolean] :true
#
# @param [Array] external_repo_supported_php_versions
# The PHP versions the additional repositories support
# @option external_repo_supported_php_versions [Array] :[]
# default: empty array, default for OS from Hiera Data in module
#
# @param [Boolean] manage_internal_repo
# manage an internal package repository, parameter is to control
# app streams in CENTOS/RHEL/Fedora, Repos on FreeBSD or Archlinux
# @option external_repo [Boolean] :false
# true
# @option external_repo [Boolean] :true
#
# @param [Array] os_supported_php_versions
# The PHP versions the OS supports by default, without any additional repositories,
# on RHEL/CentOS and Fedora this includes app streams, on FreeBSD or Archlinux it manages Repos
# @option os_supported_php_versions [Array] :[]
# default: empty array, default for OS from Hiera Data in module
#
# @param [String] php_version
# Set the PHP version to install
# @option php_version [String] :'7.4'
# defaults: latest stable PHP Version available on php.net, default for OS from Hiera Data in module
#
class php::repo (
Boolean $manage_internal_repo = $php::manage_internal_repo,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if we need internal and external as definition. isn't it simply repository? why we need to differ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my opinion yes, because there are operating systems where we need to manage internal and external repos (RHEL/CENTOS/Fedora) app channel/Remi and if I want to have different data without interpolation this seems to be the way to go

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't know about centos & co, you manage the internal and the external repo for php?

is it relevant if it is external or internal? isn't it just multiple repositories?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its been a while since I worked with Fedora based systems, it only affects RHEL8/CENTOS8 and Fedora IDK which version RHEL8 was branched from +. For me this seems important, because it will prevent users declaring internal and external php versions/sources parallel.

Boolean $manage_external_repo = $php::manage_external_repo,
Optional[Array] $external_repo_supported_php_versions = $php::external_repo_supported_php_versions,
Array $os_supported_php_versions = $php::os_supported_php_versions,
Optional[String[1]] $php_version = $php::php_version,

case $facts['os']['family'] {
'Debian': {
# no contain here because apt does that already
case $facts['os']['name'] {
'Debian': {
include php::repo::debian
}
'Ubuntu': {
include php::repo::ubuntu
}
default: {
fail($msg_no_repo)
}
}
}
'FreeBSD': {}
'Suse': {
contain php::repo::suse
}
'RedHat': {
contain 'php::repo::redhat'
}
default: {
fail($msg_no_repo)
) {
assert_private()

if (($manage_internal_repo or $manage_external_repo) and (($php_version in $external_repo_supported_php_versions) or ($php_version in $os_supported_php_versions))) {
case $facts['os']['name'] {
'Archlinux': { contain 'php::repo::archlinux' }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for arch and freebsd, should we just do nothing? since those don't have external repositories.

Suggested change
'Archlinux': { contain 'php::repo::archlinux' }
'Archlinux': { }

Copy link
Member Author

@SimonHoenscheid SimonHoenscheid Sep 29, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The classes are for generally managing repos, is there no need to connect a specific one, for a specific php version? Kind of the same problem exists for RHEL 8 style distros, there is the need to activate a specific app stream

Copy link
Member Author

@SimonHoenscheid SimonHoenscheid Oct 3, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will implement this

'FreeBSD': { contain 'php::repo::freebsd' }
'RedHat', 'CentOS', 'Fedora': { contain 'php::repo::redhat' }
'Debian', 'Ubuntu': { contain 'php::repo::debian' }
'OpenSuSE', 'SLES': { contain 'php::repo::suse' }
default: { contain 'php::repo::fallback' }
}
}
}
13 changes: 13 additions & 0 deletions manifests/repo/archlinux.pp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# @api private
# @summary Configure archlinux repo
#
# @param [Hash] external_repo_details
# The PHP versions the additional repositories support
# @option external_repo_details [Hash] :{}
# default: empty Hash, default for OS from Hiera Data in module
#
class php::repo::archlinux (
Hash[String[1], Hash] $external_repo_details = $php::external_repo_details,
) {
assert_private()
}
66 changes: 10 additions & 56 deletions manifests/repo/debian.pp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,67 +1,21 @@
# Configure debian apt repo
# @api private
# @summary Configure debian apt repo
#
# === Parameters
#
# [*location*]
# Location of the apt repository
#
# [*release*]
# Release of the apt repository
#
# [*repos*]
# Apt repository names
#
# [*include_src*]
# Add source source repository
#
# [*key*]
# Public key in apt::key format
#
# [*dotdeb*]
# Enable special dotdeb handling
#
# [*sury*]
# Enable special sury handling
# @param [Hash] external_repo_details
# The PHP versions the additional repositories support
# @option external_repo_details [Hash] :{}
# default: empty Hash, default for OS from Hiera Data in module
#
class php::repo::debian (
$location = 'https://packages.dotdeb.org',
$release = 'wheezy-php56',
$repos = 'all',
$include_src = false,
$key = {
'id' => '6572BBEF1B5FF28B28B706837E3F070089DF5277',
'source' => 'http://www.dotdeb.org/dotdeb.gpg',
},
$dotdeb = true,
$sury = true,
Hash[String[1], Hash] $external_repo_details = $php::external_repo_details,
) {
assert_private()

include 'apt'

apt::source { "source_php_${release}":
location => $location,
release => $release,
repos => $repos,
include => {
'src' => $include_src,
'deb' => true,
},
key => $key,
}

if ($sury and $php::globals::php_version in ['7.1','7.2']) {
apt::source { 'source_php_sury':
location => 'https://packages.sury.org/php/',
repos => 'main',
include => {
'src' => $include_src,
'deb' => true,
},
key => {
id => '15058500A0235D97F5D10063B188E2B695BD4743',
source => 'https://packages.sury.org/php/apt.gpg',
},
$external_repo_details.each |String $repository, Hash $repository_details| {
apt::source { $repository:
* => $repository_details,
}
}
}
8 changes: 8 additions & 0 deletions manifests/repo/fallback.pp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# @api private
# @summary the fallback class, throws an error, if called
class php::repo::fallback () {
assert_private()
$msg_no_repo = "No repo available for ${facts['os']['family']}/${facts['os']['name']}"

fail($msg_no_repo)
}
12 changes: 12 additions & 0 deletions manifests/repo/freebsd.pp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# @api private
# @summary Configure Freebsd repos
#
# @param [Hash] external_repo_details
# The PHP versions the additional repositories support
# @option external_repo_details [Hash] :{}
# default: empty Hash, default for OS from Hiera Data in module
class php::repo::freebsd (
Hash[String[1], Hash] $external_repo_details = $php::external_repo_details,
) {
assert_private()
}
Loading