Skip to content

Commit

Permalink
Fix issues 230, 232, 235, and 249
Browse files Browse the repository at this point in the history
* Moved defaults to module data, and removed the params class
* Privatized all `splunk::enterprise::` and `splunk::forwarder::`
  install, config, and service classes
* Added a `$release` param, which replaces the `$version` param
  * For ensurable package_providers, the release is used as the
    Splunk package ensure, if specified
  * The release no longer defaults to a specific version and build,
    instead, the Splunk package resource defaults ensure to 'installed'
  * Added a Splunk::Release type
* Added a service_ensure param, per voxpupuli#249
* Modified splunk*_version facts to be part of splunkforwarder and
  splunkenterprise fact hashes
* Removed init.pp, which only served to confuse
* `$[enterprise,forwarder]_package_src` and `$package_source` params
  renamed to `$managed_package_source` and `$unmanaged_package_source`,
  for clarity
* Fixed: enterprise and forwarder password classes cross-referenced params

Fixed voxpupuli#230
Fixed voxpupuli#232
Fixed voxpupuli#235
Fixed voxpupuli#249
  • Loading branch information
nick-markowski committed Jul 31, 2019
1 parent 6e54f1a commit 62c9b97
Show file tree
Hide file tree
Showing 39 changed files with 1,129 additions and 1,005 deletions.
128 changes: 65 additions & 63 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,17 @@ or apt to install these components if they're self-hosted.

### What splunk affects

* Installs the Splunk/Forwarder package and manages their config files. It does
not purge them by default.
* Installs the splunk or splunkforwarder package and manages their config
files. It does not purge them by default.
* The module will set up both Splunk Enterprise and Splunk Forwarder to run as
the 'root' user on POSIX platforms.
* By default, enables Splunk Enterprise and Splunk Forwarder boot-start, and
uses the vendor-generated service file to manage the splunk service.

### Setup Requirements

#### Module Installation

To begin using this module, use the Puppet Module Tool (PMT) from the command
line to install this module:

Expand All @@ -62,19 +64,26 @@ the --target-dir directive.
You can also use r10k or code-manager to deploy the module so ensure that you
have the correct entry in your Puppetfile.

Once the module is in place, there is just a little setup needed.
#### Package Dependencies

Once the module is in place, you will need to ensure the splunk package(s) are
available.

If your environment has the splunk package(s) available, and the supplied
`package_provider` supports it, it is not required for you to manage the splunk
packages.

First, you will need to place your downloaded splunk installers into the files
Otherwise, you will need to place your downloaded splunk installers into the files
directory, `<module_path>/splunk/files/`. If you're using r10k or code-manager
you'll need to override the `splunk::params::src_root` parameter to point at a
modulepath outside of the Splunk module because each deploy will overwrite the
files.
you'll need to override the `splunk::enterprise::src_root` or
`splunk::forwarder::src_root` parameter to point at a modulepath outside of the
Splunk module because each deploy will overwrite the files.

The files must be placed according to directory structure example given below.

The expected directory structure is:

$root_url/
$src_root/
└── products/
├── universalforwarder/
│ └── releases/
Expand All @@ -89,7 +98,7 @@ The expected directory structure is:

A semi-populated example files directory might then contain:

$root_url/
$src_root/
└── products/
├── universalforwarder/
│ └── releases/
Expand All @@ -109,43 +118,69 @@ A semi-populated example files directory might then contain:
├── splunk-7.0.0-c8a78efdd40f-linux-2.6-intel.deb
└── splunk-7.0.0-c8a78efdd40f-linux-2.6-x86_64.rpm

Second, you will need to supply the `splunk::params` class with three critical
pieces of information.

* The version of Splunk you are using
* The build of Splunk you are using
* The root URL to use to retrieve the packages

In the example given above, the version is 7.0.0, the build is c8a78efdd40f,
and the root URL is puppet:///modules/splunk. See the splunk::params class
documentation for more information.

### Beginning with splunk

Once the Splunk packages are hosted in the users repository or hosted by the
Puppet Server in the modulepath the module is ready to deploy.

## Usage

If a user is installing Splunk Enterprise with packages provided from their
modulepath, this is the most basic way of installing Splunk Server with default
settings:
### Splunk Enterprise

If splunk is already installed on the target node, the splunk `version` and
`build` will be determined by the `splunkenterprise` fact. You can simply
include the module on your node:

```puppet
include ::splunk::enterprise
include splunk::enterprise
```

This is the most basic way of installing the Splunk Universal Forwarder with
default settings:
Otherwise, if splunk is not installed, you will need to follow the instructions
in Setup Requirements. You may need to specify `src_root` if the defaults are
not suitable for your environment. You will also need to specify `release`,
in the format `version-build` as follows:

```yaml
---
splunk::enterprise::release: '7.0.0-c8a78efdd40f'
```
```puppet
class { '::splunk::params':
server => $my_splunk_server,
}
include splunk::enterprise
```

### Splunk Forwarder

If splunkforwarder is already installed on the target node, the splunk
`version` and `build` will be determined by the `splunkforwarder` fact. You
will only need to specify a `server`:

```yaml
---
splunk::forwarder::server: <your.servers.ip.addr>
```
```puppet
include ::splunk::forwarder
```

Otherwise, if splunkforwarder is not installed, you will need to follow the
instructions in Setup Requirements. You may need to specify `src_root` if the
defaults are not suitable for your environment. You will also need to specify
`release`, in the format `version-build` as follows:

```yaml
---
splunk::forwarder::release: '7.0.0-c8a78efdd40f'
splunk::forwarder::server: <your.servers.ip.addr>
```
```puppet
include ::splunk::forwarder
```

### Splunk Types

Once both Splunk Enterprise and Splunk Universal Forwarder have been deployed
on their respective nodes, the Forwarder is ready to start sending logs.

Expand Down Expand Up @@ -181,47 +216,14 @@ Alternatively the the `splunk::enterprise::password::seed` class can be used ind

### Upgrade splunk and splunkforwarder packages

This module has the ability to install *and* upgrade the splunk and splunkforwarder packages. All you have to do is declare `package_ensure => 'latest'` when calling the `::splunk` or `::splunk::forwarder` classes.

Upgrades from 7.0.X to >= 7.0.X are not tested.
Upgrades have not been tested with this module.

#### Upgrade Example

The following code will install the 6.6.8 version of the splunk forwarder. Then
comment out the 6.6.8 version and build values and uncomment the 7.1.2 version
and build values. Running puppet again will perform the following:

1. splunk forwarder package is upgraded
1. splunk service is stopped as part of the package upgrade process
1. new license agreement is automatically accepted
1. license agreement must be accepted or the splunk service will fail to start
1. splunk service is started

```puppet
# Tell the module to get packages directly from Splunk.
class { '::splunk::params':
version => '6.6.8',
build => '6c27a8439c1e',
#version => '7.1.2',
#build => 'a0c72a66db66',
src_root => 'https://download.splunk.com',
}
# Specifying package_ensure => 'latest' will ensure that the splunk and
# splunkforwarder packages will be upgraded when you specify newer values for
# version and build.
class { '::splunk::forwarder':
package_ensure => 'latest',
}
```
## Reference

See in file [REFERENCE.md](REFERENCE.md).

## Limitations

- Currently tested manually on Centos 7, but we will eventually add automated
testing and are targeting compatibility with other platforms.
- Tested with Puppet 5.x
- New installations of splunk up to version 7.2.X are supported, but upgrades
from 7.0.X to >= 7.0.X are not fully tested
Expand Down
57 changes: 57 additions & 0 deletions data/common.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
splunk::enterprise::manage_password: false
splunk::enterprise::seed_password: false
splunk::enterprise::reset_seeded_password: false
splunk::enterprise::secret: 'hhy9DOGqli4.aZWCuGvz8stcqT2/OSJUZuyWHKc4wnJtQ6IZu2bfjeElgYmGHN9RWIT3zs5hRJcX1wGerpMNObWhFue78jZMALs3c3Mzc6CzM98/yGYdfcvWMo1HRdKn82LVeBJI5dNznlZWfzg6xdywWbeUVQZcOZtODi10hdxSJ4I3wmCv0nmkSWMVOEKHxti6QLgjfuj/MOoh8.2pM0/CqF5u6ORAzqFZ8Qf3c27uVEahy7ShxSv2K4K41z'
splunk::enterprise::password_hash: '$6$pIE/xAyP9mvBaewv$4GYFxC0SqonT6/x8qGcZXVCRLUVKODj9drDjdu/JJQ/Iw0Gg.aTkFzCjNAbaK4zcCHbphFz1g1HK18Z2bI92M0'
splunk::enterprise::password_content: ":admin:${password_hash}::Administrator:admin:changeme@example.com::"
splunk::enterprise::manage_package_source: true
splunk::enterprise::src_root: 'https://download.splunk.com'
splunk::enterprise::service_ensure: 'running'
splunk::enterprise::splunkd_port: 8089
splunk::enterprise::logging_port: 9997
splunk::enterprise::web_httpport: 8000
splunk::enterprise::server: 'splunk'
splunk::enterprise::boot_start: true
splunk::enterprise::use_default_config: true
splunk::enterprise::input_default_host: "${facts.fqdn}"
splunk::enterprise::input_connection_host: 'dns'
splunk::enterprise::splunkd_listen: '127.0.0.1'
splunk::enterprise::purge_alert_actions: false
splunk::enterprise::purge_authentication: false
splunk::enterprise::purge_authorize: false
splunk::enterprise::purge_deploymentclient: false
splunk::enterprise::purge_distsearch: false
splunk::enterprise::purge_indexes: false
splunk::enterprise::purge_inputs: false
splunk::enterprise::purge_limits: false
splunk::enterprise::purge_outputs: false
splunk::enterprise::purge_props: false
splunk::enterprise::purge_server: false
splunk::enterprise::purge_serverclass: false
splunk::enterprise::purge_transforms: false
splunk::enterprise::purge_uiprefs: false
splunk::enterprise::purge_web: false

splunk::forwarder::server: 'splunk'
splunk::forwarder::service_ensure: 'running'
splunk::forwarder::manage_package_source: true
splunk::forwarder::src_root: 'https://download.splunk.com'
splunk::forwarder::boot_start: true
splunk::forwarder::use_default_config: true
splunk::forwarder::splunkd_listen: '127.0.0.1'
splunk::forwarder::splunkd_port: 8089
splunk::forwarder::logging_port: 9997
splunk::forwarder::purge_deploymentclient: false
splunk::forwarder::purge_outputs: false
splunk::forwarder::purge_inputs: false
splunk::forwarder::purge_props: false
splunk::forwarder::purge_transforms: false
splunk::forwarder::purge_web: false
splunk::forwarder::manage_password: false
splunk::forwarder::seed_password: false
splunk::forwarder::reset_seeded_password: false
splunk::forwarder::secret: 'hhy9DOGqli4.aZWCuGvz8stcqT2/OSJUZuyWHKc4wnJtQ6IZu2bfjeElgYmGHN9RWIT3zs5hRJcX1wGerpMNObWhFue78jZMALs3c3Mzc6CzM98/yGYdfcvWMo1HRdKn82LVeBJI5dNznlZWfzg6xdywWbeUVQZcOZtODi10hdxSJ4I3wmCv0nmkSWMVOEKHxti6QLgjfuj/MOoh8.2pM0/CqF5u6ORAzqFZ8Qf3c27uVEahy7ShxSv2K4K41z'
splunk::forwarder::password_hash: '$6$pIE/xAyP9mvBaewv$4GYFxC0SqonT6/x8qGcZXVCRLUVKODj9drDjdu/JJQ/Iw0Gg.aTkFzCjNAbaK4zcCHbphFz1g1HK18Z2bI92M0'
splunk::forwarder::password_content: ":admin:${password_hash}::Administrator:admin:changeme@example.com::"
splunk::forwarder::addons: {}
25 changes: 25 additions & 0 deletions data/kernel/Linux.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---

splunk::enterprise::package_name: 'splunk'
splunk::enterprise::splunk_user: 'root'
splunk::enterprise::staging_dir: '/opt/staging/splunk'
splunk::enterprise::homedir: '/opt/splunk'
splunk::enterprise::path_delimiter: '/'
splunk::enterprise::seed_config_file: "%{lookup('splunk::enterprise::homedir')}/etc/system/local/user-seed.conf"
splunk::enterprise::password_config_file: "%{lookup('splunk::enterprise::homedir')}/etc/passwd"
splunk::enterprise::secret_file: "%{lookup('splunk::enterprise::homedir')}/etc/splunk.secret"
splunk::enterprise::confdir: "%{lookup('splunk::enterprise::homedir')}/etc"
splunk::enterprise::src_subdir: 'linux'
splunk::enterprise::install_options: []

splunk::forwarder::package_name: 'splunkforwarder'
splunk::forwarder::staging_dir: '/opt/staging/splunk'
splunk::forwarder::path_delimiter: '/'
splunk::forwarder::homedir: '/opt/splunkforwarder'
splunk::forwarder::confdir: "%{lookup('splunk::forwarder::homedir')}/etc"
splunk::forwarder::src_subdir: 'linux'
splunk::forwarder::install_options: []
splunk::forwarder::splunk_user: 'root'
splunk::forwarder::password_config_file: "%{lookup('splunk::forwarder::homedir')}/etc/passwd"
splunk::forwarder::seed_config_file: "%{lookup('splunk::forwarder::homedir')}/etc/system/local/user-seed.conf"
splunk::forwarder::secret_file: "%{lookup('splunk::forwarder::homedir')}/etc/splunk.secret"
25 changes: 25 additions & 0 deletions data/kernel/SunOS.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---

splunk::enterprise::package_name: 'splunk'
splunk::enterprise::splunk_user: 'root'
splunk::enterprise::staging_dir: '/opt/staging/splunk'
splunk::enterprise::homedir: '/opt/splunk'
splunk::enterprise::path_delimiter: '/'
splunk::enterprise::seed_config_file: "%{lookup('splunk::enterprise::homedir')}/etc/system/local/user-seed.conf"
splunk::enterprise::password_config_file: "%{lookup('splunk::enterprise::homedir')}/etc/passwd"
splunk::enterprise::secret_file: "%{lookup('splunk::enterprise::homedir')}/etc/splunk.secret"
splunk::enterprise::confdir: "%{lookup('splunk::enterprise::homedir')}/etc"
splunk::enterprise::src_subdir: 'solaris'
splunk::enterprise::install_options: []

splunk::forwarder::package_name: 'splunkforwarder'
splunk::forwarder::staging_dir: '/opt/staging/splunk'
splunk::forwarder::path_delimiter: '/'
splunk::forwarder::homedir: '/opt/splunkforwarder'
splunk::forwarder::confdir: "%{lookup('splunk::forwarder::homedir')}/etc"
splunk::forwarder::src_subdir: 'solaris'
splunk::forwarder::install_options: []
splunk::forwarder::splunk_user: 'root'
splunk::forwarder::password_config_file: "%{lookup('splunk::forwarder::homedir')}/etc/passwd"
splunk::forwarder::seed_config_file: "%{lookup('splunk::forwarder::homedir')}/etc/system/local/user-seed.conf"
splunk::forwarder::secret_file: "%{lookup('splunk::forwarder::homedir')}/etc/splunk.secret"
37 changes: 37 additions & 0 deletions data/kernel/Windows.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---

splunk::enterprise::package_name: 'Splunk Enterpirse'
splunk::enterprise::splunk_user: 'Administrator'
splunk::enterprise::staging_dir: "%{facts.archive_windir}\\splunk"
splunk::enterprise::homedir: 'C:\\Program Files\\Splunk'
splunk::enterprise::path_delimiter: '\\'
splunk::enterprise::seed_config_file: "%{lookup('splunk::enterprise::homedir')}\\etc\\system\\local\\user-seed.conf"
splunk::enterprise::password_config_file: "%{lookup('splunk::enterprise::homedir')}\\etc\\passwd"
splunk::enterprise::confdir: "%{lookup('splunk::enterprise::homedir')}\\etc"
splunk::enterprise::src_subdir: 'windows'
splunk::enterprise::install_options: [
{INSTALLDIR: "%{lookup('splunk::enterprise::homedir')}"},
{SPLUNKD_PORT: "%{lookup('splunk::enterprise::splunkd_port')}"},
'ARGEETOLICENSE=Yes',
'LAUNCHSPLUNK=0']

splunk::forwarder::splunk_user: 'Administrator'
splunk::forwarder::package_name: 'UniversalForwarder'
splunk::forwarder::staging_dir: "%{facts.archive_windir}\\splunk"
splunk::forwarder::homedir: 'C:\\Program Files\\SplunkUniversalForwarder'
splunk::forwarder::confdir: "%{lookup('splunk::forwarder::homedir')}\\etc"
splunk::forwarder::src_subdir: 'windows'
splunk::forwarder::path_delimiter: '\\'
splunk::forwarder::password_config_file: "%{lookup('splunk::forwarder::homedir')}\\etc\\passwd"
splunk::forwarder::seed_config_file: "%{lookup('splunk::forwarder::homedir')}\\etc\\system\\local\\user-seed.conf"
splunk::forwarder::install_options: [
{INSTALLDIR: "%{lookup('splunk::forwarder::homedir')}"},
'AGREETOLICENSE=Yes',
'LAUNCHSPLUNK=0',
'SERVICESTARTTYPE=auto',
'WINEVENTLOG_APP_ENABLE=1',
'WINEVENTLOG_SEC_ENABLE=1',
'WINEVENTLOG_SYS_ENABLE=1',
'WINEVENTLOG_FWD_ENABLE=1',
'WINEVENTLOG_SET_ENABLE=1',
'ENABLEADMON=1']
5 changes: 5 additions & 0 deletions data/os/Debian.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---

splunk::enterprise::package_provider: 'dpkg'

splunk::forwarder::package_provider: 'dpkg'
5 changes: 5 additions & 0 deletions data/os/RedHat.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---

splunk::enterprise::package_provider: 'rpm'

splunk::forwarder::package_provider: 'rpm'
5 changes: 5 additions & 0 deletions data/os/Solaris.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---

splunk::enterprise::package_provider: 'sun'

splunk::forwarder::package_provider: 'sun'
5 changes: 5 additions & 0 deletions data/os/windows.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---

splunk::enterprise::package_provider: 'windows'

splunk::forwarder::package_provider: 'windows'
16 changes: 16 additions & 0 deletions hiera.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
version: 5

defaults:
datadir: data
data_hash: yaml_data

hierarchy:
- name: "OS family"
path: "os/%{facts.os.family}.yaml"

- name: "Kernel"
path: "kernel/%{facts.kernel}.yaml"

- name: "common"
path: "common.yaml"
Loading

0 comments on commit 62c9b97

Please sign in to comment.