About

This module helps install OpenShift Origin Platform As A Service. Through the declaration of the openshift_origin class, you can configure the OpenShift Origin Broker, Node and support services including ActiveMQ, MongoDB, named and OS settings including firewall, services, and ntp.

Authors

• Jamey Owens

• Ben Klang

• Ben Langfeld

• Krishna Raman

• N. Harrison Ripps

Requirements

• Puppet >= 3

• Facter >= 1.6.17

Installation

The module can be installed directly via the puppet utility from PuppetForge:

puppet module install openshift/openshift_origin

Additionally, the module can be obtained from the github repository.

1. Download the Zip file from github

2. Upload the Zip file to your Puppet Master.

3. Unzip the file. This will create a new directory called puppet-openshift_origin-{commit hash}

4. Rename this directory to just openshift_origin and place it in your modulepath.

Configuration

There is one class (openshift_origin) that needs to be declared on all nodes managing any component of OpenShift Origin. These nodes are configured using the parameters of this class.

Using Parameterized Classes

Using Parameterized Classes

Example: Single host (broker+console+node) which uses the Avahi MDNS and htpasswd Auth plugin:

class { 'openshift_origin' :
  domain                        => 'example.com',
  node_unmanaged_users          => ['root'],
  development_mode              => true,
  conf_node_external_eth_dev    => 'eth0',
  install_method                => 'yum',
  register_host_with_nameserver => true,
  broker_auth_plugin            => 'htpasswd',
  broker_dns_plugin             => 'avahi',
}

Example: Single host (broker+console+node) which uses the Kerberos Auth plugin and GSS-TSIG.

class { 'openshift_origin' :
  domain                        => 'example.com',
  node_unmanaged_users          => ['root'],
  development_mode              => true,
  conf_node_external_eth_dev    => 'eth0',
  install_method                => 'yum',
  register_host_with_nameserver => true,
  broker_auth_plugin            => 'remote-user',
  broker_dns_plugin             => 'nsupdate',
  bind_krb_principal            => $hostname,
  bind_krb_keytab               => '/etc/dns.keytab'
  broker_krb_keytab             => '/etc/http.keytab',
  broker_krb_auth_realms        => 'EXAMPLE.COM',
  broker_krb_service_name       => $hostname,
}

Please note:

• The Broker needs to be enrolled in the KDC as a host, host/node_fqdn as well as a service, HTTP/node_fqdn

• Keytab should be generated, is located on the Broker machine, and Apache should be able to access it (chown apache <kerberos_keytab>)

• Like the example config below:

  • set broker_auth_plugin to 'kerberos'

  • set broker_krb_keytab and bind_krb_keytab to the absolute file location of the keytab

  • set broker_krb_auth_realms to the kerberos realm that the Broker host is enrolled with

  • set broker_krb_service_name to the FQDN of the enrolled kerberos service, e.g. $hostname

• After setup, to test:

  • authentication: kinit <user> then curl -Ik --negotiate -u : <node_fqdn>

  • GSS-TSIG (should return nil):

    $ cd /var/www/openshift/broker
    $ bundle --local
    $ rails console
    $ d = OpenShift::DnsService.instance
    $ d.register_application "appname", "namespace", "node_fqdn"
    => nil

  • For any errors, on the Broker, check /var/log/openshift/broker/httpd/error_log.

Parameters

roles

Choose from the following roles to be configured on this node.

• broker - Installs the broker and console.

• node - Installs the node and cartridges.

• msgserver - Installs ActiveMQ message broker.

• datastore - Installs MongoDB (not sharded/replicated)

• nameserver - Installs a BIND dns server configured with a TSIG key for updates.

• load_balancer - Installs HAProxy and Keepalived for Broker API high-availability.

Default: ['broker','node','msgserver','datastore','nameserver']

install_method

Choose from the following ways to provide packages:

1. none - install sources are already set up when the script executes (default)

2. yum - set up yum repos manually

   • repos_base

   • os_repo

   • os_updates_repo

   • jboss_repo_base

   • jenkins_repo_base

   • optional_repo

Default: yum

repos_base

Base path to repository for OpenShift Origin

• Nightlies:

  • Fedora: https://mirror.openshift.com/pub/origin-server/nightly/fedora-19

  • RHEL: https://mirror.openshift.com/pub/origin-server/nightly/rhel-6

• Release-2:

  • Fedora: https://mirror.openshift.com/pub/origin-server/release/2/fedora-19

  • RHEL: https://mirror.openshift.com/pub/origin-server/release/2/rhel-6

Default: Fedora-19 Nightlies

architecture

CPU Architecture to use for the definition OpenShift Origin yum repositories Defaults: $::architecture (from facter)

• RHEL:

  • x86_64

• CentOS:

  • x86_64

override_install_repo

Repository path override. Uses dependencies from repos_base but uses override_install_repo path for OpenShift RPMs. Used when doing local builds.

Default: none

os_repo

The URL for a Fedora 19/RHEL 6 yum repository used with the "yum" install method. Should end in x86_64/os/.

Default: no change

os_updates

The URL for a Fedora 19/RHEL 6 yum updates repository used with the "yum" install method. Should end in x86_64/.

Default: no change

jboss_repo_base

The URL for a JBoss repositories used with the "yum" install method. Does not install repository if not specified.

jenkins_repo_base

The URL for a Jenkins repositories used with the "yum" install method. Does not install repository if not specified.

optional_repo

The URL for a EPEL or optional repositories used with the "yum" install method. Does not install repository if not specified.

domain

The network domain under which apps and hosts will be placed.

Default: example.com

broker_hostname

node_hostname

nameserver_hostname

msgserver_hostname

datastore_hostname

Default: the root plus the domain, e.g. broker.example.com - except nameserver=ns1.example.com

These supply the FQDN of the hosts containing these components. Used for configuring the host’s name at install, and also for configuring the broker application to reach the services needed.

Note	if installing a nameserver, the script will create DNS entries for the hostnames of the other components being installed on this host as well. If you are using a nameserver set up separately, you are responsible for all necessary DNS entries.

datastore1_ip_addr|datastore2_ip_addr|datastore3_ip_addr

Default: undef

IP addresses of the first 3 MongoDB servers in a replica set. Add datastoreX_ip_addr parameters for larger clusters.

nameserver_ip_addr

Default: IP of a name server instance or current IP if installing on this node. This is used by every node to configure its primary name server.

Default: the current IP (at install)

bind_key

When the nameserver is remote, use this to specify the HMAC-MD5 key for updates. This is the "Key:" field from the .private key file generated by dnssec-keygen. This field is required on all nodes.

bind_krb_keytab

When the nameserver is remote, Kerberos keytab together with principal can be used instead of the HMAC-MD5 key for updates.

bind_krb_principal

When the nameserver is remote, this Kerberos principal together with Kerberos keytab can be used instead of the HMAC-MD5 key for updates.

conf_nameserver_upstream_dns

List of upstream DNS servers to use when installing a nameserver on this node.

Default: ['8.8.8.8']

broker_ip_addr

This is used for the node to record its broker. Also is the default for the nameserver IP if none is given.

Default: the current IP (at install)

broker_cluster_members

An array of broker hostnames that will be load-balanced for high-availability.

Default: undef

broker_cluster_ip_addresses

An array of Broker IP addresses within the load-balanced cluster.

Default: undef

broker_virtual_ip_address

The virtual IP address that will front-end the Broker cluster.

Default: undef

broker_virtual_hostname

The hostame that represents the Broker API cluster. This name is associated to broker_virtual_ip_address and added to Named for DNS resolution.

Default: "broker.${domain}"

load_balancer_master

Sets the state of the load-balancer. Valid options are true or false. true sets the load-balancer as the active listener for the Broker cluster Virtual IP address. Only 1 load_balancer_master is allowed within a Broker cluster.

Default: false

load_balancer_auth_password

The password used to secure communication between the load-balancers within a Broker cluster.

Default: 'changeme'

node_ip_addr

This is used for the node to give a public IP, if different from the one on its NIC.

Default: the current IP (at install)

configure_ntp

Enabling this configures NTP. It is important that the time be synchronized across hosts because MCollective messages have a TTL of 60 seconds and may be dropped if the clocks are too far out of synch. However, NTP is not necessary if the clock will be kept in synch by some other means.

Default: true

Note	Passwords used to secure various services. You are advised to specify only alphanumeric values in this script as others may cause syntax errors depending on context. If non-alphanumeric values are required, update them separately after installation.

ntp_servers

If configure_ntp is set to true (default), ntp_servers allows users to specify an array of NTP servers used for clock synchronization.

Default: ['time.apple.com iburst', 'pool.ntp.org iburst', 'clock.redhat.com iburst']

Note	Use iburst after every ntp server definition to speed up the initial synchronization.

msgserver_cluster

Default: false

Set to true to cluster ActiveMQ for high-availability and scalability of OpenShift message queues.

msgserver_cluster_members

Default: undef

An array of ActiveMQ server hostnames to be included in the ActiveMQ cluster. Required when parameter msgserver_cluster is set to true.

mcollective_cluster_members

Default: $msgserver_cluster_members

An array of ActiveMQ server hostnames to be included in the ActiveMQ cluster. Required when parameter msgserver_cluster is set to true.

msgserver_password

Default 'changeme'

Password used by ActiveMQ’s amquser. The amquser is used to authenticate ActiveMQ inter-cluster communication. Only used when msgserver_cluster is true.

msgserver_admin_password

This is the admin password for the ActiveMQ admin console, which is not needed by OpenShift but might be useful in troubleshooting.

Default: scrambled

mcollective_user

mcollective_password

This is the user and password shared between broker and node for communicating over the mcollective topic channels in ActiveMQ. Must be the same on all broker and node hosts.

Default: mcollective/marionette

mongodb_admin_user

mongodb_admin_password

These are the username and password of the administrative user that will be created in the MongoDB datastore. These credentials are not used by in this script or by OpenShift, but an administrative user must be added to MongoDB in order for it to enforce authentication. Note: The administrative user will not be created if CONF_NO_DATASTORE_AUTH_FOR_LOCALHOST is enabled.

Default: admin/mongopass

mongodb_broker_user

mongodb_broker_password

These are the username and password of the normal user that will be created for the broker to connect to the MongoDB datastore. The broker application’s MongoDB plugin is also configured with these values.

Default: openshift/mongopass

mongodb_name

This is the name of the database in MongoDB in which the broker will store data.

Default: openshift_broker

mongodb_port

Default: '27017'

The TCP port used for MongoDB to listen on.

mongodb_replicasets

Default: false

Enable/disable MongoDB replica sets for database high-availability.

mongodb_replica_name

Default: 'openshift'

The MongoDB replica set name when $mongodb_replicasets is true.

mongodb_replica_primary

Default: undef

Set the host as the primary with true or secondary with false.

mongodb_replica_primary_ip_addr

Default: undef

The IP address of the Primary host within the MongoDB replica set.

mongodb_replicasets_members

Default: undef

An array of [host:port] of replica set hosts. Example: ['10.10.10.10:27017', '10.10.10.11:27017', '10.10.10.12:27017']

mongodb_keyfile

Default: '/etc/mongodb.keyfile'

The file containing the $mongodb_key used to authenticate MongoDB replica set members.

mongodb_key

Default: 'changeme'

The key used by members of a MongoDB replica set to authenticate one another.

openshift_user1

openshift_password1

This user and password are entered in the /etc/openshift/htpasswd file as a demo/test user. You will likely want to remove it after installation (or just use a different auth method).

Default: demo/changeme

conf_broker_auth_salt

conf_broker_auth_public_key

conf_broker_auth_private_key

conf_broker_auth_key_password

Salt, public and private keys used when generating secure authentication tokens for Application to Broker communication. Requests like scale up/down and jenkins builds use these authentication tokens. This value must be the same on all broker nodes.

Default: Self signed keys are generated. Will not work with multi-broker setup.

conf_broker_session_secret

conf_console_session_secret

Session secrets used to encode cookies used by console and broker. This value must be the same on all broker nodes.

conf_valid_gear_sizes

List of all gear sizes this will be used in this OpenShift installation.

Default: ['small']

conf_default_gear_size

Default gear size if one is not specified.

Default: ['small']

conf_default_gear_capabilities

List of all gear sizes that newly created users will be able to create.

Default: ['small']

broker_dns_plugin

DNS plugin used by the broker to register application DNS entries. Options:

• nsupdate - nsupdate based plugin. Supports TSIG and GSS-TSIG based authentication. Uses bind_key for TSIG and bind_krb_keytab, bind_krb_principal for GSS_TSIG auth.

• avahi - sets up a MDNS based DNS resolution. Works only for all-in-one installations.

broker_auth_plugin

Authentication setup for users of the OpenShift service. Options:

• mongo - Stores username and password in mongo.

• kerberos - Kerberos based authentication. Uses broker_krb_service_name, broker_krb_auth_realms, broker_krb_keytab values.

• htpasswd - Stores username/password in a htaccess file.

• ldap - LDAP based authentication. Uses broker_ldap_uri.

Default: htpasswd

broker_krb_service_name

The KrbServiceName value for mod_auth_kerb configuration

broker_krb_auth_realms

The KrbAuthRealms value for mod_auth_kerb configuration

broker_krb_keytab

The Krb5KeyTab value of mod_auth_kerb is not configurable — the keytab is expected in /var/www/openshift/broker/httpd/conf.d/http.keytab

broker_ldap_uri

URI to the LDAP server (e.g. ldap://ldap.example.com:389/ou=People,dc=my-domain,dc=com). Set <code>broker_auth_plugin</code> to <code>ldap</code> to enable this feature.

node_container_plugin

Specify the container type to use on the node. Options:

• selinux - This is the default OpenShift Origin container type.

node_frontend_plugins

Specify one or more plugins to use register HTTP and web-socket connections for applications. Options:

• apache-mod-rewrite - Mod-Rewrite based plugin for HTTP and HTTPS requests. Well suited for installations with a lot of creates/deletes/scale actions.

• apache-vhost - VHost based plugin for HTTP and HTTPS. Suited for installations with less app create/delete activity. Easier to customize. If apache-mod-rewrite is also selected, apache-vhost will be ignored

• nodejs-websocket - Web-socket proxy listening on ports 8000/8444

• haproxy-sni-proxy - TLS proxy using SNI routing on ports 2303 through 2308 requires /usr/sbin/haproxy15 (haproxy-1.5-dev19 or later).

Default: ['apache-mod-rewrite','nodejs-websocket']

node_unmanaged_users

List of user names who have UIDs in the range of OpenShift gears but must be excluded from OpenShift gear setups.

Default: []

conf_node_external_eth_dev

External facing network device. Used for routing and traffic control setup.

Default: eth0

conf_node_supplementary_posix_groups

Name of supplementary UNIX group to add a gear to.

development_mode

Set development mode and extra logging.

Default: false

install_login_shell

Install a Getty shell which displays DNS, IP and login information. Used for all-in-one VM installation.

register_host_with_nameserver

Setup DNS entries for this host in a locally installed bind DNS instance.

Default: false

install_cartridges

List of cartridges to be installed on the node. Options:

• 10gen-mms-agent

• cron

• diy

• haproxy

• mongodb

• nodejs

• perl

• php

• phpmyadmin

• postgresql

• python

• ruby

• jenkins

• jenkins-client

• mariadb (will install mysql on RHEL)

• jbossews

• jbossas

• jbosseap

Default: ['10gen-mms-agent','cron','diy','haproxy','mongodb', 'nodejs','perl','php','phpmyadmin','postgresql', 'python','ruby','jenkins','jenkins-client','mariadb']

Manual Tasks

This script attempts to automate as many tasks as it reasonably can. Unfortunately, it is constrained to setting up only a single host at a time. In an assumed multi-host setup, you will need to do the following after the script has completed.

1. Set up DNS entries for hosts.

   If you installed BIND with the script, then any other components installed with the script on the same host received DNS entries. Other hosts must all be defined manually, including at least your node hosts. oo-register-dns may prove useful for this.

2. Copy public rsync key to enable moving gears.

   The broker rsync public key needs to go on nodes, but there is no good way to script that generically. Nodes should not have password-less access to brokers to copy the .pub key, so this must be performed manually on each node host:

   # scp root@broker:/etc/openshift/rsync_id_rsa.pub /root/.ssh/
   (above step will ask for the root password of the broker machine)
   # cat /root/.ssh/rsync_id_rsa.pub >> /root/.ssh/authorized_keys
   # rm /root/.ssh/rsync_id_rsa.pub

   If you skip this, each gear move will require typing root passwords for each of the node hosts involved.

3. Copy ssh host keys between the node hosts.

   All node hosts should identify as the same host, so that when gears are moved between hosts, ssh and git don’t give developers spurious warnings about the host keys changing. So, copy /etc/ssh/ssh_* from one node host to all the rest (or, if using the same image for all hosts, just keep the keys from the image).