vm-facts

This role parses the existing hostvars of VMs and generates vars used by xen_vman, zfs-storage, zfs-snap-manager, zfs-auto-snapshot and iscsi-target. With correct vars set, this role is able to gather and set all facts needed to create ZFS filesystems/ZVOLs, share them via NFS or iSCSI, snapshot them (locally and for remote replication) and configure XEN instances for them.

Requirements

A Linux distribution.

Role Variables

Name	Default / Mandatory	Description
vm_facts_generate_storage_facts	False	Flag to activate fact generation for storage variables (ZFS, NFS, iSCSI).
vm_facts_generate_backup_facts	False	Flag to activate fact generation for storage variables (ZFS).
vm_facts_generate_hypervisor_facts	False	Flag to activate fact generation for hypervisor variables (Xen).
vm_facts_default_hypervisor_host	✔️	Inventory name of the default hypervisor, used when a VM doesn't specify its own hypervisor_host.
vm_facts_default_backup_host	✔️	Inventory name of the default backup server, used when a VM doesn't specify its own backup_host.
vm_facts_default_backup_replication_host	✔️	Inventory name of the default backup replication server, used when a VM doesn't specify its own backup_replication_host.
vm_facts_default_storage_host	✔️	Inventory name of the default storage server, used when a VM doesn't specify its own storage_host.
vm_facts_limit_hosts	✖️	If defined, only the hosts in this list will be considered for fact generation.

vm_facts_generate_storage_facts and vm_facts_generate_backup_facts are mutually exclusive with vm_facts_generate_storage_facts being prioritized.

Role Variables (needed for storage or backup)

Name	Default / Mandatory	Description
vm_facts_storage_zfs_parent_prefix	tank/	[only needed on storage and backup] A prefix string for ZFS filesystems and ZVOLs, e.g. tank/vms/.
vm_facts_backup_zfs_parent_prefix	tank/	[only needed on storage] A prefix string for ZFS filesystems and ZVOLs, e.g. tank/vms/. This is used for setting the target filesystems on snapshot configurations (i.e. where on the backup host shall the filesystem on this storage host be placed). The full backup target filesystem is generated by appending the used storage_host of the vm and /vms/, followed by the organization and vm hostname.
vm_facts_backup_replication_zfs_parent_prefix	tank/	[only needed on backup] This is the ZFS dataset prefix on the backup_replication_host where backups are replicated to.
vm_facts_nfs_options	[no_root_squash]	[only needed on storage] A list of NFS options, e.g. a default IP that is added to all exports. The option format must conform to the ZFS sharenfs attribute format. The role adds by default the hypervisor_host or the default hypervisor_host IP as rw=@{{ hypervisor_host.ansible_host }} option.
vm_facts_default_root_reservation		[only needed on storage and backup] If this var is set (e.g. to 10G or any other allowed value for ZFS reservation attribute), root filesystems will use it as reservation by default. Otherwise, the size of the VM will be set as reservation when choosing storage. Independently, custom reservation may be set in the vm.filesystems variable
vm_facts_default_storage_type	filesystem	[only needed on storage and backup] This storage type will be chosen if the vm block does not specify one itself.
vm_facts_check_manual_snapshots	{}	[only needed on storage and backup] Use this variable to specify additional datasets to be checked using Icinga. Keys are frequencies, values are lists of datasets. See section snapshot files below.
iscsi_default_initiators	✔️ (if iSCSI is used)	[only needed on storage] This var is explained in iscsi-target.
iscsi_default_portals	✔️ (if iSCSI is used)	[only needed on storage] This var is explained in iscsi-target.

Role Variables (hypervisor)

Name	Default / Mandatory	Description
vm_facts_default_cidr_suffix	/24	This suffix will be appended to all IPs in vm.interfaces if the IP in question does not already define a CIDR subnet mask.

Role Variables (VM hostvars)

As this role looks at all hostvars, some variables, especially the vm block are relevant. This table only lists the options used in this role, see xen-vman for other possible and mandatory vars inside the vm dict.

vm

Name	Default / Mandatory	Description
org	✔️	The organization this VM belongs to. Depending on this value, the filesystem or ZVOL will be placed in a different hierarchy.
storage_type	filesystem	Use blockdevice to create a ZVOL (ZFS virtual blockdevice of static size) and export it (as {{ org }}-{{ name }}) via iSCSI (only on storage). Use filesystem (the default) to create a root filesystem {{ org }}/{{ name }}-root and optionally other filesystems (see filesystems) and export them via NFS (only enabled on type storage).
create_storage	True	Whether to automatically create storage filesystems/blockdevices for this VM.
create_backup	True	Whether to automatically create backup filesystems/blockdevices for this VM.
size	✔️	Size of the root blockdevice or filesystem, e.g. 15G. The size can only be changed later on if storage_type=filesystem.
filesystems	[{'name': root, 'zfs_attributes':{'quota': {{ size }}, 'reservation': {{ size &#124 default(vm_facts_default_root_reservation) }}}, 'nfs_options': {{ vm_facts_nfs_options }} + [rw=@{{ interface.ip }},rw=@{{ hypervisor_host.ansible_host }}]}]	A list containing filesystem definitions, see filesystems - this var is only relevant if storage_type=filesystem.
local_snapshots	[hourly]	List of snapshot frequencies to be activated for zfs-auto-snapshot. For blockdevices the attribute needs to be set here, for filesystems, the value may be overridden in vm.filesystems. This role supports frequent, hourly, daily, weekly and monthly intervals.
description	✔️	Description of this VM's purpose. This var may also be on the root level of the VM host in question.
storage_host	{{ vm_facts_default_storage_host }}	Storage host to use for this VM.
hypervisor_host	{{ vm_facts_default_hypervisor_host }}	Hypervisor host to use for this VM.
backup_host	{{ vm_facts_default_backup_host }}	Host to which this VM shall be backed up to.
backup_replication_host	{{ vm_facts_default_backup_replication_host }}	Host to which the backup_host of this VM shall replicate its backups to.
pull_storage_from	✖️	The storage hostname to pull the VM datasets from. This will add a list item to the generated vm_facts_move_storages var, consisting of a dict containing the keys source_storage, source_dataset, target_storage, target_dataset_suffix, source_backup_dataset, target_backup_dataset and backup_host needed to move a VM.
pull_hypervisor_from	✖️	The hypervisor hostname to pull the VM configuration from. This will add a list item to the generated vm_facts_move_hypervisors var, consisting of a dict containing the keys source_hypervisor, destination_hypervisor, org and host needed to move a VM.

snapshot files

On storage hosts, the role will create files in /var/local that contain the list of datasets that should be snapshotted in a specific interval e.g. /var/local/icinga2_check_zfs_snapshots_daily. This is done for the local_snapshots intervals specified for the datasets.

Regardless of the actual selection of this interval for zfs-auto-snapshot, every dataset on storage or backup hosts will be appended to the daily file. This feature is used to be able to check the correct creation and transfer of snapshots by zfs-snap-manager.

filesystems

vm.filesystems is a list of dicts that describe filesystems for one VM. A root filesystem will always be created, with quota set to the VM size value, reservation set to the global default value or also the VM size (see above) and the the default NFS options as well as no_root_squash added. On hosts with vm_facts_variant=backup, no NFS options will be added and the filesystems or volumes will be set to readonly and the quota set to none. Quotas in ZFS sum up not only the filesystem, but also its decendants such as snapshots. Therefore the quota intended for the storage of a VM will limit the amount of snapshots of it that can be stored on the backup host. The filesystems var is only respected if storage_type is filesystem.

Name	Default / Mandatory	Description
name	✔️	name-suffix of the filesystem. The final name consists of the vm name and the filesystem name, delimited by a dash.
zfs_attributes	{}	A dict containing any ZFS attributes that shall be set for this filesystem. It is recommended to define a quota, though this may also be done in the default ZFS attributes on the actual zfs-storage server.
nfs_options	[no_root_squash,rw=@{{ ansible_host }},rw=@{{ hypervisor_host.ansible_host }}]	A list of NFS options that are set for this NFS export. The options have to conform to the ZFS sharenfs attribute format. The options defined in vm_nfs_options will be set in addition to this value in every case. Also, read-write access will be granted to the VM and its hypervisor in every case (but not on backup servers).
local_snapshots	{{ vm.local_snapshots }}	List of snapshot frequencies to be activated for zfs-auto-snapshot
storage_custom_fs_prefix	The default fs prefix of the storage host	A custom filesystem prefix used on the storage host for this filesystem

Example Playbook

storage01 hostvars

vm_facts_generate_storage_facts: True
vm_facts_storage_zfs_parent_prefix: tank/vms/
iscsi_default_initiators:
 - name: 'iqn.1994-05.com.redhat:client1'
   authentication:
     userid: myuser
     password: mypassword
     userid_mutual: sharedkey
     password_mutual: sharedsecret
iscsi_default_portals:
 - ip: 192.168.10.3
vm_facts_nfs_options:
 - sync
vm_facts_default_root_reservation: 10G
interfaces:
 - ip: '192.168.10.3'

hypervisor01 hostvars

vm_facts_generate_hypervisor_facts: True
interfaces:
 - ip: '192.168.10.2'

web01 hostvars

vm:
  description: VM to host static websites
  memory: 2048
  vcpus: 4
  org: misc
  size: 15G
  storage_type: filesystem
  storage_host: storage01
  hypervisor_host: hypervisor01
  filesystems:
   - name: data
     zfs_attributes:
       quota: 50G
  interfaces:
   - mac: 'AA:BB:CC:FE:19:AA'
     ip:  '192.168.10.52'
   - mac: 'AA:BB:CC:FE:19:AB'
     ip:  '192.168.100.52'
ansible_host: 192.168.10.52

Result

Assuming the vm is named web01, these two filesystems will be created on the storage server:

Name	ZFS attributes
tank/vms/misc/web01-root	quota=15G, reservation=10G, sharenfs=no_root_squash,rw=@192.168.10.2,rw=@192.168.10.52
tank/vms/misc/web01-data	quota=50G, sharenfs=no_root_squash,rw=@192.168.10.2,rw=@192.168.10.52

The hypervisor will have an additional entry in his xen_vman_vms list:

xen_vman_vms:
 - description: VM to host static websites
   memory: 2048
   vcpus: 4
   org: misc
   size: 15G
   storage_host: storage01
   hypervisor_host: hypervisor01
   storage_type: filesystem
   interfaces:
    - mac: 'AA:BB:CC:FE:19:AA'
      ip:  '192.168.10.52'
    - mac: 'AA:BB:CC:FE:19:AB'
      ip:  '192.168.100.52'

Note that filesystems has been removed as it is not needed by the hypervisor.

License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Author Information

• Michel Weitbrecht (SlothOfAnarchy) michel.weitbrecht@stuvus.uni-stuttgart.de