Ansible

This infrastructure section is both a information source of current ansible usage options and a guide on how to use ansible. Ansible is a simple it automation software. It uses ssh to connect to machines and deploy/configure/install software and more. More information can be found on their website.
It is basically an easy way to execute a command on many machines at the same time or many commands on one machine. For detailed information about documentation visit ansible-doc

There is one Controller Machine: the machine where Ansible is installed, responsible for running the provisioning on the servers you are managing. Most of the time it is installed on the administrators working station. From there ssh keys need to be set up for all machines that should be managed by ansible.
To download the newest version of ansible add the the repository as described:

sudo apt-add-repository ppa:ansible/ansible 
sudo apt-get update 

Set up already configured ssh hosts in ..ansible-scripts/hosts. Already configured = Passwordless ssh login on the hosts with ssh public keys, configured in the ~/.ssh/config file.

Introduction

You can group hosts with [some-name]. Later you can specify commands only for some groups.
Now you can use different commands:

CommandEffect
ansible all -m ping Ping all nodes
ansible all -m ping -u paulth Ping all nodes as user paulth
ansible new -m ping -u paulth -b Ping all nodes in group new as user paulth with privilige escalation as sudo
ansible all -m ping -u paulth -b –become-user batman Ping all nodes as paulth sudoing to batman
ansible all -m setup Show all information of remote system

To run a live command use the a flag:

ansible all -a "/usr/bin/whoami" 
#run with 10 parallel forks
ansible new -a "/sbin/reboot" -f 10

The next step is to differentiate between ad-hoc commands and playbooks. Ad-hoc commands are for quick tasks and commands that you don't want to save for later. Playbooks are different steps and commands packed into one yml file. They are useful for basic setup.
In the table above there were already examples for ad-hoc commands. For more commands have a look at this site.
For Playbooks have a look at the next section:

Executing a playbook is done from the root of the ansible-scripts directory e.g.:

ansible-playbook plays/vault_test.yml

This is a list with available playbooks that can be executed. the playbook most likely contains roles that are documented below:

vault-test

Usage/Result:

Checks if the ansible vault key is correct and displays all variables defined for group named 'test'

Variables:
NameNote
test_variablevariable stored in usual text-file and showed in the output
test_secret_variablevariable stored encrypted in an ansible vault file and also shown in the playbook run output!
Modules:
NameParameters/Keys
debugmsgto display statements during execution we used the module debug.The key msg is used for string outputs
Notes:

In the playbook we could have used the original name of the secret variable as they were defined in the vault file but for practical reasons we prefer to store all variables in one file and when needed define secret variables separately and redefine them additionally in the non-secret file.

snipe-it

Usage/Result

Deploy a full SnipeIT installation. With an apache and Mysql database. The Result should be a fresh SnipeIT Installation available under snipeit_env_url, ready to create admin user and further configurations.

Important Variables
NameRolesNote
common_hostname_chaircommonwas defined inside the playbook and committed to the role 'common'
apache2_sitesapachecontains fqhn and the webroot
mysql_root_passmysqlMYSQL root password stored under defaults/vault.yml
mysql_user_passmysqlMYSQL user password stored under defaults/vault.yml
Basic structure

this part explains the first part of the playbook (all other parts contain almost the same basic elements and are therefore ignored).

name: Perform common Tasks                      # the name of the task that would be displayed if you run your playbook
host: snipeit                                   # name of the remote-machine you want to perform you tasks on
remote_user: i11                                # connect to the remote-machine as the named user
become: yes                                     # execute all commands as superuser -> No password is required!
gather_facts: false                             # if false is set, no facts(variables) about the remote-system
                                                # are discovered and saved
tasks:                                          
  - include_role:                               # list of directories of roles which are used
      name: roles/common
    vars:                                       # list of different variables (one variable can have more than  
      common_hostname_chair: snipeit-dev        # one attribute: -> var-name.attribute)
tags: common                                    # if you want to exclude or specify the tasks you want to
                                                # perform in your playbook use ad-hoc commands:
                                                # --tags <tag-name>; or
                                                # --skip-tags <tag-name>
Included Roles

Fore more Information about the content and modules of each role go to the next chapter

Namemain function
commonset the hostname to snipeit-dev.cm.in.tum.de
apachedeploy a apache server
mysqlsecure installation of mysql
snipeitinstallation of snipeit and preparation of all important folders and files

adduser

Usage/Result:
$ ansible-playbook plays/adduser.yml -e "username=test sudo= host=testhost" # example execution

Results in a new user test with passwordless sudo rights on every host from the [testhost] - block from the hosts file.

Variables:
NameRequiredNoteWhere to put it?
usernameyesname of the user to createwithin -e quote
sudonoif set (a simple sudo= is enough), gives the created user sudo rights (adds him to adm group)within -e quote
hostnodefine another hostblock to execute the playbook on (default is “add_users”)within -e quote
ssh-keyyeshas to exist as <username>.pubpubkeys folder
remote_useryeshas to have sudo rights and passwordless ssh login on hostsedit in playbook
Notes:

This playbook requires a ssh-key called <username>.pub in the pubkeys folder, because it automatically adds it to the authorized_keys file of the created user on the hosts. If you don't want the user to create to have sudo rights, then don't pass a sudo= as an extra variable. Same goes for the host. Only pass those variables if you want to make use of them!

ssh_keys

Usage/Result:
$ ansible-playbook plays/ssh_keys.yml -e "username=test replace= host=testhost key_to_replace=replace" # example execution replace

Replaces the key replace.key with test.key in authorized_keys file on every host in the [testhost] block from the hosts file.

$ ansible-playbook plays/ssh_keys.yml -e "username=test append=" # example execution append

Appends the key test.key to authorized_keys file of user test on every host in the [ssh_keys] block from the hosts file.

$ ansible-playbook plays/ssh_keys.yml -e "username=test delete= host=otherhost" # example execution delete

Deletes the key test.key from authorized_keys file of user test on every host in the [otherhost] block from the hosts file.

Variables:
NameRequiredNoteWhere to put it?
usernameyesname of the user to edit the ssh-keyswithin -e quote
hostnodefine another hostblock to execute the playbook on (default is “ssh_keys”)within -e quote
ssh-key <username>.pubyeshas to exist at all times for executionpubkeys folder
key_to_replacenoOptional: name of the key to replace without .pub extension (default: <username>_delete.key). It (either variant) has to exist in the pubkeys folder!within -e quote
appendOption 1If defined, appends the <username>.key to <username> authorized_keys filewithin -e quote
replaceOption 2If defined, replaces (if passed as variable) <key_to_replace>.pub or <username>_delete.pub with <username>.pub for <username> in authorized_keys filewithin -e quote
deleteOption 3If defined, deletes <username>.key from authorized_keys file of <username>within -e quote
remote_useryeshas to have sudo rights and passwordless ssh login on hostsedit in playbook
Notes:

This playbook requires a ssh-key called <username>.pub in the pubkeys folder, because it is the key, that will be appended/replaced with/deleted to/in/from the authorized_keys file of the user on the hosts. The key_to_replace variable is optional, which means you don't need to pass it for successful execution! It will try to use <username>_delete.pub instead, which has to exist in this case!
You can only pass and use one option for execution!

get_vm_users

Functionality

Parses the output of getent passwd as well as /etc/login.defs to list all local users on an OpenNebula or RBG-VM. The users are then stored in the according database. An entry for the VM is created if not already present. Note that this only works for local users; LDAP-Users will NOT be listed!

update_db

Functionality

Reads the contents of /etc/login.user.allowed and /etc/security/group.conf and creates corresponding user entries in the VM-database

lastlog

Functionality

retrieves the last date every user logged into the virtual machine and stores the result in the correct database (For either ONE or RBG-VMs). The variable users should be defined in the hostvars.

update-vm-users

Functionalit

clears and recreates the files /etc/login.user.allowed and /etc/security/group.conf, which control which users are allowed to log in and which users are granted sudo priviledges via the adm group.

Variables

NameTypeNotes
usersList of Stringdefines which users are allowed to log in. Should be defined in hostvars
sudousersList of StringThese users are added to the adm group upon login. Should be defined in hostvars

new-one-vm

Functionality

Instantiates a new OpenNebula VM and creates a corresponding database entry.

Variables

NameTypeNotes
vm_nameStringwill also be the Hostname of the VM
purposeStringWhat the VM will be used for
networkStringeither mwn or chair
usersList of StringWhich users are allowed to log in
sudousersList of StringWhich users are granted sudo access
survey_cpuFloatCPU Power allocated to VM
survey_vcpuIntegerNumber of CPUs the VM sees
survey_diskStringDisk size with suffix (KB, MB or GB)
survey_memoryStringRAM size with Suffix
servicesList of Stringlist of services to be installed on the VM (see deploy_host)
custom_varsJSON Dictionarylist of extra variables (can be used to set configurations for services, etc.)

add_ldap_users

Functionality

Creates necessary entries in /etc/login.user.allowed and /etc/security/group.conf to allow new user to log in and/or give them sudo access. Also creates or updates the corresponding database entries.

Variables

NameTypeNotes
login_usersList of StringNew users to grant ssh login access
sudo_usersList of StringUsers to grant sudo access. They are automatically also allowed to log in via ssh

common

Functionality

Changes the hostname in /etc/hosts accordingly to the common_hostname_chair variable and disables SSH root login in all instances (do NOT change this, unless you REALLY need to). It also disables password authentication for SSH by default.

Important Variables
VariableTypeFunction
common_hostname_chairStringThe system identifier under which the machine should be known to DNS and other hosts
disable_ssh_pw_loginBooleanDefault: yes. Change to no, if password authentication is desired

mysql

Functionality

Rolls out a whole MySQL setup with databases on the remote machine.

Important Variables
NameTypeFunction
mysql_root_passVault VariableMySQL root password. Edit in the vault.yml file, and ensure re-encryption after editing
mysql_user_passVault VariableDefault password for new MySQL-users. Edit in the vault.yml file
mysql_dbsArrayConsists of database name and state. Those are the databases, MySQL will have available after installation
mysql_usersArrayConsists of user name, state, password and priviliges the user will have
mysql_secure_installationBooleanDefault: yes. Deletes anonymous user “” for the database
mysql_service_enabledBooleanDefault: yes. Whether MySQL should be enabled, after installation
mysql_service_statestarted/stoppedDefault: started. The state MySQL will be, after installation

ldap_login

Functionality

Enables LDAP-Authentication on a machine. migrates local users to ldap users if possible

Note: A former configuration used the file /etc/login.user-sudo.allowed to manage which users are granted sudo access. Now every user gets an entry in /etc/security/group.conf. The task-file migrate-sudo-users.yml may be used to update the configuration and create the new entries.

Important Variables

Variable NameTypeFunction
usersList of StringLDAP users allowed to log in (should be in hostvars)
sudousersList of StringLDAP users grantedd sudo access (should be in hostvars)

apache

Functionality

Rolls out a whole Apache Installation with a custom webpage (content is needed though) and SSL and Redirects, if necessary and configured. For SSL functionality, you need to insert the required files in the ssl_certs directory of the apache role directory and define the correct filenames in the playbook (see SSL variables).

Important Variables
VariableTypeFunction
apache2_packagesArrayDefault: apache2. Packages needed for the Apache Installation
apache2_sitesArrayid: identifier used in sites-available
fqdn: Domain name used for the webpage
webroot: Directory, where the webpage content will be stored and accessed from
ip: IP adress, under which the webpage will explicitly be able to be accessed from
port: Port number used for the webpage
state: Whether the webpage will be available after installation
sslArrayport: Port number under which SSL will communicate
key_name: Name of the key file in ssl_certs
cert_name: Name of the certificate file in ssl_certs
chain_name: Name of the chain file in ssl_certs (this one is optional for SSL functionality)
apache2_remove_defaultBooleanDefault: no. Whether the Apache Default pages should be disabled during installation
apache2_service_enabledBooleanDefault: yes. Whether Apache should be enabled after installation
apache2_service_statestarted/stoppedDefault: started. The state the Apache service should be in after installation
apache2_modulesArrayid: Name of the module, which should be enabled

snipe-it

NameParameterFunction

TODO

onevm

Functionality

Provides tasks to instantiate virtual machines on OpenNebula and manage LDAP-users who have access to them.

Task Files

FileFunctionalityRequired VariablesNotes
instantiate.ymlinstantiates a new virtual machinevm_name, network, users, sudousersincludes tasks from addusers.yml
addusers.ymlgrants login/sudo access on a vm to new users by updating databaseVM_ID, users, sudouserssudousers are automatically also granted login permission
removeusers.ymlrevokes permissions on vm for specified usersVM_ID, users, sudousersusers specified in sudousers will still be able to log in, those specified in users will be revoked all permissions
updateusers.ymlupdates /etc/login.users.allowed and /etc/security/group.conf.users, sudousers

Variables

Variable NameTypeFunction
vm_nameStringName of the virtual machine to instantiate
networkStringNetwork to attach to VM. Either “chair” or “mwn”
usersList of StringUsers who are allowed to log in
sudousersList of StringUsers who are allowed sudoaccess

In general you are going to execute one Playbook. This Playbook consists of tasks, Roles or Modules which will be performed on the remote-machines you have listed on top of each task. If you wish to work with Variables you can define them either directly inside a playbook, or in a different file (see: directory structure). The concept Ansible Scripts are managed in this directory structure:

group_vars/            # variables for specific host groups
    all.yml            # variables which can be used by all roles/playbooks
    dbservers/         # variables for e.g. dbservers host group
        vars.yml       # non secret variables
        vault.yml      # secret variables protected by ansible vault
    ...                # other group folders as dbserver
play/                  # folder with playbooks as one yml files
roles/                 # contains roles used in the playbooks
    common/            # folder with the role 'common' used for all servers, vms, etc.
        defaults/      # contains default variables for role named 'common'
            main.yml   # non-secret variables
            vault.yml  # secret variables which must be encryptd by ansible vault!
        tasks/         # tasks for common role
            main.yml   # task file
        files/         #contains files which can be deployed via this role
        handlers/      #contains handlers, which may be used by this role or even anywhere outside this role
    mysql/             # mysql role folder
ansible.cfg            # scripts specific ansible variables, e.g. point to vault password
hosts                  # text file with host groups, should contain all servers
vpass                  # ansible vault file containing the vault password, ignored by git!
  • Add a new group_vars folder for the group your playbook is written for, create one vars and one vault yml file
  • Add group variables to the vars file, copy secret variables to vaul file and prepend with vault_, link values to vars file, e.g.
    mysql_root_pass: '{{ vault_mysql_root_pass }}'
  • This can also be done for specific roles under ..role/defaults/vault.yml, it must be separately included in the role tasks file ..role/tasks/main.yml
    - name: Include Vault Variables
      include_vars: ../defaults/vault.yml 
  • Encrypt vault file with ansible-vault
    ansible-vault encrypt group_vars/dbservers/vault.yml
    # use different command for decryption in order to change passwords, etc.
    ansible-vault decrypt group_vars/dbservers/vault.yml 

During the execution of an ansible-playbook passwords may be to be set for services. We save those passwords in the vault-file and encrypt those. In order to let ansible properly decrypt the files, ask your supervisor for the vault-password, safe it as vpass in the root directory of the repository and execute the Makefile.
It is very important to execute the Makefile at least once after cloning the repository and before the first commit. The Makefile sets up git hooks, that check whether every vault file in the repository is encrypted before every commit.