To receive notifications about scheduled maintenance, please subscribe to the mailing-list gitlab-operations@sympa.ethz.ch. You can subscribe to the mailing-list at https://sympa.ethz.ch

Commit b2a4430d authored by Bengt Giger's avatar Bengt Giger
Browse files

Added ansible_client

parent 7ed4a2c7
Pipeline #89233 failed
......@@ -19,25 +19,15 @@ before_script:
- molecule --version
# Test timeservice on dedicated gitlab-runner server
timeservice-chrony-test:
timeservice-test:
stage: tests
tags:
- docker
script:
- cd roles/timeservice;
molecule test;
only:
changes:
- .gitlab-ci.yml
- roles/timeservice/**/*
timeservice-ntp-test:
stage: tests
tags:
- docker
script:
- cd roles/timeservice;
molecule test -s ntp;
only:
only:
changes:
- .gitlab-ci.yml
- roles/timeservice/**/*
......@@ -103,3 +93,16 @@ rsyslog_forwarder-test:
changes:
- .gitlab-ci.yml
- roles/rsyslog_forwarder/**/*
# Test ansible_client on dedicated runner (systemd)
ansible_client-test:
stage: tests
tags:
- docker
script:
- cd roles/;
molecule test;
only:
changes:
- .gitlab-ci.yml
- roles/ansible_client/**/*
......@@ -8,6 +8,7 @@ Collection of configurations specific for ETH Zurich:
- `console_configuration`: configure system console for Swiss keyboards (and others)
- `timeservice`: timezone, `chrony` and `ntp` settings for Zurich
- `system_update_manager`: Install OS updates, reboot system if necessary
- `ansible_client`: manage pull and cron jobs, and vault password
# Installation
......
......@@ -16,8 +16,8 @@ readme: README.md
# A list of the collection's content authors. Can be just the name or in the format 'Full Name <email> (url)
# @nicks:irc/im.site#channel'
authors:
- bgiger
- Bengt Giger <bgiger@ethz.ch>
- Dan Hochstrasser <danh@ethz.ch>
### OPTIONAL but strongly recommended
# A short summary description of the collection
......
Ansible Job Installation
========================
This role will call an `ansible-pull` which
- runs at the end of this role
- runs at every reboot
- is executed regularly
These modes may be combined as desired.
Distributions tested with Molecule:
- CentOS 7/8
- Debian 10 Buster
- Ubuntu 20.04
Usage
-----
- ansible_pull_repository: by default, this variable is empty. A git URL must be supplied, either:
- to a public repo URL
- to a private URL, with an access key, i.e. https://oauth2:<gitlab_access_key>@gitlab.ethz.ch/id-cd-lnx/ansible/id-cd-lnx.git
- ansible_pull_inventory: defaults to "inventory"
- ansible_pull_playbook: defaults to "site.yml"
- ansible_pull_tags: defaults to empty
- ansible_pull_checkout: defaults to "master", may be a tag, branch or commit
- ansible_pull_sleep: if you install a cronjob, this parameter will randomly
delay the configuration run to reduce the load on the git server. Defaults
to 1740, which means the run will be delayed by 0 to 1740 seconds. Should be
near the interval of the cronjob.
- ansible_pull_cron_user: user to run the cronjob (default: root)
- ansible_pull_cron_schedule: cron job schedule, default '0/30 * * * *'
- ansible_pull_install_postboot: install a job which executes ansible-pull at
the end of a system boot (default: true)
- ansible_pull_run_once: run ansible pull once during the execution of this role
(default: true).
- ansible_pull_vault_password: sets a vault password for the ansible run
- ansible_pull_vault_pw_file_location: defines the location of the vault password file
You may also prepare a dedicated user to push configurations later, using priviledge escalation. Variables are:
- ansible_push_user_create: set to true to manage the user; defaults to false
- ansible_push_user_name: defaults to "ansible"
- ansible_push_user_home: home directory (/var/lib/ansible)
- ansible_push_user_sudo: install a configuration in /etc/sudoers.d. Default: true
- ansible_push_user_sudo_perms: permissions for sudo, defaults to "ALL=(ALL) NOPASSWD: ALL"
- ansible_push_user_password: set a password, if desired. Defaults to "" (set no password)
- ansible_push_user_sshkey: provide a ssh public key (empty, no default)
---
ansible_pull_install_cronjob: true
ansible_pull_cron_user: root
ansible_pull_cron_schedule: '0,30 * * * *'
ansible_pull_install_postboot: true
ansible_pull_run_once: true
ansible_pull_vault_password: ""
ansible_pull_vault_pw_file_location: "~/.vault_pw.txt"
ansible_pull_repository: ""
ansible_pull_inventory: inventory
ansible_pull_playbook: site.yml
ansible_pull_tags: ""
ansible_pull_vars: ""
ansible_pull_checkout: master
ansible_pull_sleep: 1740
ansible_pull_pre_cmd: ""
ansible_pull_post_cmd: "| /usr/local/bin/ansible-logger.py"
ansible_pull_directory: /tmp/pull_run
ansible_push_user_create: false
ansible_push_user_name: ansible
ansible_push_user_home: /var/lib/ansible
ansible_push_user_sudo: true
ansible_push_user_sudo_perms: "ALL=(ALL) NOPASSWD: ALL"
ansible_push_user_password: ""
ansible_push_user_sshkey: ""
ansible_pip_use: false
ansible_pip_ansible_version: 2.10.6
#!/usr/bin/python2
import sys
import json
import logging, logging.handlers
FORMAT='%(asctime)-15s %(message)s'
logging.basicConfig(format=FORMAT)
logger = logging.getLogger("ansible")
handler = logging.handlers.SysLogHandler(address = "/dev/log")
logger.addHandler(handler)
ansible_log = sys.stdin
logitems = json.load(ansible_log)
failed_tasks = []
changed_tasks = []
for play in logitems["plays"]:
for task in play["tasks"]:
for host in task["hosts"]:
# for item in task["hosts"][host]:
if "failed" in task["hosts"][host].keys() and task["hosts"][host]["failed"] == True:
failed_tasks.append(task)
if "changed" in task["hosts"][host].keys() and task["hosts"][host]["changed"] == True:
changed_tasks.append(task)
if len(failed_tasks) > 0:
logger.setLevel(logging.ERROR)
print("Failed Tasks")
print("------------")
for task in failed_tasks:
logger.error("ansible-cron: %s", "failed: %s" % task["task"]["name"])
if len(changed_tasks) > 0:
logger.setLevel(logging.WARN)
print("Changed Tasks")
print("-------------")
for task in changed_tasks:
logger.warning("ansible-cron: %s", "changed: %s" % task["task"]["name"])
---
- name: Converge
hosts: all
tasks:
- name: "Include ansible_client"
include_role:
name: "ansible_client"
vars:
ansible_pull_vault_password: "1234"
ansible_pull_repository: https://gitlab.ethz.ch/ansible-community/collections/system_configuration.git
ansible_pull_run_once: false
ansible_pull_tags: mytag
ansible_pull_vars: "{'myvar':'myvalue'}"
ansible_push_user_create: true
ansible_pip_use: true
---
dependency:
name: galaxy
driver:
name: docker
lint: |
set -e
yamllint -c molecule/default/yaml-lint.yml .
ansible-lint -c molecule/default/ansible-lint.yml tasks
platforms:
- name: el7
# systemd enabled container setup
image: geerlingguy/docker-centos7-ansible:latest
command: ""
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
privileged: true
pre_build_image: true
- name: el8
# systemd enabled container setup
image: geerlingguy/docker-centos8-ansible:latest
command: ""
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
privileged: true
pre_build_image: true
- name: debian10
# systemd enabled container setup
image: geerlingguy/docker-debian10-ansible:latest
command: ""
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
privileged: true
pre_build_image: true
- name: ubuntu2004
# systemd enabled container setup
image: geerlingguy/docker-ubuntu2004-ansible:latest
command: ""
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
privileged: true
pre_build_image: true
provisioner:
name: ansible
verifier:
name: ansible
inventory:
host_vars:
instance:
ansible_user: ansible
---
- name: Prepare
hosts: all
tasks:
- name: Update apt cache
apt:
update_cache: true
when: ansible_os_family == "Debian"
- name: Ensure /etc/cron.d exists (missing in some Docker images)
file:
path: /etc/cron.d
state: directory
---
# This is an example playbook to execute Ansible tests.
- name: Verify
hosts: all
remote_user: ansible
become: true
become_user: root
tasks:
- name: Retrieve version of /usr/local/bin/ansible
command: /usr/local/bin/ansible --version
register: ansible_version
- name: Test if current ansible is installed by pip
assert:
that: "ansible_version.major == 2 and ansible_version.minor == 10"
- name: Get content of vault password file
command: "cat /root/.vault_pw.txt"
register: vaultfile
- name: Test if vault password is set
assert:
that: "'1234' is in vaultfile.stdout"
- name: Get content of ansible-pull script
command: "grep ansible-playbook /usr/local/bin/ansible_pull.sh"
register: playbook_cmd
- name: Test if inventory is set
assert:
that: "'-i inventory' is in playbook_cmd.stdout"
- name: Test if vault file is set
assert:
that: "'--vault-password-file ~/.vault_pw.txt' is in playbook_cmd.stdout"
- name: Test if tags are set
assert:
that: "'-t mytag' is in playbook_cmd.stdout"
- name: Test if vars are set
assert:
that: "'myvalue' is in playbook_cmd.stdout"
- name: Get content of crontab file
command: "cat /etc/cron.d/ansible_pull"
register: crontab
- name: Test if crontab is correct
assert:
that: "'0,30 * * * * root /usr/local/bin/ansible_pull.sh' is in crontab.stdout"
- name: Get content of postboot service
command: "grep ExecStart /etc/systemd/system/ansible_postboot.service"
register: postboot
- name: Test if firstboot service is configured
assert:
that: "'ExecStart=/usr/local/bin/ansible_pull.sh --firstboot' is in postboot.stdout"
- name: Get firstboot systemd unit state
command: systemctl status ansible_postboot.service
register: postboot_service
ignore_errors: true
- name: Test if firstboot service is enabled
assert:
that: "'ansible_postboot.service; enabled' is in postboot_service.stdout"
- name: Get latest log entries
shell: journalctl -n 100 | grep "USER=root" | grep BECOME-SUCCESS
register: journal
- name: Verify that privilege escalation was used
assert:
that: journal.stdout_lines | length > 0
---
extends: default
rules:
line-length:
max: 140
level: warning
---
- name: Ensure pip is installed
package:
name: python3-pip
- name: Update pip to latest version
pip:
name: pip
state: latest
executable: pip3
- name: Install ansible from PyPi
pip:
name: ansible
version: "{{ ansible_pip_ansible_version }}"
executable: pip3
---
- name: Ensure ansible user exists
user:
name: "{{ ansible_push_user_name }}"
system: true
home: "{{ ansible_push_user_home }}"
- name: Set password for ansible user
user:
name: "{{ ansible_push_user_name }}"
password: "{{ ansible_push_user_password }}"
when: ansible_push_user_password | length != 0
- name: Install ansible sudoer permissions
template:
src: templates/ansible.sudoers.j2
dest: /etc/sudoers.d/ansible
mode: 0600
when: ansible_push_user_sudo
- name: Install ansible ssh key
authorized_key:
user: "{{ ansible_push_user_name }}"
key: "{{ ansible_push_user_sshkey }}"
when: ansible_push_user_sshkey | length != 0
---
- name: Install python3 on EL7, not standard
yum:
name: python3
when: ansible_os_family | lower == "redhat" and ansible_distribution_major_version | int == 7
- name: Ensure required ansible-client packages are installed
package:
name: "{{ ansible_client_packages }}"
state: latest
when: ansible_pull_install_postboot or ansible_pull_run_once
- name: Install ansible from PyPi if selected
include_tasks: ansible_pip.yml
when: ansible_pip_use
- name: Install Ansible pull script
block:
- name: Copies vault password file
template:
src: templates/vault_pw.txt
dest: "{{ ansible_pull_vault_pw_file_location }}"
mode: 0600
when: ansible_pull_vault_password | length != 0
- name: Install pull script
template:
src: templates/ansible_pull.sh.j2
dest: /usr/local/bin/ansible_pull.sh
mode: 0755
- name: Install log formatter
copy:
src: ansible-logger.py
dest: /usr/local/bin/ansible-logger.py
mode: 0755
- name: Ensure python2 is available
package:
name: python2
when:
- ansible_pull_repository
- ansible_pull_install_postboot or ansible_pull_run_once or ansible_pull_install_cronjob
- name: Install Ansible post boot execution
template:
src: templates/ansible_postboot.service.j2
dest: /etc/systemd/system/ansible_postboot.service
mode: "0644"
when:
- ansible_pull_repository | length != 0
- ansible_pull_install_postboot or ansible_pull_run_once
- name: Activate Ansible post boot execution
systemd:
name: ansible_postboot
enabled: true
daemon_reload: true
when:
- ansible_pull_install_postboot
- ansible_pull_repository | length != 0
- name: Get service facts
service_facts:
- name: Deactivate Ansible post boot execution
systemd:
name: ansible_postboot
enabled: false
daemon_reload: true
when: not ansible_pull_install_postboot and 'ansible_postboot.service' in ansible_facts.services
- name: Install Ansible pull cronjob
template:
src: templates/ansible_pull.j2
dest: /etc/cron.d/ansible_pull
mode: 0600
when:
- ansible_pull_install_cronjob
- ansible_pull_repository | length != 0
- name: Uninstall Ansible pull cronjob
file:
path: /etc/cron.d/ansible_pull
state: absent
when:
- not ansible_pull_install_cronjob
- name: Run Ansible playbook once
systemd:
name: ansible_postboot
state: restarted
daemon_reload: true
when:
- ansible_pull_run_once
- ansible_pull_repository | length != 0
- name: Create ansible user
include_tasks: ansible_user.yml
when: ansible_push_user_create
{{ ansible_push_user_name }} {{ ansible_push_user_sudo_perms }}
[Unit]
Description=Run Ansible after reboot
After=default.target
[Service]
Type=oneshot
ExecStart=/usr/local/bin/ansible_pull.sh --firstboot
[Install]
WantedBy=default.target
# Managed by Ansible
{{ ansible_pull_cron_schedule }} {{ ansible_pull_cron_user }} /usr/local/bin/ansible_pull.sh
#!/bin/bash
sleeptime={{ ansible_pull_sleep }}
firstboot=0
case "$1" in
--nowait)
sleeptime=0
;;
--firstboot)
sleeptime=0
firstboot=1
;;
*)
true
;;
esac
{# Will fill the variable ansible_pull_vault_pw_file_location_arg if ansible pull vault password is defined #}
{%- set ansible_pull_vault_pw_file_location_arg = "" -%}
{%- if ansible_pull_vault_password != "" -%}
{%- set ansible_pull_vault_pw_file_location_arg="--vault-password-file " + ansible_pull_vault_pw_file_location -%}
{%- endif -%}
{# Tags are filled during installation of this script. May look a bit
# weird if there were no tags.
#
# If no tags had been supplied, the next part will be skipped and the ansible_pull_tags_arg variable is empty #}
{%- set ansible_pull_tags_arg = "" -%}
{%- if ansible_pull_tags != "" -%}
{%- set ansible_pull_tags_arg="-t " + ansible_pull_tags -%}
{%- endif -%}
{# Same procedure for more variables #}
{%- set ansible_pull_vars_arg = "" -%}
{%- if ansible_pull_vars != "" -%}
{%- set ansible_pull_vars_arg="-e " + ansible_pull_vars -%}
{%- endif -%}
sleep $sleeptime
attempts=0
while :
do
git clone --quiet --branch {{ ansible_pull_checkout }} {{ ansible_pull_repository }} {{ ansible_pull_directory }}
rc=$?
if [ "$rc" == "0" ]; then
break
# 128: could not connect
elif [ "$rc" != "0" ]; then
attempts=$(($attempts))+1
if [ $(($attempts)) -gt 10 ];
then
exit $rc
fi
fi
done
if [ "$firstboot" == "0" ]; then
export ANSIBLE_STDOUT_CALLBACK=json
fi
cd {{ ansible_pull_directory }}
{{ ansible_pull_pre_cmd }} ansible-playbook --connection=local -i {{ ansible_pull_inventory }} -l $(hostname -s) {{ ansible_pull_vault_pw_file_location_arg }} {{ ansible_pull_tags_arg }} {{ ansible_pull_vars_arg }} {{ ansible_pull_playbook }} {{ ansible_pull_post_cmd }}
ansible_rc=${PIPESTATUS[0]}
if [ "$ansible_rc" == 0 ]
then
rm -rf {{ ansible_pull_directory }}
else
echo "#### ERROR while executing ansible-playbook return code: $ansible_rc ####"
rm -rf {{ ansible_pull_directory }}
exit $ansible_rc
fi
# satellite-register may update all packages, but excludes Ansible. During first boot of a newly
# registered VM, we have to update Ansible separately