DEBOPS.RESOURCES(5)

NAME
debops.resources - Manage custom file resources through Ansible inventory

SYNOPSIS

debops run service/resources [--limit group,host,...] [--diff] [--check] [--tags tag1,tag2,...] [--skip-tags tag1,tag2,...] [<ansible-playbook options>] ...

DESCRIPTION

This role allows management of custom file paths, file contents and archives on remote hosts without the need to create a separate Ansible role. You can use inventory lists to define what files to copy, what directories should exist on a remote host, what online resources to download, and so on.

GETTING STARTED

  Example inventory

The debops.resources role is included by default in the common.yml DebOps playbook, you don't need to add hosts to any groups to enable it.

Role provides a special variable, resources__src which points to ansible/resources/ directory located in the DebOps project directory, relative to the currently used Ansible inventory. This variable can be used in the item.src keys of the file/archive lists to use the files from a central location relative to the current DebOps project directory.

The ansible/resources/ directory is not created automatically, and role does not check the existence of the specified files before using them.

An example usage:

resources__host_paths:
  - '/tmp/example-dir'
resources__host_files:
  - src: '{{ resources__src + "file.txt" }}'
    dest: '/tmp/example-dir/file.txt'

  Example playbook

If you are using this role without DebOps, here's an example Ansible playbook that uses the debops.resources role:

---
- name: Manage custom resources
  collections: [ 'debops.debops', 'debops.roles01',
                 'debops.roles02', 'debops.roles03' ]
  hosts: [ 'debops_all_hosts', 'debops_service_resources' ]
  become: True
  environment: '{{ inventory__environment | d({})
                   | combine(inventory__group_environment | d({}))
                   | combine(inventory__host_environment  | d({})) }}'
  roles:
    - role: resources
      tags: [ 'role::resources', 'skip::resources' ]

  Ansible tags

You can use Ansible --tags or --skip-tags parameters to limit what tasks are performed during Ansible run. This can be used after a host was first configured to speed up playbook execution, when you are sure that most of the configuration is already in the desired state.

Available role tags:

role::resources
    Main role tag, should be used in the playbook to execute all tasks.

role::resources:paths
    Manage paths on the remote hosts.

role::resources:urls
    Manage online resources on the remote hosts.

role::resources:archives
    Unpack archives on the remote hosts.

role::resources:files
    Manage file contents on the remote hosts.

role::resources:delayed_paths
    Manage delayed paths on the remote hosts.

role::resources:commands
    Run specified commands on the remote hosts.

role::resources:capabilities
    Manage file privileges on the remote hosts.

DEFAULT VARIABLE DETAILS

Some of debops.resources default variables have more extensive configuration than simple strings or lists, here you can find documentation and examples for them.

  resources__templates

The debops.resources role supports dynamic generation of directories, templated files and symlinks using the with_filetree <https://docs.ansible.com/ansible/2.5/plugins/lookup/filetree.html> Ansible lookup plugin.

The file, directory and symlink management is limited - the managed resources will be owned by root UNIX account and will be placed in the root UNIX group, however the specific file mode will be preserved; for example if you create a file with 0600 permissions, the same permissions will be set by the role on the remote host. You can use the resources__paths functionality to modify file/directory ownership afterwards.

WARNING: The task ensures that each directory in the path exists, including permissions. You have to set specific permissions for certain directories like /root (0700) or /tmp (1777) in order to not modify them in unexpected manner.

For this functionality to work, the role expects a specific directory structure located in the ansible/resources/ directory (or wherever the resources__src variable points to):

ansible/resources/
└── templates/
    ├── by-group/
    │   ├── all/
    │   ├── group-name1/
    │   └── group-name2/
    └── by-host/
        ├── hostname1/
        └── hostname2/

The with_filetree Ansible lookup plugin will look for resources to manage in specific hostname directory, then of all the groups the current host is in (based on the content of the variable group_names), then in the by-group/all/ directory. The resource found first in this order wins and no further checks are performed; this means that you can put a file in the by-group/all/ directory and then override it using a host-specific directory. The groups directories are read in the order dictated by Ansible during inventory parsing.

See Ansible - Playbooks Variables <https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html#accessing-information-about-other-hosts-with-magic-variables> to learn about the group_names variable, and Ansible - Working with Inventory <https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html#how-variables-are-merged>

for more information on how to use ansible_group_priority to change the merge order for groups of the same level (after the parent/child order is resolved).

Each directory structure starts at the root of the filesystem (/), so to create a file in a subdirectory you need to recreate the entire path. For example, to create the /var/lib/application/custom.txt file, it needs to be placed in:

ansible/resources/templates/by-group/all/var/lib/application/custom.txt

In the templates, you can reference variables from the Ansible facts (including local facts managed by other roles) and Ansible inventory. Referencing variables from other roles might work only if these roles are included in the playbook, however that is not idempotent and should be avoided.

  resources__paths

These lists can be used to create directories, symlinks, set permissions and ownership, etc. Each element of the list is a YAML dictionary with a set of parameters. See the documentation of the Ansible ansible.builtin.file module <https://docs.ansible.com/ansible/latest/collections/ansible/builtin/file_module.html> for details about what parameters can be used and their format. Here's are additional details for certain parameters:

item.path or item.dest or item.name
    Specify absolute path of the target directory/file on the remote host. If not specified, the entire entry is treated as a directory path.

item.state
    Optional. Specify state of the given path. If not specified, the element is treated as a directory which will be created if it doesn't exist.

item.acl
    Optional. Please take a look ACL support section.

item.access_time and item.modification_time
    Optional. As documented by the Ansible ansible.builtin.file module <https://docs.ansible.com/ansible/latest/collections/ansible/builtin/file_module.html> except that DebOps defines a more human readable and standard compliant format. Example: 2023-05-23T23:42:42

item.access_time_format and item.modification_time_format
    Optional. If not defined, it defaults to resources__time_format.

  Examples

Create a set of directories on all hosts:

resources__paths:
  - '/tmp/dir1'
  - '/tmp/dir2'

Create a public WWW directory on the user account and symlink it to the webroot directory served by the HTTP server:

resources__host_paths:
  - path: '/home/user1/public'
    owner: 'user1'
    group: 'user1'
    mode: '0755'
  - path: '/srv/www/sites/example.com'
    state: 'directory'
  - path: '/srv/www/sites/example.com/public'
    src:  '/home/user1/public'
    state: 'link'

Remove specified path:

resources__paths:
  - path: '/tmp/removed'
    state: 'absent'

  resources__repositories

These lists can be used to clone or update remote git repositories. You can use all parameters of the git Ansible module to manage the repositories, with some exceptions. The role recognizes these additional parameters:

item.repo, item.url or item.src
    Required. The URL of the git repository to clone..

item.dest or item.name or item.path
    Required. Path where the specified repository should be cloned to.

_update
    Optional, boolean. This is a replacement of the update git module parameter, due to the string being a reserved word in Python. You can use this to enable or disable repository update.

owner
    Optional. If specified, the role will use the Ansible become functionality to switch to a specified UNIX user account before cloning the repository. The account must exist on the host before it can be used. If not specified, the role will use the root account.

    The specified UNIX account needs to have access to the destination directory. The parent directories are created automatically, as long as the access permissions allow. You can create or change directory permissions as needed using the resources__paths variables.

item.acl
    Optional. Please take a look ACL support section.

  Examples

Clone the Ansible repository to the host:

resources__repositories:
  - repo: 'https://github.com/ansible/ansible'
    dest: '/usr/local/src/github.com/ansible/ansible'

Clone a private repository, accessible using a SSH key. The UNIX account specified as the owner, or root account when otherwise, needs to have the SSH key accepted by the repository. This example uses Gitea <https://gitea.io/> instance as the source of the git repository:

resources__repositories:
  - repo: 'ssh://git@git.example.org:29418/namespace/repository.git'
    owner: 'username'
    dest: '~username/src/git.example.org/namespace/repository'
    accept_hostkey: True

  resources__urls

These lists can be used to download online resources over HTTP, HTTPS or FTP protocols. Each element of a list is a YAML dictionary with parameters. You can use all parameters of the get_url Ansible module; see its documentation for the parameter list and syntax.

To download resources over HTTPS, the content must be served over a valid TLS/SSL certificate recognized by the remote host. If you use self-signed certificates, check the debops.pki for how to add custom Root CA Certificates on your hosts.

Here are some important parameters used by the role:

item.url or item.src
    Required. The URL of the resource to download.

item.dest or item.name or item.path
    Required. Path where downloaded resource should be stored.

item.acl
    Optional. Please take a look ACL support section.

  Examples

Download a HTML file from a webserver on all hosts:

resources__urls:
  - src: 'http://www.example.com/page.html'
    dest: '/tmp/page.html'

  resources__archives

These lists can be used to unpack archives located on Ansible Controller to remote hosts. Each element of the list is a YAML dictionary with parameters recognized by the Ansible ansible.builtin.unarchive module <https://docs.ansible.com/ansible/latest/collections/ansible/builtin/unarchive_module.html>. For details about their use, see the module documentation.

The resources__src variable can be used to point the role to a custom, central location, by default located in the DebOps project directory.

Here are some more important parameters:

item.src
    Required. Path to the archive located on Ansible Controller.

item.dest or item.name or item.path
    Required. Path on the remote host where the archive should be unpacked.

item.acl
    Optional. Please take a look ACL support section.

  Examples

Unpack the home directory contents of a particular user on a specific host. The tarball is located at ansible/resources/home.tar on the Ansible Controller, in DebOps project directory:

resources__host_archives:
  - src: '{{ resources__src + "home.tar" }}'
    dest: '/home/user'
    owner: 'user'
    group: 'user'

  resources__files

These lists can be used to manage content or copy files from the Ansible Controller to remote hosts. Each element of a list is a YAML dictionary with parameters used by the Ansible ansible.builtin.copy module <https://docs.ansible.com/ansible/latest/collections/ansible/builtin/copy_module.html>. See its documentation for parameter advanced usage and syntax.

The resources__src variable can be used to point the role to a custom, central location, by default located in the DebOps project directory.

Here are some more important parameters:

item.dest or item.name or item.path
    Required. Path to the destination file on the remote host.

item.src
    Path to the source file on the Ansible Controller. Alternatively you can use item.content to provide the file contents directly in the inventory.

item.content
    String or YAML text block with the file contents to put in the destination file. Alternatively you can use item.src to provide the path to the source file on Ansible Controller.

item.state
    Optional. If not specified, or if specified and present, the file(s) will be created. If specified and absent, file will be removed.

item.acl
    Optional. Please take a look ACL support section.

  Examples

Copy file from the ansible/resources/ directory to all remote hosts:

resources__files:
  - src: '{{ resources__src + "path/to/file" }}'
    dest: '/tmp/file'

Create a custom cron task that restarts a service daily:

resources__host_files:
  - dest: '/etc/cron.daily/service-restart'
    mode: '0755'
    content: |
      #!/bin/sh
      # {{ ansible_managed }}
      test -x /usr/bin/service && systemctl restart service

  Python Virtual Environment management

The debops.resources role can be used to create and manage a Python Virtual Environment through the use of the ansible.builtin.pip <https://docs.ansible.com/ansible/latest/collections/ansible/builtin/pip_module.html> Ansible module. Users can create new virtual environments and install Python packages inside of them.

Virtual environments are defined using resources__*_pip default variables. Each variable is a list of YAML dictionaries with ansible.builtin.pip module parameters (check its documentation for details). Some of the more important parameters:

name
    String or a YAML list of Python packages to install in a virtual environment (if defined) or system-wide.

virtualenv
    Absolute path to the directory where Python virtual environment should be deployed and managed. If not specified, the role will install Python packages system-wide in /usr/local/ subdirectories.

owner
    UNIX account which should be the owner of a given Python environment. If not specified, root UNIX account will be used by default.

  Examples

Create an unprivileged UNIX account using the debops.users Ansible role. Create a virtualenv inside of the home directory and install Ansible and DebOps Python packages:

users__host_accounts:
  - name: 'deploy'
    group: 'deploy'
    home: '/home/deploy'
resources__host_pip:
  - owner: 'deploy'
    virtualenv: '/home/deploy/venv/debops'
    name:
      - 'ansible-core'
      - 'debops'

  ACL support

Some of debops.resources variables also have the possibility to manage the ACLs (resources__paths, resources__repositories, resources__urls, resources__archives and resources__files).

  Examples

Create a directory on all hosts and allow adm group to access to any new content:

resources__paths:
  - dest: '/tmp/dir1'
    acl:
      - default: True
        etype: 'group'
        entity: 'adm'
        permissions: 'rX'
      - default: True
        etype: 'user'
        entity: 'joe'
        permissions: 'rX'

Remove ACLs related to joe user on a file on all hosts:

resources__files:
  - dest: '/tmp/file'
    state: 'present'
    acl:
      - etype: 'user'
        entity: 'joe'
        state: 'absent'

  Parameters related to ACL

item.acl
    Optional. Configure filesystem ACL entries of the current file or directory. This parameter is a list of YAML dictionaries. See the documentation of the Ansible ansible.posix.acl module <https://docs.ansible.com/ansible/latest/collections/ansible/posix/acl_module.html> for details about each parameters (what they can be used to and their format) as well as the acl(5) <https://manpages.debian.org/acl(5)>, setfacl(1) <https://manpages.debian.org/setfacl(1)> and getfacl <https://manpages.debian.org/getfacl> manual pages. Some useful parameters:

default
    Optional, boolean. If True, set a given ACL entry as the default for new files and directories inside a given directory. Only works with directories and can't be removed with state set to absent.

entity
    Name of the UNIX user account or group that a given ACL entry applies to.

etype
    Specify the ACL entry type to configure. Valid choices: user, group, mask, other.

permissions
    Specify the permission to apply for a given ACL entry. This parameter cannot be specified when the state of an ACL entry is set to absent.

recursive
    Apply a given ACL entry recursively to all entities in a given path.

state
    Optional. If not specified or present, the ACL entry will be created. If absent, the ACL entry will be removed. The query state doesn't make sense in this context and shouldn't be used.

  resources__replacements

The resources__*_replacements variables can be used to modify existing files using regexp matches. The role uses the ansible.builtin.replace <https://docs.ansible.com/ansible/latest/collections/ansible/builtin/replace_module.html> Ansible module to perform the operation. All parameters of this module should be supported.

  Examples

Remove the use_authtok token from the PAM common password configuration file:

resources__replacements:
  - path: '/etc/pam.d/common-password'
    regexp: 'use_authtok\s+enforce_for_root'
    replace: 'enforce_for_root'

  Syntax

The resources__*_replacements variables are defined as list of YAML dictionary entries with specific parameters. Check the documentation of the ansible.builtin.replace module to see the available parameters:

ansible-doc -t module ansible.builtin.replace

  resources__commands

The resources__*_commands variables can be used to define shell commands or small scripts which should be executed on the remote hosts. This can be useful to, for example, start a systemd service created previously using the resources__files variables.

This is not a replacement for a fully-fledged Ansible role. The interface is extremely limited, and you need to ensure idempotency inside of the script or command you are executing. The debops.resources role can be executed at different points in the main playbook, which you should also take into account.

  Examples

Set up a simple example systemd service and start it:

resources__files:
  - content: |
      [Unit]
      Description=Example Service
      [Service]
      Type=simple
      ExecStart=/bin/true
      RemainAfterExit=yes
      [Install]
      WantedBy=multi-user.target
    dest: '/etc/systemd/system/example.service'
    mode: '0644'
resources__commands:
  - name: 'Reload systemd and start example service'
    shell: |
      if ! systemctl is-active example.service ; then
          systemctl daemon-reload
          systemctl start example.service
      fi

  Syntax

Each shell command entry is defined by a YAML dictionary with specific parameters:

name
    Required. A name of a given shell command displayed during Ansible execution, not used for anything else in the task. Multiple configuration entries with the same name parameter are merged together.

script / shell / command
    Required. String or YAML text block that contains the command or script to execute on the remote host. The contents will be passed to the shell Ansible module.

chdir
    Optional. Specify the path to the directory on the remote host where the script should be executed.

creates
    Optional. Specify the path of the file on the remote host - if it's present, the shell module will not execute the script.

removes
    Optional. Specify the path of the file on the remote host - if it's absent, the shell module will not execute the script.

executable
    Optional. Specify the command interpreter to use. If not specified, /bin/bash will be used by default.

state
    Optional. If not specified or present, the shell command will be executed as normal by the role. If absent, the shell command will not be executed by the role. If ignore, the configuration entry will not be evaluated by the role during execution. This can be used to conditionally activate and deactivate different shell commands on the Ansible level.

no_log
    Optional, boolean. If True, Ansible will not display the task contents or record them in the log. It's useful to avoid recording sensitive data like passwords.

  resources__capabilities

These lists allow you to manipulate file privileges using the Linux capabilities(7) system. You can use all parameters supported by the Ansible community.general.capabilities module <https://docs.ansible.com/ansible/latest/collections/community/general/capabilities_module.html>. The role recognizes these additional alias for parameters:

item.path or item.name
    Required. Specifies the path to the file to be managed

  Examples

Allow Bluetooth LE scanning as non root:

resources__host_file_capabilities:
  - path: '/usr/bin/hcitool'
    capability: 'cap_net_raw,cap_net_admin+eip'

AUTHOR
Maciej Delmanowski

COPYRIGHT
2014-2024, Maciej Delmanowski, Nick Janetakis, Robin Schneider and others