DEBOPS.METRICBEAT(5)

NAME
debops.metricbeat - Manage Metricbeat service using Ansible

SYNOPSIS

debops run service/metricbeat [--limit group,host,...] [--diff] [--check] [--tags tag1,tag2,...] [--skip-tags tag1,tag2,...] [<ansible-playbook options>] ...

DESCRIPTION

Metricbeat <https://www.elastic.co/beats/metricbeat>, from Elastic <https://www.elastic.co/> is part of the Elastic Stack. Metricbeat can be used to gather various metrics (CPU usage, disk I/O, application status, etc.) from host and services and send them to Elasticsearch database for processing.

The metricbeat Ansible role configures Metricbeat on Debian/Ubuntu hosts. The software itself will be installed using the debops.extrepo Ansible role.

GETTING STARTED

  Upstream package is used by default

The debops.metricbeat role depends on the debops.extrepo Ansible role to configure access to the Elastic.co APT repository. This means that usually the latest available Metricbeat release will be installed by default. If you require older releases, you can use the debops.apt_preferences role to select the desired package version.

Alternatively, you can "mask" the metricbeat__extrepo__dependent_sources variable in the Ansible inventory and configure the APT repositories yourself via the debops.apt role. You can also use the debops.elastic_co role to configure Elastic APT repositories specifically.

  Default configuration

The metricbeat role includes a basic configuration for Metricbeat which will by default enable metrics in the system module.

The default configuration does not try to connect to any existing Elasticsearch cluster. You should finish the configuration in the Ansible inventory to tailor the role to your own environment. An example configuration for an Elasticsearch + Kibana environment is provided below:

---
# Example configuration for Metricbeat deployment with an Elasticsearch cluster
# as the data destination. Kibana is used as the UI layer.
---
# Configuration for all Metricbeat instances
# File: ansible/inventory/group_vars/debops_all_hosts/metricbeat.yml
metricbeat__configuration:
  - name: 'output_elasticsearch'
    config:
      output.elasticsearch:
        hosts:
          - 'https://es1.example.org:9200'
          - 'https://es2.example.org:9200'
          - 'https://es3.example.org:9200'
        username: 'remote_monitoring_user'
        password: '${REMOTE_USER_PASSWORD}'
  - name: 'setup_kibana'
    config:
      setup.kibana:
        host: "https://kibana.example.org:443"
        # Elasticsearch account which can create Kibana dashboards
        username: 'elastic'
        password: '{{ lookup("file", secret + "/elasticsearch/credentials/"
                      + "built-in/elastic/password") }}'
  - name: 'setup_ilm'
    config:
      setup.ilm.overwrite: true
  - name: 'monitoring'
    config:
      monitoring:
        enabled: true
        elasticsearch:
          username: 'remote_monitoring_user'
          password: '${REMOTE_USER_PASSWORD}'
# Store the Metricbeat user password in Metricbeat keystore. The Elasticsearch
# cluster should be deployed first to ensure that the passwords are present.
metricbeat__keys:
  - RMOTE_USER_PASSWORD: '{{ lookup("file", secret + "/elasticsearch/credentials/"
                             + "built-in/remote_monitoring_user/password") }}'
---
# Configuration for Elasticsearch nodes
# File: ansible/inventory/group_vars/debops_service_elasticsearch/metricbeat.yml
metricbeat__group_snippets:
  - name: 'modules.d/elasticsearch-xpack.yml'
    config:
      - module: 'elasticsearch'
        hosts: [ 'https://{{ ansible_fqdn }}:9200' ]
        username: 'remote_monitoring_user'
        password: '${REMOTE_USER_PASSWORD}'
        xpack.enabled: True
        period: '10s'
---
# Configuration for Kibana nodes
# File: ansible/inventory/group_vars/debops_service_kibana/metricbeat.yml
metricbeat__group_snippets:
  - name: 'modules.d/kibana-xpack.yml'
    config:
      - module: 'kibana'
        hosts: [ 'localhost:5601' ]
        username: 'remote_monitoring_user'
        password: '${REMOTE_USER_PASSWORD}'
        xpack.enabled: True
        period: '10s'

For ease of use, the examples are provided in the debops.metricbeat role documentation in the DebOps monorepo. Elasticsearch and Kibana services can be configured using the debops.elasticsearch and debops.kibana roles, respectively.

Check the Metricbeat configuration <https://www.elastic.co/guide/en/beats/metricbeat/current/configuring-howto-metricbeat.html> upstream documentation to learn more about configuring Metricbeat for your environment and requirements.

  Role debugging tips

Most of the configuration files generated by the role are protected by the Ansible no_log keyword. To make debugging easier, you can use the debops__no_log variable in an inventory (in the development environment) or with Ansible --extra-vars parameter (one time, for production environment) to disable log protection. See the variable documentation for more details.

  Example inventory

To install and configure Metricbeat on a host, it needs to be included in the [debops_service_metricbeat] Ansible inventory group:

[debops_all_hosts]
hostname
[debops_service_metricbeat]
hostname

Fully-fledged Elastic stack can be deployed like this:

[debops_all_hosts]
es1     ansible_host=es1.example.org
es2     ansible_host=es2.example.org
es3     ansible_host=es3.example.org
kibana  ansible_host=kibana.example.org
[debops_service_elasticsearch]
es1
es2
es3
[debops_service_kibana]
kibana
[debops_service_metricbeat:children]
debops_service_elasticsearch
debops_service_kibana

  Example playbook

If you are using this role without DebOps, here's an example Ansible playbook that uses the debops.metricbeat role:

---
- name: Manage Metricbeat service
  collections: [ 'debops.debops', 'debops.roles01',
                 'debops.roles02', 'debops.roles03' ]
  hosts: [ 'debops_service_metricbeat' ]
  become: True
  environment: '{{ inventory__environment | d({})
                   | combine(inventory__group_environment | d({}))
                   | combine(inventory__host_environment  | d({})) }}'
  roles:
    - role: extrepo
      tags: [ 'role::extrepo', 'skip::extrepo', 'role::metricbeat' ]
      extrepo__dependent_sources:
        - '{{ metricbeat__extrepo__dependent_sources }}'
    - role: metricbeat
      tags: [ 'role::metricbeat', 'skip::metricbeat' ]

  Ansible tags

You can use Ansible --tags or --skip-tags parameters to limit what tasks are performed during Ansible run. This can be used after host is first configured to speed up playbook execution, when you are sure that most of the configuration has not been changed.

Available role tags:

role::metricbeat
    Main role tag, should be used in the playbook to execute all of the role tasks as well as role dependencies.

DEFAULT VARIABLE DETAILS

Some of the debops.metricbeat default variables have more extensive configuration than simple strings or lists, here you can find documentation and examples for them.

  metricbeat__configuration

The metricbeat__*_configuration variables define the contents of the /etc/metricbeat/metricbeat.yml configuration file. Each variable contains a list of YAML dictionaries; each dictionary defines a part of the configuration which gets merged together during Ansible execution.

You can read the Metricbeat configuration documentation <https://www.elastic.co/guide/en/beats/metricbeat/current/configuring-howto-metricbeat.html> to learn more about configuring Metricbeat itself.

  Examples

Configure Metricbeat to output its data to Elasticsearch on another host:

metricbeat__configuration:
  - name: 'output_elasticsearch'
    config:
      output.elasticsearch:
        hosts:
          - 'elasticsearch.example.org:9200'

Configure Elasticsearch output, but over an encrypted connection (requires X-Pack support) using certificates managed by the debops.pki role. The access to the cluster is protected by a password, stored in the Metricbeat keystore:

metricbeat__configuration:
  - name: 'output_elasticsearch'
    config:
      output.elasticsearch:
        hosts:
          - 'https://elasticsearch.example.org:9200'
        ssl:
          certificate_authorities: '/etc/pki/realms/domain/CA.crt'
          certificate: '/etc/pki/realms/domain/default.crt'
          key: '/etc/pki/realms/domain/default.key'
        password: '${ELASTIC_PASSWORD}'

The metricbeat__original_configuration variable contains the configuration that comes with the metricbeat APT package re-implemented for consumption by the role. The metricbeat__default_configuration variable contains some additional configuration enabled by default.

  Syntax

Each configuration entry is a YAML dictionary with specific parameters:

name
    Required. An identifier for a particular configuration entry, not used otherwise. The configuration entries with the same name parameter override each other.

config
    Required. A dictionary which holds the Metricbeat configuration written in YAML. The config values from different configuration entries are merged recursively using the combine Ansible filter into a final YAML document.

    YAML keys can be specified in a tree-like structure:

    output:
      elasticsearch:
        hosts:
          - 'elasticsearch.example.org:9200'

    Or, they can be defined on a single line, separated by dots:

    output.elasticsearch.hosts: [ 'elasticsearch.example.org:9200' ]

    The combine Ansible filter does not automatically expand the dot-notation to a tree-like structure. Therefore it's important to use the same style thruought the configuration, otherwise the final YAML document will have duplicate entries.

state
    Optional. If not specified or present, the configuration will be included in the generated /etc/metricbeat/metricbeat.yml configuration file. if absent, the configuration will not be included in the final file. If ignore, the entry will not be evaluated by Ansible during execution.

  metricbeat__snippets

The metricbeat__*_snippets variables define the placement and contents of various *.yml files under the /etc/metricbeat/ directory. The files can include Metricbeat configuration in YAML format.

  Examples

Configure Metricbeat to gather the nginx metrics and send them to Elasticsearch:

metricbeat__snippets:
  - name: 'modules.d/nginx.yml'
    config:
      - module: 'nginx'
        metricsets:
          - 'stubstatus'
        period: '10s'
        hosts: [ 'http://127.0.0.1' ]
        server_status_path: 'nginx_status'

You can find more example configurations in the metricbeat__default_snippets variable.

  Syntax

Each configuration entry is a YAML dictionary with specific parameters:

name
    Required. Path of the configuration file, relative to the /etc/metricbeat/ directory, with all needed subdirectories. The name parameter is also used as an identifier, entries with the same name parameter override each other in order of appearance.

    Metricbeat includes by default all files in the modules.d/*.yml path. Don't use the metricbeat.yml as the filename, otherwise you will override the main configuration file.

config
    Required. A dictionary which holds the Metricbeat configuration written in YAML. The value can either be a dictionary or a list of dictionaries, the result in the generated file will always be a list.

state
    Optional. If not specified or present, the configuration file will be generated. If absent, the configuration file will not be generated, and an existing file will be removed. If ignore, the entry will not be evaluated by Ansible during execution.

comment
    Optional. Comment to be included at the top of the generated file.

mode
    Optional. Specify the filesystem permissions of the generated file. If not specified, 0600 will be used by default.

  metricbeat__keys

The metricbeat__*_keys variables define the contents of the Metricbeat keystore <https://www.elastic.co/guide/en/beats/metricbeat/current/keystore.html> used to keep confidential data like passwords or access tokens. The keys can be referenced in the Metricbeat configuration files using the ${secret_key} syntax.

  Examples

Add an Elasticsearch password used for access over a secure connection. The password is retrieved from the secret/ directory on the Ansible Controller, managed by the debops.secret Ansible role:

metricbeat__keys:
  - ELASTIC_PASSWORD: '{{ lookup("file", secret + "/elastic-stack/elastic/password") }}'

Update an existing key with new content (presence of the force parameter will update the key on each Ansible run):

metricbeat__keys:
  - name: 'ELASTIC_PASSWORD'
    value: 'new-elasticsearch-password'
    force: True

Remove a key from the Metricbeat keystore:

metricbeat__keys:
  - name: 'ELASTIC_PASSWORD'
    state: 'absent'

  Syntax

Each key entry is defined by a YAML dictionary. The keys can be defined using a simple format, with dictionary key being the secret key name, and its value being the secret value. In this case you should avoid the name or value as the secret keys.

Alternatively, secret keys can be defined using YAML dictionaries with specific parameters:

name
    Required. Name of the secret key to store in the Metricbeat keystore.

value
    Optional. A string with the value which should be stored under a given key.

state
    Optional. If not specified or present, the key will be inserted into the keystore. If absent, the key will be removed from the keystore.

force
    Optional, boolean. If present and True, the specified key will be updated in the keystore.

AUTHOR
Maciej Delmanowski

COPYRIGHT
2014-2024, Maciej Delmanowski, Nick Janetakis, Robin Schneider and others