mod_pubsub_alertmanager/README.md
author Matthew Wild <mwild1@gmail.com>
Tue, 16 Apr 2024 15:01:40 +0100
changeset 5892 432efc39572f
parent 5489 67190744b1eb
permissions -rw-r--r--
mod_http_admin_api: Return roles for existing invites, if any
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
4626
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
     1
---
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
     2
labels:
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
     3
- 'Stage-Alpha'
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
     4
summary: Alertmanager webhook receiver for pubsub
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
     5
---
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
     6
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
     7
# Introduction
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
     8
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
     9
This module lets
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    10
[Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/)
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    11
publish alerts to [pubsub][doc:pubsub] via
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    12
[webhooks](https://prometheus.io/docs/alerting/latest/configuration/#webhook_config).
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    13
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    14
# Setup
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    15
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    16
The relevant pubsub nodes must be created and configured somehow.
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    17
Because the request IP address is used to publish, the `publisher`
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    18
affiliation should be given to the IP address Alertmanager sends
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    19
webhooks from.
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    20
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    21
# Configuration
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    22
4627
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    23
## Prometheus
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    24
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    25
A Prometheus `rule_files` might contain something along these lines:
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    26
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    27
``` yaml
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    28
groups:
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    29
- name: Stuff
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    30
  rules:
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    31
  - alert: Down
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    32
    expr: up == 0
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    33
    for: 5m
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    34
    annotations:
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    35
      title: 'Stuff is down!'
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    36
    labels:
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    37
      severity: 'critical'
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    38
```
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    39
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    40
## Alertmanager
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    41
On the Alertmanager site the webhook configuration may look something
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    42
like this:
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    43
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    44
``` yaml
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    45
receivers:
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    46
- name: pubsub
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    47
  webhook_configs:
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    48
  - url: http://pubsub.localhost:5280/pubsub_alertmanager
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    49
```
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    50
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    51
And then finally some Alertmanager routes would point at that receiver:
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    52
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    53
``` yaml
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    54
route:
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    55
  receiver: pubsub
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    56
```
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    57
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    58
## Prosody
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    59
4626
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    60
On the Prosody side, apart from creating and configuring the node(s)
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    61
that will be used, configure your pubsub service like this:
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    62
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    63
``` lua
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    64
Component "pubsub.example.com" "pubsub"
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    65
modules_enabled = {
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    66
    "pubsub_alertmanager",
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    67
}
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    68
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    69
-- optional extra settings:
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    70
alertmanager_body_template = [[
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    71
*ALARM!* {annotations.title?Alert} is {status}
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    72
Since {startsAt}{endsAt& until {endsAt}}
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    73
Labels: {labels%
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    74
  {idx}: {item}}
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    75
Annotations: {annotations%
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    76
  {idx}: {item}}
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    77
]]
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    78
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    79
alertmanager_node_template = "alerts/{alert.labels.severity}"
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    80
```
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    81
4628
eba7e68120d8 mod_pubsub_alertmanager: Add some words about the node template
Kim Alvefur <zash@zash.se>
parents: 4627
diff changeset
    82
If no node template is given, either an optional part after
eba7e68120d8 mod_pubsub_alertmanager: Add some words about the node template
Kim Alvefur <zash@zash.se>
parents: 4627
diff changeset
    83
"pubsub_alertmanager" in the HTTP path is used as node, or the string
eba7e68120d8 mod_pubsub_alertmanager: Add some words about the node template
Kim Alvefur <zash@zash.se>
parents: 4627
diff changeset
    84
"alerts". Here, an alerts would be published to different nodes based on
eba7e68120d8 mod_pubsub_alertmanager: Add some words about the node template
Kim Alvefur <zash@zash.se>
parents: 4627
diff changeset
    85
the 'severity' label, so e.g. `alerts/critical` in this example.
eba7e68120d8 mod_pubsub_alertmanager: Add some words about the node template
Kim Alvefur <zash@zash.se>
parents: 4627
diff changeset
    86
4627
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    87
## All Options
622c6308d7af mod_pubsub_alertmanager: Minimal example of Prometheus and Alertmanager config
Kim Alvefur <zash@zash.se>
parents: 4626
diff changeset
    88
4626
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    89
Available configuration options:
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    90
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    91
`alertmanager_body_template`
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    92
:   Template for the textual representation of alerts.
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    93
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    94
`alertmanager_node_template`
61ce3394fe8b mod_pubsub_alertmanager: Add a README with initial docs
Kim Alvefur <zash@zash.se>
parents:
diff changeset
    95
:   Template for the pubsub node name, defaults to `"{path?alerts}"`
5489
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
    96
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
    97
`alertmanager_path_configs`
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
    98
:   Per-path configuration variables (see below).
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
    99
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   100
### Per-path configuration
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   101
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   102
It's possible to override configuration options based on the path suffix. For
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   103
example, if a request is made to `http://prosody/pubsub_alertmanager/foo` the
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   104
path suffix is `foo`. You can then supply the following configuration:
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   105
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   106
``` lua
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   107
alertmanager_path_configs = {
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   108
    foo = {
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   109
        node_template = "alerts/{alert.labels.severity}";
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   110
        publisher = "user@example.net";
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   111
    };
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   112
}
67190744b1eb mod_pubsub_alertmanager: Support for per-path config overrides
Matthew Wild <mwild1@gmail.com>
parents: 4628
diff changeset
   113
```