配置

Alertmanager 通过命令行标志和配置文件进行配置。命令行标志配置不可变的系统参数,配置文件定义抑制规则、通知路由和通知接收器。

可视化编辑器 可以帮助构建路由树。

要查看所有可用的命令行标志,请运行 alertmanager -h

Alertmanager 可以在运行时重新加载其配置。如果新的配置格式不正确,则不会应用更改,并且会记录错误。通过向进程发送 SIGHUP 信号或向 /-/reload 端点发送 HTTP POST 请求来触发配置重新加载。

配置文件简介

要指定要加载哪个配置文件,请使用 --config.file 标志。

./alertmanager --config.file=alertmanager.yml

该文件使用 YAML 格式 编写,其方案如下所述。方括号表示参数是可选的。对于非列表参数,值设置为指定的默认值。

通用占位符定义如下

  • <duration>:与正则表达式 ((([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?|0) 匹配的持续时间,例如 1d1h30m5m10s
  • <labelname>:与正则表达式 [a-zA-Z_][a-zA-Z0-9_]* 匹配的字符串
  • <labelvalue>:Unicode 字符串
  • <filepath>:当前工作目录中的有效路径
  • <boolean>:可以取值 truefalse 的布尔值
  • <string>:常规字符串
  • <secret>:表示秘密的常规字符串,例如密码
  • <tmpl_string>:在使用前进行模板扩展的字符串
  • <tmpl_secret>:在使用前进行模板扩展的字符串,表示秘密
  • <int>:整数值
  • <regex>:任何有效的 RE2 正则表达式(正则表达式在两端都已锚定。要取消锚定正则表达式,请使用 .*<regex>.*。)

其他占位符将单独指定。

提供的 有效示例文件 显示了在上下文中如何使用。

文件布局和全局设置

全局配置指定在所有其他配置上下文中有效的参数。它们也充当其他配置部分的默认值。其他顶级部分将在本页下方进行说明。

global:
  # The default SMTP From header field.
  [ smtp_from: <tmpl_string> ]
  # The default SMTP smarthost used for sending emails, including port number.
  # Port number usually is 25, or 587 for SMTP over TLS (sometimes referred to as STARTTLS).
  # Example: smtp.example.org:587
  [ smtp_smarthost: <string> ]
  # The default hostname to identify to the SMTP server.
  [ smtp_hello: <string> | default = "localhost" ]
  # SMTP Auth using CRAM-MD5, LOGIN and PLAIN. If empty, Alertmanager doesn't authenticate to the SMTP server.
  [ smtp_auth_username: <string> ]
  # SMTP Auth using LOGIN and PLAIN.
  [ smtp_auth_password: <secret> ]
  # SMTP Auth using LOGIN and PLAIN.
  [ smtp_auth_password_file: <string> ]
  # SMTP Auth using PLAIN.
  [ smtp_auth_identity: <string> ]
  # SMTP Auth using CRAM-MD5.
  [ smtp_auth_secret: <secret> ]
  # The default SMTP TLS requirement.
  # Note that Go does not support unencrypted connections to remote SMTP endpoints.
  [ smtp_require_tls: <bool> | default = true ]

  # The API URL to use for Slack notifications.
  [ slack_api_url: <secret> ]
  [ slack_api_url_file: <filepath> ]
  [ victorops_api_key: <secret> ]
  [ victorops_api_key_file: <filepath> ]
  [ victorops_api_url: <string> | default = "https://alert.victorops.com/integrations/generic/20131114/alert/" ]
  [ pagerduty_url: <string> | default = "https://events.pagerduty.com/v2/enqueue" ]
  [ opsgenie_api_key: <secret> ]
  [ opsgenie_api_key_file: <filepath> ]
  [ opsgenie_api_url: <string> | default = "https://api.opsgenie.com/" ]
  [ wechat_api_url: <string> | default = "https://qyapi.weixin.qq.com/cgi-bin/" ]
  [ wechat_api_secret: <secret> ]
  [ wechat_api_corp_id: <string> ]
  [ telegram_api_url: <string> | default = "https://api.telegram.org" ]
  [ webex_api_url: <string> | default = "https://webexapis.com/v1/messages" ]
  # The default HTTP client configuration
  [ http_config: <http_config> ]

  # ResolveTimeout is the default value used by alertmanager if the alert does
  # not include EndsAt, after this time passes it can declare the alert as resolved if it has not been updated.
  # This has no impact on alerts from Prometheus, as they always include EndsAt.
  [ resolve_timeout: <duration> | default = 5m ]

# Files from which custom notification template definitions are read.
# The last component may use a wildcard matcher, e.g. 'templates/*.tmpl'.
templates:
  [ - <filepath> ... ]

# The root node of the routing tree.
route: <route>

# A list of notification receivers.
receivers:
  - <receiver> ...

# A list of inhibition rules.
inhibit_rules:
  [ - <inhibit_rule> ... ]

# DEPRECATED: use time_intervals below.
# A list of mute time intervals for muting routes.
mute_time_intervals:
  [ - <mute_time_interval> ... ]

# A list of time intervals for muting/activating routes.
time_intervals:
  [ - <time_interval> ... ]

路由相关设置允许配置如何根据时间路由、聚合、限制和静默告警。

<route>

路由块定义路由树中的一个节点及其子节点。如果未设置,则其可选配置参数将继承自其父节点。

每个告警都从配置的顶级路由进入路由树,该路由必须匹配所有告警(即没有任何配置的匹配器)。然后它遍历子节点。如果将 continue 设置为 false,则在第一个匹配的子节点后停止。如果在匹配节点上将 continue 设置为 true,则告警将继续与后续同级节点匹配。如果告警与节点的任何子节点都不匹配(没有匹配的子节点或不存在),则根据当前节点的配置参数处理告警。

请参阅 Alertmanager 概念 以了解有关分组的更多信息。

[ receiver: <string> ]
# The labels by which incoming alerts are grouped together. For example,
# multiple alerts coming in for cluster=A and alertname=LatencyHigh would
# be batched into a single group.
#
# To aggregate by all possible labels use the special value '...' as the sole label name, for example:
# group_by: ['...']
# This effectively disables aggregation entirely, passing through all
# alerts as-is. This is unlikely to be what you want, unless you have
# a very low alert volume or your upstream notification system performs
# its own grouping.
[ group_by: '[' <labelname>, ... ']' ]

# Whether an alert should continue matching subsequent sibling nodes.
[ continue: <boolean> | default = false ]

# DEPRECATED: Use matchers below.
# A set of equality matchers an alert has to fulfill to match the node.
match:
  [ <labelname>: <labelvalue>, ... ]

# DEPRECATED: Use matchers below.
# A set of regex-matchers an alert has to fulfill to match the node.
match_re:
  [ <labelname>: <regex>, ... ]

# A list of matchers that an alert has to fulfill to match the node.
matchers:
  [ - <matcher> ... ]

# How long to initially wait to send a notification for a group
# of alerts. Allows to wait for an inhibiting alert to arrive or collect
# more initial alerts for the same group. (Usually ~0s to few minutes.)
# If omitted, child routes inherit the group_wait of the parent route.
[ group_wait: <duration> | default = 30s ]

# How long to wait before sending a notification about new alerts that
# are added to a group of alerts for which an initial notification has
# already been sent. (Usually ~5m or more.) If omitted, child routes
# inherit the group_interval of the parent route.
[ group_interval: <duration> | default = 5m ]

# How long to wait before sending a notification again if it has already
# been sent successfully for an alert. (Usually ~3h or more). If omitted,
# child routes inherit the repeat_interval of the parent route.
# Note that this parameter is implicitly bound by Alertmanager's
# `--data.retention` configuration flag. Notifications will be resent after either
# repeat_interval or the data retention period have passed, whichever
# occurs first. `repeat_interval` should be a multiple of `group_interval`.
[ repeat_interval: <duration> | default = 4h ]

# Times when the route should be muted. These must match the name of a
# mute time interval defined in the mute_time_intervals section.
# Additionally, the root node cannot have any mute times.
# When a route is muted it will not send any notifications, but
# otherwise acts normally (including ending the route-matching process
# if the `continue` option is not set.)
mute_time_intervals:
  [ - <string> ...]

# Times when the route should be active. These must match the name of a
# time interval defined in the time_intervals section. An empty value
# means that the route is always active.
# Additionally, the root node cannot have any active times.
# The route will send notifications only when active, but otherwise
# acts normally (including ending the route-matching process
# if the `continue` option is not set).
active_time_intervals:
  [ - <string> ...]

# Zero or more child routes.
routes:
  [ - <route> ... ]

示例

# The root route with all parameters, which are inherited by the child
# routes if they are not overwritten.
route:
  receiver: 'default-receiver'
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h
  group_by: [cluster, alertname]
  # All alerts that do not match the following child routes
  # will remain at the root node and be dispatched to 'default-receiver'.
  routes:
  # All alerts with service=mysql or service=cassandra
  # are dispatched to the database pager.
  - receiver: 'database-pager'
    group_wait: 10s
    matchers:
    - service=~"mysql|cassandra"
  # All alerts with the team=frontend label match this sub-route.
  # They are grouped by product and environment rather than cluster
  # and alertname.
  - receiver: 'frontend-pager'
    group_by: [product, environment]
    matchers:
    - team="frontend"

  # All alerts with the service=inhouse-service label match this sub-route.
  # the route will be muted during offhours and holidays time intervals.
  # even if it matches, it will continue to the next sub-route
  - receiver: 'dev-pager'
    matchers:
      - service="inhouse-service"
    mute_time_intervals:
      - offhours
      - holidays
    continue: true

    # All alerts with the service=inhouse-service label match this sub-route
    # the route will be active only during offhours and holidays time intervals.
  - receiver: 'on-call-pager'
    matchers:
      - service="inhouse-service"
    active_time_intervals:
      - offhours
      - holidays

<time_interval>

time_interval 指定一个命名的时间间隔,可以在路由树中引用该间隔以在一天中的特定时间静默/激活特定路由。

name: <string>
time_intervals:
  [ - <time_interval_spec> ... ]

<time_interval_spec>

time_interval_spec 包含时间间隔的实际定义。语法支持以下字段

- times:
  [ - <time_range> ...]
  weekdays:
  [ - <weekday_range> ...]
  days_of_month:
  [ - <days_of_month_range> ...]
  months:
  [ - <month_range> ...]
  years:
  [ - <year_range> ...]
  location: <string>

所有字段都是列表。在每个非空列表中,至少必须满足一个元素才能匹配该字段。如果未指定字段,则任何值都将匹配该字段。为了使时间点与完整的时间间隔匹配,所有字段都必须匹配。某些字段支持范围和负索引,并在下面详细介绍。如果未指定时区,则时间被视为 UTC 时间。要使时间点与完整的时间间隔匹配,所有字段都必须匹配。一些字段支持范围和负索引,并在下面详细介绍。如果未指定时区,则时间被视为 UTC 时间。

time_range:包含开始时间并排除结束时间的范围,以便于表示在小时边界开始/结束的时间。例如,start_time: '17:00'end_time: '24:00' 将在 17:00 开始,并在 24:00 之前立即结束。它们指定如下

    times:
    - start_time: HH:MM
      end_time: HH:MM

weekday_range:一周中的日期列表,其中一周从星期日开始,到星期六结束。日期应按名称指定(例如“星期日”)。为方便起见,还接受 <start_day>:<end_day> 格式的范围,并且两端都包含在内。例如:['monday:wednesday','saturday', 'sunday']

days_of_month_range:一个月中的数字日期列表。日期从 1 开始。也接受负值,从月底开始,例如 1 月份的 -1 表示 1 月 31 日。例如:['1:5', '-3:-1']。扩展到月份的开始或结束之外将导致其被钳位。例如,在 2 月份指定 ['1:31'] 将根据闰年将实际结束日期钳位到 28 或 29。两端都包含在内。

month_range:由不区分大小写的名称(例如“一月”)或数字标识的日历月份列表,其中一月 = 1。也接受范围。例如,['1:3', 'may:august', 'december']。两端都包含在内。

year_range:年份的数字列表。接受范围。例如,['2020:2022', '2030']。两端都包含在内。

location:与 IANA 时区数据库中的位置匹配的字符串。例如,'Australia/Sydney'。该位置提供时间间隔的时区。例如,位置为 'Australia/Sydney' 的时间间隔包含以下内容

    times:
    - start_time: 09:00
      end_time: 17:00
    weekdays: ['monday:friday']

将包括在澳大利亚悉尼的当地时间上午 9:00 到下午 5:00 之间,在周一到周五之间的任何时间。

您还可以使用 'Local' 作为位置来使用 Alertmanager 运行的机器的本地时间,或使用 'UTC' 表示 UTC 时间。如果未提供时区,则时间间隔被视为 UTC 时间。**注意:**在 Windows 上,除非您使用 ZONEINFO 环境变量提供自定义时区数据库,否则仅支持 LocalUTC

抑制允许根据另一个告警集的存在来静默一组告警。这允许在系统或服务之间建立依赖关系,以便在发生中断时仅发送一组互连告警中最相关的告警。

请参阅 Alertmanager 概念 以了解有关抑制的更多信息。

<inhibit_rule>

当存在与另一组匹配器匹配的告警(源)时,抑制规则会静默与一组匹配器匹配的告警(目标)。目标和源告警必须对 equal 列表中的标签名称具有相同的标签值。

从语义上讲,缺少的标签和具有空值的标签是相同的。因此,如果 equal 中列出的所有标签名称都从源告警和目标告警中丢失,则抑制规则将适用。

为了防止告警抑制自身,与规则的目标端和源端都匹配的告警不能被满足相同条件的告警抑制(包括自身)。但是,我们建议以一种告警永远不会同时匹配目标端和源端的方式选择目标和源匹配器。这样更容易理解,并且不会触发此特殊情况。

# DEPRECATED: Use target_matchers below.
# Matchers that have to be fulfilled in the alerts to be muted.
target_match:
  [ <labelname>: <labelvalue>, ... ]
# DEPRECATED: Use target_matchers below.
target_match_re:
  [ <labelname>: <regex>, ... ]

# A list of matchers that have to be fulfilled by the target
# alerts to be muted.
target_matchers:
  [ - <matcher> ... ]

# DEPRECATED: Use source_matchers below.
# Matchers for which one or more alerts have to exist for the
# inhibition to take effect.
source_match:
  [ <labelname>: <labelvalue>, ... ]
# DEPRECATED: Use source_matchers below.
source_match_re:
  [ <labelname>: <regex>, ... ]

# A list of matchers for which one or more alerts have
# to exist for the inhibition to take effect.
source_matchers:
  [ - <matcher> ... ]

# Labels that must have an equal value in the source and target
# alert for the inhibition to take effect.
[ equal: '[' <labelname>, ... ']' ]

标签匹配器

标签匹配器将告警与路由、静默和抑制规则匹配。

重要:Prometheus 正在为标签和指标添加对 UTF-8 的支持。为了在 Alertmanager 中也支持 UTF-8,Alertmanager 0.27 及更高版本对匹配器使用了新的解析器,该解析器有一些向后不兼容的更改。虽然大多数匹配器将向前兼容,但有些则不会。Alertmanager 正在运行一个过渡期,在此期间它同时支持 UTF-8 和经典匹配器,并提供了一些工具来帮助您为过渡做好准备。

如果这是一个新的 Alertmanager 安装,我们建议在创建 Alertmanager 配置文件之前启用 UTF-8 严格模式。您可以在此处找到有关如何启用 UTF-8 严格模式的说明。

如果这是一个现有的 Alertmanager 安装,我们建议在启用 UTF-8 严格模式之前,以默认模式(称为回退模式)运行 Alertmanager。在此模式下,如果您需要在启用 UTF-8 严格模式之前对配置文件进行任何更改,Alertmanager 将记录警告。Alertmanager 将在接下来的两个版本中将 UTF-8 严格模式设为默认模式,因此尽快过渡非常重要。

无论 Alertmanager 安装是新安装还是现有安装,您还可以使用 amtool 验证 Alertmanager 配置文件是否与 UTF-8 严格模式兼容,然后再在 Alertmanager 服务器中启用它。您无需运行 Alertmanager 服务器即可执行此操作。您可以在此处找到有关如何使用 amtool 验证 Alertmanager 配置文件的说明。

Alertmanager 服务器操作模式

在过渡期间,Alertmanager 支持三种操作模式。这些被称为回退模式、UTF-8 严格模式和经典模式。回退模式是默认模式。

Alertmanager 服务器的操作人员应在过渡期结束前过渡到 UTF-8 严格模式。Alertmanager 将在接下来的两个版本中将 UTF-8 严格模式设为默认模式,因此尽快过渡非常重要。

回退模式

Alertmanager 以称为回退模式的特殊模式作为其默认模式运行。作为操作员,您不应该体验到路由、静默或抑制规则的工作方式有任何差异。

在回退模式下,配置首先被解析为 UTF-8 匹配器,如果与 UTF-8 解析器不兼容,则被解析为经典匹配器。如果您的 Alertmanager 配置包含与 UTF-8 解析器不兼容的匹配器,Alertmanager 将将其解析为经典匹配器并记录警告。此警告还包括有关如何将匹配器从经典匹配器更改为 UTF-8 匹配器的建议。例如

ts=2024-02-11T10:00:00Z caller=parse.go:176 level=warn msg="Alertmanager is moving to a new parser for labels and matchers, and this input is incompatible. Alertmanager has instead parsed the input using the classic matchers parser as a fallback. To make this input compatible with the UTF-8 matchers parser please make sure all regular expressions and values are double-quoted. If you are still seeing this message please open an issue." input="foo=" origin=config err="end of input: expected label value" suggestion="foo=\"\""

此处,匹配器 foo= 可以通过双引号括起表达式的右侧使其成为有效的 UTF-8 匹配器,得到 foo=""。这两个匹配器是等效的,但是对于 UTF-8 匹配器,匹配器的右侧是一个必填字段。

在极少数情况下,配置可能会导致 UTF-8 和经典解析器之间出现不一致。当匹配器在两个解析器中都有效,但由于增加了对 UTF-8 的支持,导致根据使用哪个解析器而产生不同的解析结果时,就会发生这种情况。如果您的 Alertmanager 配置存在不一致,Alertmanager 将使用经典解析器并记录警告。例如

ts=2024-02-11T10:00:00Z caller=parse.go:183 level=warn msg="Matchers input has disagreement" input="qux=\"\xf0\x9f\x99\x82\"\n" origin=config

应逐案查看任何不一致的情况,因为根据不一致的性质,配置可能不需要在启用 UTF-8 严格模式之前进行更新。例如,\xf0\x9f\x99\x82 是 🙂 表情符号的字节序列。如果目的是匹配字面上的 🙂 表情符号,则无需更改。但是,如果目的是匹配字面上的 \xf0\x9f\x99\x82,则应将匹配器更改为 qux="\\xf0\\x9f\\x99\\x82"

UTF-8 严格模式

在 UTF-8 严格模式下,Alertmanager 禁用对经典匹配器的支持。

alertmanager --config.file=config.yml --enable-feature="utf8-strict-mode"

对于新的 Alertmanager 安装,以及在解决所有不兼容匹配器的警告后,现有的 Alertmanager 安装应启用此模式。在解决所有不兼容匹配器的警告之前,Alertmanager 不会以 UTF-8 严格模式启动。

ts=2024-02-11T10:00:00Z caller=coordinator.go:118 level=error component=configuration msg="Loading configuration file failed" file=config.yml err="end of input: expected label value"

在过渡期结束时,UTF-8 严格模式将成为 Alertmanager 的默认模式。

经典模式

经典模式等效于 Alertmanager 0.26.0 及更早版本。

alertmanager --config.file=config.yml --enable-feature="classic-mode"

如果您怀疑回退模式或 UTF-8 严格模式存在问题,可以使用此模式。在这种情况下,请在 GitHub 上提交问题,并提供尽可能多的信息。

验证

您可以在 Alertmanager 服务器中启用 UTF-8 严格模式之前,使用 amtool 验证 Alertmanager 配置文件是否与 UTF-8 严格模式兼容。您无需运行 Alertmanager 服务器即可执行此操作。

就像 Alertmanager 服务器一样,如果配置不兼容或存在不一致,amtool 将记录警告。

amtool check-config config.yml
Checking 'config.yml'
level=warn msg="Alertmanager is moving to a new parser for labels and matchers, and this input is incompatible. Alertmanager has instead parsed the input using the classic matchers parser as a fallback. To make this input compatible with the UTF-8 matchers parser please make sure all regular expressions and values are double-quoted. If you are still seeing this message please open an issue." input="foo=" origin=config err="end of input: expected label value" suggestion="foo=\"\""
level=warn msg="Matchers input has disagreement" input="qux=\"\\xf0\\x9f\\x99\\x82\"\n" origin=config
  SUCCESS
Found:
 - global config
 - route
 - 2 inhibit rules
 - 2 receivers
 - 0 templates

amtool 中没有记录警告时,您就知道配置与 UTF-8 严格模式兼容。

amtool check-config config.yml
Checking 'config.yml'  SUCCESS
Found:
 - global config
 - route
 - 2 inhibit rules
 - 2 receivers
 - 0 templates

您还可以以 UTF-8 严格模式使用 amtool 作为额外的验证级别。您将知道配置无效,因为命令将失败。

amtool check-config config.yml --enable-feature="utf8-strict-mode"
level=warn msg="UTF-8 mode enabled"
Checking 'config.yml'  FAILED: end of input: expected label value

amtool: error: failed to validate 1 file(s)

您将知道配置有效,因为命令将成功。

amtool check-config config.yml --enable-feature="utf8-strict-mode"
level=warn msg="UTF-8 mode enabled"
Checking 'config.yml'  SUCCESS
Found:
 - global config
 - route
 - 2 inhibit rules
 - 2 receivers
 - 0 templates

<matcher>

UTF-8 匹配器

UTF-8 匹配器由三个标记组成。

  • 标签名称的未加引号的文字或双引号字符串。
  • =!==~!~ 之一。= 表示等于,!= 表示不等于,=~ 表示匹配正则表达式,!~ 表示不匹配正则表达式。
  • 正则表达式或标签值的未加引号的文字或双引号字符串。

未加引号的文字可以包含除保留字符之外的所有 UTF-8 字符。这些字符是空格,以及 { } ! = ~ , \ " ' ` 中的所有字符。例如,foo[a-zA-Z]+Προμηθεύς(希腊语的 Prometheus)都是有效的未加引号的文字。但是,foo! 不是有效的文字,因为 ! 是一个保留字符。

双引号字符串可以包含所有 UTF-8 字符。与未加引号的文字不同,没有保留字符。您甚至可以使用 UTF-8 代码点。例如,"foo!""bar,baz""\"baz qux\"""\xf0\x9f\x99\x82" 都是有效的双引号字符串。

经典匹配器

经典匹配器是一个字符串,其语法灵感来自 PromQL 和 OpenMetrics。经典匹配器的语法由三个标记组成。

  • 有效的 Prometheus 标签名称。
  • =!==~!~ 之一。= 表示等于,!= 表示字符串不相等,=~ 用于正则表达式相等,!~ 用于正则表达式不相等。它们具有与 PromQL 选择器中已知的含义相同的含义。
  • UTF-8 字符串,可以包含在双引号中。在每个标记之前或之后,可以有任意数量的空格。

第 3 个标记可以是空字符串。在第 3 个标记内,应用 OpenMetrics 转义规则:\" 用于双引号,\n 用于换行符,\\ 用于字面反斜杠。未转义的 " 不能出现在第 3 个标记内(只能作为第一个或最后一个字符)。但是,允许使用字面换行符,以及不后跟 \n" 的单个 \ 字符。在这种情况下,它们充当字面反斜杠。

匹配器的组合

您可以组合匹配器以创建复杂的匹配表达式。组合后,所有匹配器都必须匹配才能使整个表达式匹配。例如,表达式 {alertname="Watchdog", severity=~"warning|critical"} 将匹配标签为 alertname=Watchdog, severity=critical 的告警,但不会匹配标签为 alertname=Watchdog, severity=none 的告警,因为虽然告警名称是 Watchdog,但严重性既不是警告也不是严重。

您可以使用 YAML 列表将匹配器组合成表达式

matchers:
  - alertname = Watchdog
  - severity =~ "warning|critical"

或作为 PromQL 风格的表达式,其中每个匹配器用逗号分隔

{alertname="Watchdog", severity=~"warning|critical"}

允许单个尾随逗号

{alertname="Watchdog", severity=~"warning|critical",}

开括号 { 和闭括号 } 是可选的。

alertname="Watchdog", severity=~"warning|critical"

但是,两者必须都存在或都不存在。您不能有未完成的开括号或闭括号。

{alertname="Watchdog", severity=~"warning|critical"
alertname="Watchdog", severity=~"warning|critical"}

您也不能有重复的开括号或闭括号。

{{alertname="Watchdog", severity=~"warning|critical",}}

空格(空格、制表符和换行符)允许在双引号外部使用,并且对匹配器本身没有影响。例如

{
   alertname = "Watchdog",
   severity =~ "warning|critical",
}

等价于

{alertname="Watchdog",severity=~"warning|critical"}

更多示例

以下是一些更多示例。

  1. 两个相等匹配器组合为 YAML 列表

    matchers:
      - foo = bar
      - dings != bums
    
  2. 两个匹配器组合为简短形式的 YAML 列表

    matchers: [ foo = bar, dings != bums ]
    

如下所示,在简短形式中,最好使用双引号来避免逗号等特殊字符带来的问题。

   matchers: [ "foo = \"bar,baz\"", "dings != bums" ]
  1. 您还可以将两个匹配器放入一个类似 PromQL 的字符串中。此处单引号最有效。

    matchers: [ '{foo="bar", dings!="bums"}' ]
    
  2. 为了避免 YAML 中的转义和引用规则带来的问题,您还可以使用 YAML 块。

    matchers:
      - |
          {quote=~"She said: \"Hi, all!( How're you…)?\""}
    

这些接收器设置允许配置通知目标(接收器)和基于 HTTP 的接收器的 HTTP 客户端选项。

<receiver>

接收器是一个或多个通知集成的命名配置。

注意:作为取消过去关于新接收器的暂停的一部分,已达成协议,除了现有要求外,新的通知集成将需要一个具有推送访问权限的已承诺维护者。

# The unique name of the receiver.
name: <string>

# Configurations for several notification integrations.
discord_configs:
  [ - <discord_config>, ... ]
email_configs:
  [ - <email_config>, ... ]
msteams_configs:
  [ - <msteams_config>, ... ]
opsgenie_configs:
  [ - <opsgenie_config>, ... ]
pagerduty_configs:
  [ - <pagerduty_config>, ... ]
pushover_configs:
  [ - <pushover_config>, ... ]
slack_configs:
  [ - <slack_config>, ... ]
sns_configs:
  [ - <sns_config>, ... ]
telegram_configs:
  [ - <telegram_config>, ... ]
victorops_configs:
  [ - <victorops_config>, ... ]
webex_configs:
  [ - <webex_config>, ... ]
webhook_configs:
  [ - <webhook_config>, ... ]
wechat_configs:
  [ - <wechat_config>, ... ]

<http_config>

http_config 允许配置接收器用于与基于 HTTP 的 API 服务通信的 HTTP 客户端。

# Note that `basic_auth` and `authorization` options are mutually exclusive.

# Sets the `Authorization` header with the configured username and password.
# password and password_file are mutually exclusive.
basic_auth:
  [ username: <string> ]
  [ password: <secret> ]
  [ password_file: <string> ]

# Optional the `Authorization` header configuration.
authorization:
  # Sets the authentication type.
  [ type: <string> | default: Bearer ]
  # Sets the credentials. It is mutually exclusive with
  # `credentials_file`.
  [ credentials: <secret> ]
  # Sets the credentials with the credentials read from the configured file.
  # It is mutually exclusive with `credentials`.
  [ credentials_file: <filename> ]

# Optional OAuth 2.0 configuration.
# Cannot be used at the same time as basic_auth or authorization.
oauth2:
  [ <oauth2> ]

# Whether to enable HTTP2.
[ enable_http2: <bool> | default: true ]

# Optional proxy URL.
[ proxy_url: <string> ]
# Comma-separated string that can contain IPs, CIDR notation, domain names
# that should be excluded from proxying. IP and domain names can
# contain port numbers.
[ no_proxy: <string> ]
# Use proxy URL indicated by environment variables (HTTP_PROXY, http_proxy, HTTPS_PROXY, https_proxy, NO_PROXY, and no_proxy)
[ proxy_from_environment: <boolean> | default: false ]
# Specifies headers to send to proxies during CONNECT requests.
[ proxy_connect_header:
  [ <string>: [<secret>, ...] ] ]

# Configure whether HTTP requests follow HTTP 3xx redirects.
[ follow_redirects: <bool> | default = true ]

# Configures the TLS settings.
tls_config:
  [ <tls_config> ]

<oauth2>

使用客户端凭据授予类型的 OAuth 2.0 身份验证。Alertmanager 使用给定的客户端访问和密钥从指定的端点获取访问令牌。

client_id: <string>
[ client_secret: <secret> ]

# Read the client secret from a file.
# It is mutually exclusive with `client_secret`.
[ client_secret_file: <filename> ]

# Scopes for the token request.
scopes:
  [ - <string> ... ]

# The URL to fetch the token from.
token_url: <string>

# Optional parameters to append to the token URL.
endpoint_params:
  [ <string>: <string> ... ]

# Configures the token request's TLS settings.
tls_config:
  [ <tls_config> ]

# Optional proxy URL.
[ proxy_url: <string> ]
# Comma-separated string that can contain IPs, CIDR notation, domain names
# that should be excluded from proxying. IP and domain names can
# contain port numbers.
[ no_proxy: <string> ]
# Use proxy URL indicated by environment variables (HTTP_PROXY, https_proxy, HTTPs_PROXY, https_proxy, and no_proxy)
[ proxy_from_environment: <boolean> | default: false ]
# Specifies headers to send to proxies during CONNECT requests.
[ proxy_connect_header:
  [ <string>: [<secret>, ...] ] ]

<tls_config>

tls_config 允许配置 TLS 连接。

# CA certificate to validate the server certificate with.
[ ca_file: <filepath> ]

# Certificate and key files for client cert authentication to the server.
[ cert_file: <filepath> ]
[ key_file: <filepath> ]

# ServerName extension to indicate the name of the server.
# http://tools.ietf.org/html/rfc4366#section-3.1
[ server_name: <string> ]

# Disable validation of the server certificate.
[ insecure_skip_verify: <boolean> | default = false]

# Minimum acceptable TLS version. Accepted values: TLS10 (TLS 1.0), TLS11 (TLS
# 1.1), TLS12 (TLS 1.2), TLS13 (TLS 1.3).
# If unset, Prometheus will use Go default minimum version, which is TLS 1.2.
# See MinVersion in https://pkg.go.dev/crypto/tls#Config.
[ min_version: <string> ]
# Maximum acceptable TLS version. Accepted values: TLS10 (TLS 1.0), TLS11 (TLS
# 1.1), TLS12 (TLS 1.2), TLS13 (TLS 1.3).
# If unset, Prometheus will use Go default maximum version, which is TLS 1.3.
# See MaxVersion in https://pkg.go.dev/crypto/tls#Config.
[ max_version: <string> ]

接收器集成设置

这些设置允许配置特定的接收器集成。

<discord_config>

Discord 通知通过 Discord Webhook API 发送。请参阅 Discord 的 “Webhook 简介”文章,了解如何为频道配置 Webhook 集成。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The Discord webhook URL.
# webhook_url and webhook_url_file are mutually exclusive.
webhook_url: <secret>
webhook_url_file: <filepath>

# Message title template.
[ title: <tmpl_string> | default = '{{ template "discord.default.title" . }}' ]

# Message body template.
[ message: <tmpl_string> | default = '{{ template "discord.default.message" . }}' ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<email_config>

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]

# The email address to send notifications to.
to: <tmpl_string>

# The sender's address.
[ from: <tmpl_string> | default = global.smtp_from ]

# The SMTP host through which emails are sent.
[ smarthost: <string> | default = global.smtp_smarthost ]

# The hostname to identify to the SMTP server.
[ hello: <string> | default = global.smtp_hello ]

# SMTP authentication information.
# auth_password and auth_password_file are mutually exclusive.
[ auth_username: <string> | default = global.smtp_auth_username ]
[ auth_password: <secret> | default = global.smtp_auth_password ]
[ auth_password_file: <string> | default = global.smtp_auth_password_file ]
[ auth_secret: <secret> | default = global.smtp_auth_secret ]
[ auth_identity: <string> | default = global.smtp_auth_identity ]

# The SMTP TLS requirement.
# Note that Go does not support unencrypted connections to remote SMTP endpoints.
[ require_tls: <bool> | default = global.smtp_require_tls ]

# TLS configuration.
tls_config:
  [ <tls_config> ]

# The HTML body of the email notification.
[ html: <tmpl_string> | default = '{{ template "email.default.html" . }}' ]
# The text body of the email notification.
[ text: <tmpl_string> ]

# Further headers email header key/value pairs. Overrides any headers
# previously set by the notification implementation.
[ headers: { <string>: <tmpl_string>, ... } ]

<msteams_config>

Microsoft Teams 通知通过 传入 Webhook API 端点发送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The incoming webhook URL.
# webhook_url and webhook_url_file are mutually exclusive.
[ webhook_url: <secret> ]
[ webhook_url_file: <filepath> ]

# Message title template.
[ title: <tmpl_string> | default = '{{ template "msteams.default.title" . }}' ]

# Message summary template.
[ summary: <tmpl_string> | default = '{{ template "msteams.default.summary" . }}' ]

# Message body template.
[ text: <tmpl_string> | default = '{{ template "msteams.default.text" . }}' ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<opsgenie_config>

OpsGenie 通知通过 OpsGenie API 发送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The API key to use when talking to the OpsGenie API.
[ api_key: <secret> | default = global.opsgenie_api_key ]

# The filepath to API key to use when talking to the OpsGenie API. Conflicts with api_key.
[ api_key_file: <filepath> | default = global.opsgenie_api_key_file ]

# The host to send OpsGenie API requests to.
[ api_url: <string> | default = global.opsgenie_api_url ]

# Alert text limited to 130 characters.
[ message: <tmpl_string> | default = '{{ template "opsgenie.default.message" . }}' ]

# A description of the alert.
[ description: <tmpl_string> | default = '{{ template "opsgenie.default.description" . }}' ]

# A backlink to the sender of the notification.
[ source: <tmpl_string> | default = '{{ template "opsgenie.default.source" . }}' ]

# A set of arbitrary key/value pairs that provide further detail
# about the alert.
# All common labels are included as details by default.
[ details: { <string>: <tmpl_string>, ... } ]

# List of responders responsible for notifications.
responders:
  [ - <responder> ... ]

# Comma separated list of tags attached to the notifications.
[ tags: <tmpl_string> ]

# Additional alert note.
[ note: <tmpl_string> ]

# Priority level of alert. Possible values are P1, P2, P3, P4, and P5.
[ priority: <tmpl_string> ]

# Whether to update message and description of the alert in OpsGenie if it already exists
# By default, the alert is never updated in OpsGenie, the new message only appears in activity log.
[ update_alerts: <boolean> | default = false ]

# Optional field that can be used to specify which domain alert is related to.
[ entity: <tmpl_string> ]

# Comma separated list of actions that will be available for the alert.
[ actions: <tmpl_string> ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<responder>

# Exactly one of these fields should be defined.
[ id: <tmpl_string> ]
[ name: <tmpl_string> ]
[ username: <tmpl_string> ]

# "team", "teams", "user", "escalation" or "schedule".
type: <tmpl_string>

<pagerduty_config>

PagerDuty 通知通过 PagerDuty API 发送。PagerDuty 提供了关于如何集成的 文档。Alertmanager 的 v0.11 及更高版本对 PagerDuty 的 Events API v2 的支持存在重要差异。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The routing and service keys are mutually exclusive.
# The PagerDuty integration key (when using PagerDuty integration type `Events API v2`).
# It is mutually exclusive with `routing_key_file`.
routing_key: <tmpl_secret>
# Read the Pager Duty routing key from a file.
# It is mutually exclusive with `routing_key`.
routing_key_file: <filepath>
# The PagerDuty integration key (when using PagerDuty integration type `Prometheus`).
# It is mutually exclusive with `service_key_file`.
service_key: <tmpl_secret>
# Read the Pager Duty service key from a file.
# It is mutually exclusive with `service_key`.
service_key_file: <filepath>

# The URL to send API requests to
[ url: <string> | default = global.pagerduty_url ]

# The client identification of the Alertmanager.
[ client:  <tmpl_string> | default = '{{ template "pagerduty.default.client" . }}' ]
# A backlink to the sender of the notification.
[ client_url:  <tmpl_string> | default = '{{ template "pagerduty.default.clientURL" . }}' ]

# A description of the incident.
[ description: <tmpl_string> | default = '{{ template "pagerduty.default.description" .}}' ]

# Severity of the incident.
[ severity: <tmpl_string> | default = 'error' ]

# Unique location of the affected system.
[ source: <tmpl_string> | default = client ]

# A set of arbitrary key/value pairs that provide further detail
# about the incident.
[ details: { <string>: <tmpl_string>, ... } | default = {
  firing:       '{{ template "pagerduty.default.instances" .Alerts.Firing }}'
  resolved:     '{{ template "pagerduty.default.instances" .Alerts.Resolved }}'
  num_firing:   '{{ .Alerts.Firing | len }}'
  num_resolved: '{{ .Alerts.Resolved | len }}'
} ]

# Images to attach to the incident.
images:
  [ <image_config> ... ]

# Links to attach to the incident.
links:
  [ <link_config> ... ]

# The part or component of the affected system that is broken.
[ component: <tmpl_string> ]

# A cluster or grouping of sources.
[ group: <tmpl_string> ]

# The class/type of the event.
[ class: <tmpl_string> ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<image_config>

这些字段在 PagerDuty API 文档 中有说明。

href: <tmpl_string>
src: <tmpl_string>
alt: <tmpl_string>

这些字段在 PagerDuty API 文档 中有说明。

href: <tmpl_string>
text: <tmpl_string>

<pushover_config>

Pushover 通知通过 Pushover API 发送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The recipient user's key.
# user_key and user_key_file are mutually exclusive.
user_key: <secret>
user_key_file: <filepath>

# Your registered application's API token, see https://pushover.net/apps
# You can also register a token by cloning this Prometheus app:
# https://pushover.net/apps/clone/prometheus
# token and token_file are mutually exclusive.
token: <secret>
token_file: <filepath>

# Notification title.
[ title: <tmpl_string> | default = '{{ template "pushover.default.title" . }}' ]

# Notification message.
[ message: <tmpl_string> | default = '{{ template "pushover.default.message" . }}' ]

# A supplementary URL shown alongside the message.
[ url: <tmpl_string> | default = '{{ template "pushover.default.url" . }}' ]

# Optional device to send notification to, see https://pushover.net/api#device
[ device: <string> ]

# Optional sound to use for notification, see https://pushover.net/api#sound
[ sound: <string> ]

# Priority, see https://pushover.net/api#priority
[ priority: <tmpl_string> | default = '{{ if eq .Status "firing" }}2{{ else }}0{{ end }}' ]

# How often the Pushover servers will send the same notification to the user.
# Must be at least 30 seconds.
[ retry: <duration> | default = 1m ]

# How long your notification will continue to be retried for, unless the user
# acknowledges the notification.
[ expire: <duration> | default = 1h ]

# Optional time to live (TTL) to use for notification, see https://pushover.net/api#ttl
[ ttl: <duration> ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<slack_config>

Slack 通知可以通过 传入 WebhookBot 令牌 发送。

如果使用传入 Webhook,则 api_url 必须设置为传入 Webhook 的 URL,或者写入 api_url_file 中引用的文件中。

如果使用 Bot 令牌,则 api_url 必须设置为 https://slack.com/api/chat.postMessage,Bot 令牌必须在 http_config 中设置为授权凭据,并且 channel 必须包含要发送通知到的频道的名称或频道 ID。如果使用频道名称,则 # 可选。

通知包含一个 附件

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]

# The Slack webhook URL. Either api_url or api_url_file should be set.
# Defaults to global settings if none are set here.
[ api_url: <secret> | default = global.slack_api_url ]
[ api_url_file: <filepath> | default = global.slack_api_url_file ]

# The channel or user to send notifications to.
channel: <tmpl_string>

# API request data as defined by the Slack webhook API.
[ icon_emoji: <tmpl_string> ]
[ icon_url: <tmpl_string> ]
[ link_names: <boolean> | default = false ]
[ username: <tmpl_string> | default = '{{ template "slack.default.username" . }}' ]
# The following parameters define the attachment.
actions:
  [ <action_config> ... ]
[ callback_id: <tmpl_string> | default = '{{ template "slack.default.callbackid" . }}' ]
[ color: <tmpl_string> | default = '{{ if eq .Status "firing" }}danger{{ else }}good{{ end }}' ]
[ fallback: <tmpl_string> | default = '{{ template "slack.default.fallback" . }}' ]
fields:
  [ <field_config> ... ]
[ footer: <tmpl_string> | default = '{{ template "slack.default.footer" . }}' ]
[ mrkdwn_in: '[' <string>, ... ']' | default = ["fallback", "pretext", "text"] ]
[ pretext: <tmpl_string> | default = '{{ template "slack.default.pretext" . }}' ]
[ short_fields: <boolean> | default = false ]
[ text: <tmpl_string> | default = '{{ template "slack.default.text" . }}' ]
[ title: <tmpl_string> | default = '{{ template "slack.default.title" . }}' ]
[ title_link: <tmpl_string> | default = '{{ template "slack.default.titlelink" . }}' ]
[ image_url: <tmpl_string> ]
[ thumb_url: <tmpl_string> ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<action_config>

这些字段在 Slack API 文档中关于 消息附件交互式消息 中有说明。

text: <tmpl_string>
type: <tmpl_string>
# Either url or name and value are mandatory.
[ url: <tmpl_string> ]
[ name: <tmpl_string> ]
[ value: <tmpl_string> ]

[ confirm: <action_confirm_field_config> ]
[ style: <tmpl_string> | default = '' ]
<action_confirm_field_config>

这些字段在 Slack API 文档 中有说明。

text: <tmpl_string>
[ dismiss_text: <tmpl_string> | default '' ]
[ ok_text: <tmpl_string> | default '' ]
[ title: <tmpl_string> | default '' ]

<field_config>

这些字段在 Slack API 文档 中有说明。

title: <tmpl_string>
value: <tmpl_string>
[ short: <boolean> | default = slack_config.short_fields ]

<sns_config>

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The SNS API URL i.e. https://sns.us-east-2.amazonaws.com.
#  If not specified, the SNS API URL from the SNS SDK will be used.
[ api_url: <tmpl_string> ]

# Configures AWS's Signature Verification 4 signing process to sign requests.
sigv4:
  [ <sigv4_config> ]

# SNS topic ARN, i.e. arn:aws:sns:us-east-2:698519295917:My-Topic
# If you don't specify this value, you must specify a value for the phone_number or target_arn.
# If you are using a FIFO SNS topic you should set a message group interval longer than 5 minutes
# to prevent messages with the same group key being deduplicated by the SNS default deduplication window
[ topic_arn: <tmpl_string> ]

# Subject line when the message is delivered to email endpoints.
[ subject: <tmpl_string> | default = '{{ template "sns.default.subject" .}}' ]

# Phone number if message is delivered via SMS in E.164 format.
# If you don't specify this value, you must specify a value for the topic_arn or target_arn.
[ phone_number: <tmpl_string> ]

# The  mobile platform endpoint ARN if message is delivered via mobile notifications.
# If you don't specify this value, you must specify a value for the topic_arn or phone_number.
[ target_arn: <tmpl_string> ]

# The message content of the SNS notification.
[ message: <tmpl_string> | default = '{{ template "sns.default.message" .}}' ]

# SNS message attributes.
attributes:
  [ <string>: <string> ... ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<sigv4_config>

# The AWS region. If blank, the region from the default credentials chain is used.
[ region: <string> ]

# The AWS API keys. Both access_key and secret_key must be supplied or both must be blank.
# If blank the environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are used.
[ access_key: <string> ]
[ secret_key: <secret> ]

# Named AWS profile used to authenticate.
[ profile: <string> ]

# AWS Role ARN, an alternative to using AWS API keys.
[ role_arn: <string> ]

<telegram_config>

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The Telegram API URL i.e. https://api.telegram.org.
# If not specified, default API URL will be used.
[ api_url: <string> | default = global.telegram_api_url ]

# Telegram bot token. It is mutually exclusive with `bot_token_file`.
[ bot_token: <secret> ]

# Read the Telegram bot token from a file. It is mutually exclusive with `bot_token`.
[ bot_token_file: <filepath> ]

# ID of the chat where to send the messages.
[ chat_id: <int> ]

# Message template.
[ message: <tmpl_string> default = '{{ template "telegram.default.message" .}}' ]

# Disable telegram notifications
[ disable_notifications: <boolean> | default = false ]

# Parse mode for telegram message, supported values are MarkdownV2, Markdown, HTML and empty string for plain text.
[ parse_mode: <string> | default = "HTML" ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<victorops_config>

VictorOps 通知通过 VictorOps API 发送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The API key to use when talking to the VictorOps API.
# It is mutually exclusive with `api_key_file`.
[ api_key: <secret> | default = global.victorops_api_key ]

# Reads the API key to use when talking to the VictorOps API from a file.
# It is mutually exclusive with `api_key`.
[ api_key_file: <filepath> | default = global.victorops_api_key_file ]

# The VictorOps API URL.
[ api_url: <string> | default = global.victorops_api_url ]

# A key used to map the alert to a team.
routing_key: <tmpl_string>

# Describes the behavior of the alert (CRITICAL, WARNING, INFO).
[ message_type: <tmpl_string> | default = 'CRITICAL' ]

# Contains summary of the alerted problem.
[ entity_display_name: <tmpl_string> | default = '{{ template "victorops.default.entity_display_name" . }}' ]

# Contains long explanation of the alerted problem.
[ state_message: <tmpl_string> | default = '{{ template "victorops.default.state_message" . }}' ]

# The monitoring tool the state message is from.
[ monitoring_tool: <tmpl_string> | default = '{{ template "victorops.default.monitoring_tool" . }}' ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<webhook_config>

Webhook 接收器允许配置一个通用接收器。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The endpoint to send HTTP POST requests to.
# url and url_file are mutually exclusive.
url: <secret>
url_file: <filepath>

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

# The maximum number of alerts to include in a single webhook message. Alerts
# above this threshold are truncated. When leaving this at its default value of
# 0, all alerts are included.
[ max_alerts: <int> | default = 0 ]

Alertmanager 将以以下 JSON 格式向配置的端点发送 HTTP POST 请求

{
  "version": "4",
  "groupKey": <string>,              // key identifying the group of alerts (e.g. to deduplicate)
  "truncatedAlerts": <int>,          // how many alerts have been truncated due to "max_alerts"
  "status": "<resolved|firing>",
  "receiver": <string>,
  "groupLabels": <object>,
  "commonLabels": <object>,
  "commonAnnotations": <object>,
  "externalURL": <string>,           // backlink to the Alertmanager.
  "alerts": [
    {
      "status": "<resolved|firing>",
      "labels": <object>,
      "annotations": <object>,
      "startsAt": "<rfc3339>",
      "endsAt": "<rfc3339>",
      "generatorURL": <string>,      // identifies the entity that caused the alert
      "fingerprint": <string>        // fingerprint to identify the alert
    },
    ...
  ]
}

有一个使用此功能的 集成 列表。

<wechat_config>

微信通知通过 微信 API 发送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]

# The API key to use when talking to the WeChat API.
[ api_secret: <secret> | default = global.wechat_api_secret ]

# The WeChat API URL.
[ api_url: <string> | default = global.wechat_api_url ]

# The corp id for authentication.
[ corp_id: <string> | default = global.wechat_api_corp_id ]

# API request data as defined by the WeChat API.
[ message: <tmpl_string> | default = '{{ template "wechat.default.message" . }}' ]
# Type of the message type, supported values are `text` and `markdown`.
[ message_type: <string> | default = 'text' ]
[ agent_id: <string> | default = '{{ template "wechat.default.agent_id" . }}' ]
[ to_user: <string> | default = '{{ template "wechat.default.to_user" . }}' ]
[ to_party: <string> | default = '{{ template "wechat.default.to_party" . }}' ]
[ to_tag: <string> | default = '{{ template "wechat.default.to_tag" . }}' ]

<webex_config>

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The Webex Teams API URL i.e. https://webexapis.com/v1/messages
# If not specified, default API URL will be used.
[ api_url: <string> | default = global.webex_api_url ]

# ID of the Webex Teams room where to send the messages.
room_id: <string>

# Message template.
[ message: <tmpl_string> default = '{{ template "webex.default.message" .}}' ]

# The HTTP client's configuration. You must use this configuration to supply the bot token as part of the HTTP `Authorization` header. 
[ http_config: <http_config> | default = global.http_config ]

本文档是 开源 的。请通过提交问题或拉取请求来帮助改进它。