配置

Alertmanager 通过命令行标志和配置文件进行配置。命令行标志用于配置不可变的系统参数,而配置文件定义了抑制规则、通知路由和通知接收器。

可以使用可视化编辑器 来帮助构建路由树。

要查看所有可用的命令行标志,请运行 alertmanager -h

Alertmanager 可以在运行时重新加载其配置。如果新配置格式不正确,更改将不会被应用,并且会记录一个错误。配置重新加载是通过向进程发送 SIGHUP 信号或向 /-/reload 端点发送 HTTP POST 请求来触发的。

限制

Alertmanager 通过命令行标志支持许多可配置的限制。

要限制静默的最大数量(包括过期的),请使用 --silences.max-silences 标志。您可以使用 --silences.max-per-silence-bytes 来限制单个静默的最大大小,单位是字节。

这两个限制默认都禁用。

配置文件简介

要指定要加载的配置文件,请使用 --config.file 标志。

./alertmanager --config.file=alertmanager.yml

该文件采用YAML 格式 编写,其结构由下文的模式描述。括号表示参数是可选的。对于非列表参数,值将设置为指定的默认值。

通用占位符定义如下

  • <duration>:一个持续时间,匹配正则表达式 ((([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?|0),例如 1d1h30m5m10s
  • <labelname>:一个字符串,匹配正则表达式 [a-zA-Z_][a-zA-Z0-9_]*
  • <labelvalue>:一个 Unicode 字符字符串
  • <filepath>:当前工作目录中的有效路径
  • <boolean>: 一个布尔值,可取 truefalse
  • <string>: 一个普通字符串
  • <secret>: 一个普通字符串,是秘密信息,如密码
  • <tmpl_string>:一个在使用前会被模板展开的字符串
  • <tmpl_secret>:一个在使用前会被模板展开的字符串,且它是一个敏感信息
  • <int>:一个整数值
  • <regex>:任何有效的 RE2 正则表达式 (正则表达式两端会被锚定。要取消锚定,请使用 .*<regex>.*。)

其他占位符单独指定。

一个提供的有效的示例文件 展示了如何使用。

文件布局和全局设置

全局配置指定了在所有其他配置上下文中都有效的参数。它们也用作其他配置部分的默认值。其他顶级部分在此页面的下方进行文档说明。

global:
  # The default SMTP From header field.
  [ smtp_from: <tmpl_string> ]
  # The default SMTP smarthost used for sending emails, including port number.
  # Port number usually is 25, or 587 for SMTP over TLS (sometimes referred to as STARTTLS).
  # Example: smtp.example.org:587
  [ smtp_smarthost: <string> ]
  # The default hostname to identify to the SMTP server.
  [ smtp_hello: <string> | default = "localhost" ]
  # SMTP Auth using CRAM-MD5, LOGIN and PLAIN. If empty, Alertmanager doesn't authenticate to the SMTP server.
  # PLAIN is only supported when using TLS.
  [ smtp_auth_username: <string> ]
  # SMTP Auth using LOGIN and PLAIN.
  [ smtp_auth_password: <secret> ]
  # SMTP Auth using LOGIN and PLAIN.
  [ smtp_auth_password_file: <string> ]
  # SMTP Auth using PLAIN.
  [ smtp_auth_identity: <string> ]
  # SMTP Auth using CRAM-MD5.
  [ smtp_auth_secret: <secret> ]
  # The default SMTP TLS requirement.
  # Note that Go does not support unencrypted connections to remote SMTP endpoints.
  [ smtp_require_tls: <bool> | default = true ]
  # The default TLS configuration for SMTP receivers
  [ smtp_tls_config: <tls_config> ]

  # Default settings for the JIRA integration.
  [ jira_api_url: <string> ]

  # The API URL to use for Slack notifications.
  [ slack_api_url: <secret> ]
  [ slack_api_url_file: <filepath> ]
  [ victorops_api_key: <secret> ]
  [ victorops_api_key_file: <filepath> ]
  [ victorops_api_url: <string> | default = "https://alert.victorops.com/integrations/generic/20131114/alert/" ]
  [ pagerduty_url: <string> | default = "https://events.pagerduty.com/v2/enqueue" ]
  [ opsgenie_api_key: <secret> ]
  [ opsgenie_api_key_file: <filepath> ]
  [ opsgenie_api_url: <string> | default = "https://api.opsgenie.com/" ]
  [ rocketchat_api_url: <string> | default = "https://open.rocket.chat/" ]
  [ rocketchat_token: <secret> ]
  [ rocketchat_token_file: <filepath> ]
  [ rocketchat_token_id: <secret> ]
  [ rocketchat_token_id_file: <filepath> ]
  [ wechat_api_url: <string> | default = "https://qyapi.weixin.qq.com/cgi-bin/" ]
  [ wechat_api_secret: <secret> ]
  [ wechat_api_corp_id: <string> ]
  [ telegram_api_url: <string> | default = "https://api.telegram.org" ]
  [ webex_api_url: <string> | default = "https://webexapis.com/v1/messages" ]
  # The default HTTP client configuration
  [ http_config: <http_config> ]

  # ResolveTimeout is the default value used by alertmanager if the alert does
  # not include EndsAt, after this time passes it can declare the alert as resolved if it has not been updated.
  # This has no impact on alerts from Prometheus, as they always include EndsAt.
  [ resolve_timeout: <duration> | default = 5m ]

# Files from which custom notification template definitions are read.
# The last component may use a wildcard matcher, e.g. 'templates/*.tmpl'.
templates:
  [ - <filepath> ... ]

# The root node of the routing tree.
route: <route>

# A list of notification receivers.
receivers:
  - <receiver> ...

# A list of inhibition rules.
inhibit_rules:
  [ - <inhibit_rule> ... ]

# DEPRECATED: use time_intervals below.
# A list of mute time intervals for muting routes.
mute_time_intervals:
  [ - <time_interval> ... ]

# A list of time intervals for muting/activating routes.
time_intervals:
  [ - <time_interval> ... ]

路由相关设置允许配置告警如何基于时间进行路由、聚合、限流和静默。

<route>

路由块定义了路由树中的一个节点及其子节点。如果未设置,其可选配置参数将继承自其父节点。

每个告警都会在配置的顶级路由处进入路由树,该路由必须匹配所有告警(即没有任何配置的匹配器)。然后它会遍历子节点。如果 continue 设置为 false,它会在第一个匹配的子节点后停止。如果匹配的节点上的 continue 为 true,告警将继续匹配后续的同级节点。如果告警不匹配节点的任何子节点(没有匹配的子节点,或不存在),告警将根据当前节点的配置参数进行处理。

有关分组的更多信息,请参阅 Alertmanager 概念

[ receiver: <string> ]
# The labels by which incoming alerts are grouped together. For example,
# multiple alerts coming in for cluster=A and alertname=LatencyHigh would
# be batched into a single group.
#
# To aggregate by all possible labels use the special value '...' as the sole label name, for example:
# group_by: ['...']
# This effectively disables aggregation entirely, passing through all
# alerts as-is. This is unlikely to be what you want, unless you have
# a very low alert volume or your upstream notification system performs
# its own grouping.
[ group_by: '[' <labelname>, ... ']' ]

# Whether an alert should continue matching subsequent sibling nodes.
[ continue: <boolean> | default = false ]

# DEPRECATED: Use matchers below.
# A set of equality matchers an alert has to fulfill to match the node.
match:
  [ <labelname>: <labelvalue>, ... ]

# DEPRECATED: Use matchers below.
# A set of regex-matchers an alert has to fulfill to match the node.
match_re:
  [ <labelname>: <regex>, ... ]

# A list of matchers that an alert has to fulfill to match the node.
matchers:
  [ - <matcher> ... ]

# How long to wait before sending the first notification for a new group of
# alerts. Allows to wait for alerts to arrive from other rule groups or
# Prometheus servers, and for one or more inhibiting alerts to arrive and mute
# any target alerts before the first notification.
#
# A short group_wait will reduce the time to wait before sending the first
# notification for a new group of alerts. However, if group_wait is too short
# then the first notification might not contain the complete set of expected
# alerts, and alerts that should be inhibited might not be inhibited if the
# inhibiting alerts have not arrived in time.
#
# A long group_wait will increase the time to wait before sending the first
# notification for a new group of alerts. However, if group_wait is too long
# then notifications for firing alerts might not be sent within a reasonable
# time.
#
# If an alert is resolved before group_wait has elapsed, no notification will
# be sent for that alert. This reduces noise of flapping alerts.

# A notification for any alerts that missed the initial group_wait will be
# sent at the next group_interval instead.
#
# If omitted, child routes inherit the group_wait of the parent route.
[ group_wait: <duration> | default = 30s ]

# How long to wait before sending subsequent notifications for an existing
# group of alerts after group_wait.
#
# The group_interval is a recurring timer that starts as soon as group_wait
# has elapsed. At each group_interval, Alertmanager checks if any new alerts
# have fired or any firing alerts have resolved since the last group_interval,
# and if they have a notification is sent. If they haven't, Alertmanager checks
# if the repeat_interval has elapsed instead.
#
# If omitted, child routes inherit the group_interval of the parent route.
[ group_interval: <duration> | default = 5m ]

# How long to wait before repeating the last notification. Notifications are
# not repeated if any new alerts have fired or any firing alerts have resolved
# since the last group_interval.
#
# Since the repeat_interval is checked after each group_interval, it should
# be a multiple of the group_interval. If it's not, the repeat_interval
# is rounded up to the next multiple of the group_interval.
#
# In addition, if repeat_interval is longer then `--data.retention`, the
# notification will be repeated at the end of the data retention period
# instead.
#
# If omitted, child routes inherit the repeat_interval of the parent route.
[ repeat_interval: <duration> | default = 4h ]

# Times when the route should be muted. These must match the name of a
# time interval defined in the time_intervals section.
# Additionally, the root node cannot have any mute times.
# When a route is muted it will not send any notifications, but
# otherwise acts normally (including ending the route-matching process
# if the `continue` option is not set.)
mute_time_intervals:
  [ - <string> ...]

# Times when the route should be active. These must match the name of a
# time interval defined in the time_intervals section. An empty value
# means that the route is always active.
# Additionally, the root node cannot have any active times.
# The route will send notifications only when active, but otherwise
# acts normally (including ending the route-matching process
# if the `continue` option is not set).
active_time_intervals:
  [ - <string> ...]

# Zero or more child routes.
routes:
  [ - <route> ... ]

示例

# The root route with all parameters, which are inherited by the child
# routes if they are not overwritten.
route:
  receiver: 'default-receiver'
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h
  group_by: [cluster, alertname]
  # All alerts that do not match the following child routes
  # will remain at the root node and be dispatched to 'default-receiver'.
  routes:
  # All alerts with service=mysql or service=cassandra
  # are dispatched to the database pager.
  - receiver: 'database-pager'
    group_wait: 10s
    matchers:
    - service=~"mysql|cassandra"
  # All alerts with the team=frontend label match this sub-route.
  # They are grouped by product and environment rather than cluster
  # and alertname.
  - receiver: 'frontend-pager'
    group_by: [product, environment]
    matchers:
    - team="frontend"

  # All alerts with the service=inhouse-service label match this sub-route.
  # the route will be muted during offhours and holidays time intervals.
  # even if it matches, it will continue to the next sub-route
  - receiver: 'dev-pager'
    matchers:
      - service="inhouse-service"
    mute_time_intervals:
      - offhours
      - holidays
    continue: true

    # All alerts with the service=inhouse-service label match this sub-route
    # the route will be active only during offhours and holidays time intervals.
  - receiver: 'on-call-pager'
    matchers:
      - service="inhouse-service"
    active_time_intervals:
      - offhours
      - holidays

<time_interval>

time_interval 定义了一个命名的时间间隔,可以在路由树中引用,用于在一天中的特定时间静默/激活特定路由。

name: <string>
time_intervals:
  [ - <time_interval_spec> ... ]

<time_interval_spec>

time_interval_spec 包含时间间隔的实际定义。语法支持以下字段:

- times:
  [ - <time_range> ...]
  weekdays:
  [ - <weekday_range> ...]
  days_of_month:
  [ - <days_of_month_range> ...]
  months:
  [ - <month_range> ...]
  years:
  [ - <year_range> ...]
  location: <string>

所有字段都是列表。在每个非空列表中,至少有一个元素必须满足条件才能匹配该字段。如果某个字段未指定,则任何值都将匹配该字段。为了使某个时间点匹配一个完整的时间间隔,所有字段都必须匹配。某些字段支持范围和负索引,下面将详细介绍。如果未指定时区,则时间将被视为 UTC 时间。

time_range:范围包含开始时间但不包含结束时间,以便于表示在整点开始/结束的时间。例如,start_time: '17:00'end_time: '24:00' 将从 17:00 开始,并在 24:00 之前结束。它们的指定方式如下:

times:
- start_time: HH:MM
  end_time: HH:MM

weekday_range:一周中的日期列表,一周从周日开始,到周六结束。日期应按名称指定(例如,“Sunday”)。为方便起见,范围也接受 <start_day>:<end_day> 形式,并且两端都包含。例如:['monday:wednesday','saturday', 'sunday']

days_of_month_range:月份数字列表。日期从 1 开始。也接受负值,从月份末尾开始计数,例如,1 月份的 -1 将表示 1 月 31 日。例如:['1:5', '-3:-1']。超出月份的开始或结束日期将被截断。例如,在 2 月份指定 ['1:31'] 将根据闰年将实际结束日期截断为 28 或 29。两端都包含。

month_range:日历月份列表,通过不区分大小写的名称(例如,“January”)或数字(其中 January = 1)标识。也接受范围。例如,['1:3', 'may:august', 'december']。两端都包含。

year_range:年份数字列表。接受范围。例如,['2020:2022', '2030']。两端都包含。

location:一个匹配 IANA 时区数据库中某个位置的字符串。例如,'Australia/Sydney'。该位置为时间间隔提供时区。例如,一个位置为 'Australia/Sydney' 的时间间隔,如果包含类似以下内容:

times:
- start_time: 09:00
  end_time: 17:00
weekdays: ['monday:friday']

将表示在悉尼,澳大利亚的本地时间,周一到周五,早上 9:00 到下午 5:00 之间的任何时间。

您也可以使用 'Local' 作为位置来使用 Alertmanager 运行所在机器的本地时间,或者使用 'UTC' 来表示 UTC 时间。如果未提供时区,则时间间隔默认为 UTC 时间。注意: 在 Windows 上,除非您使用 ZONEINFO 环境变量提供自定义时区数据库,否则仅支持 LocalUTC

抑制允许基于其他告警的存在来静默一组告警。这允许在系统或服务之间建立依赖关系,以便在发生故障时只发出最相关的相互关联的告警。

有关抑制的更多信息,请参阅 Alertmanager 概念

<inhibit_rule>

抑制规则在存在匹配另一组匹配器的告警(源告警)时,静默匹配一组匹配器的告警(目标告警)。目标告警和源告警必须具有 equal 列表中标签名称的相同标签值。

语义上,缺失的标签和具有空值的标签是相同的。因此,如果 equal 中列出的所有标签名称在源告警和目标告警中都缺失,则抑制规则将适用。

为防止告警抑制自身,匹配规则的目标两边的告警,可能不会被具有相同情况的告警(包括自身)所抑制。然而,我们建议选择目标和源匹配器,使其告警永远不会同时匹配两边。这更容易理解,并且不会触发这种特殊情况。

# Optional name of the inhibition rule.
# Duplicate names are allowed but will affect the per-rule metrics.
name: <string>

# DEPRECATED: Use target_matchers below.
# Matchers that have to be fulfilled in the alerts to be muted.
target_match:
  [ <labelname>: <labelvalue>, ... ]
# DEPRECATED: Use target_matchers below.
target_match_re:
  [ <labelname>: <regex>, ... ]

# A list of matchers that have to be fulfilled by the target
# alerts to be muted.
target_matchers:
  [ - <matcher> ... ]

# DEPRECATED: Use source_matchers below.
# Matchers for which one or more alerts have to exist for the
# inhibition to take effect.
source_match:
  [ <labelname>: <labelvalue>, ... ]
# DEPRECATED: Use source_matchers below.
source_match_re:
  [ <labelname>: <regex>, ... ]

# A list of matchers for which one or more alerts have
# to exist for the inhibition to take effect.
source_matchers:
  [ - <matcher> ... ]

# Labels that must have an equal value in the source and target
# alert for the inhibition to take effect.
[ equal: '[' <labelname>, ... ']' ]

标签匹配器

标签匹配器将告警匹配到路由、静默和抑制规则。

重要提示:Prometheus 正在增加对标签和度量中 UTF-8 的支持。为了在 Alertmanager 中也支持 UTF-8,Alertmanager 0.27 及更高版本有一个新的匹配器解析器,它包含一些不兼容的更改。虽然大多数匹配器将向前兼容,但有些则不会。Alertmanager 正在进行一个过渡期,在此期间它同时支持 UTF-8 和经典匹配器,并提供了一些工具来帮助您为过渡做准备。

如果这是新的 Alertmanager 安装,我们建议在创建 Alertmanager 配置文件之前启用 UTF-8 严格模式。您可以在此处找到有关如何启用 UTF-8 严格模式的说明。

如果这是现有的 Alertmanager 安装,我们建议在启用 UTF-8 严格模式之前,将 Alertmanager 运行在名为回退模式的默认模式下。在此模式下,如果您的配置文件需要进行任何更改才能启用 UTF-8 严格模式,Alertmanager 将记录一个警告。Alertmanager 将在接下来的两个版本中将 UTF-8 严格模式设为默认值,因此尽快过渡非常重要。

无论 Alertmanager 安装是新的还是现有的,您也可以使用 amtool 在 Alertmanager 服务器中启用 UTF-8 严格模式之前,验证 Alertmanager 配置文件是否与 UTF-8 严格模式兼容。您不需要运行 Alertmanager 服务器来进行此操作。您可以在此处找到有关如何使用 amtool 验证 Alertmanager 配置文件的说明。

Alertmanager 服务器运行模式

在过渡期内,Alertmanager 支持三种运行模式。它们被称为回退模式、UTF-8 严格模式和经典模式。回退模式是默认模式。

Alertmanager 服务器的运维人员应在过渡期结束前过渡到 UTF-8 严格模式。Alertmanager 将在接下来的两个版本中将 UTF-8 严格模式设为默认值,因此尽快过渡非常重要。

回退模式

Alertmanager 以一种称为回退模式的特殊模式运行,作为其默认模式。作为运维人员,您应该不会 in 您的路由、静默或抑制规则工作方式上遇到任何差异。

在回退模式下,配置首先被解析为 UTF-8 匹配器,如果与 UTF-8 解析器不兼容,则再被解析为经典匹配器。如果您的 Alertmanager 配置包含与 UTF-8 解析器不兼容的匹配器,Alertmanager 将把它们解析为经典匹配器并记录警告。此警告还包含一个关于如何将匹配器从经典匹配器更改为 UTF-8 匹配器的建议。例如:

ts=2024-02-11T10:00:00Z caller=parse.go:176 level=warn msg="Alertmanager is moving to a new parser for labels and matchers, and this input is incompatible. Alertmanager has instead parsed the input using the classic matchers parser as a fallback. To make this input compatible with the UTF-8 matchers parser please make sure all regular expressions and values are double-quoted and backslashes are escaped. If you are still seeing this message please open an issue." input="foo=" origin=config err="end of input: expected label value" suggestion="foo=\"\""

在此,匹配器 foo= 可以通过双引号括起表达式的右侧来成为一个有效的 UTF-8 匹配器,得到 foo=""。这两个匹配器是等效的,但是对于 UTF-8 匹配器,匹配器的右侧是一个必需字段。

在罕见的情况下,配置可能会导致 UTF-8 和经典解析器之间出现不一致。当一个匹配器在两个解析器中都有效,但由于增加了对 UTF-8 的支持,导致使用哪个解析器会产生不同的解析结果时,就会发生这种情况。如果您的 Alertmanager 配置存在不一致,Alertmanager 将使用经典解析器并记录警告。例如:

ts=2024-02-11T10:00:00Z caller=parse.go:183 level=warn msg="Matchers input has disagreement" input="qux=\"\\xf0\\x9f\\x99\\x82\"\n" origin=config

任何不一致的出现都应根据具体情况进行处理,因为根据不一致的性质,配置在启用 UTF-8 严格模式之前可能不需要更新。例如,\xf0\x9f\x99\x82 是 🙂 表情符号的字节序列。如果意图是匹配一个字面的 🙂 表情符号,则无需更改。但是,如果意图是匹配字面的 \xf0\x9f\x99\x82,则应将匹配器更改为 qux="\\xf0\\x9f\\x99\\x82"

UTF-8 严格模式

在 UTF-8 严格模式下,Alertmanager 禁用了对经典匹配器的支持。

alertmanager --config.file=config.yml --enable-feature="utf8-strict-mode"

此模式应为新的 Alertmanager 安装启用,并且现有的 Alertmanager 安装在解决了所有不兼容匹配器的警告之后再启用。Alertmanager 在 UTF-8 严格模式下不会启动,直到所有不兼容匹配器的警告都得到解决。

ts=2024-02-11T10:00:00Z caller=coordinator.go:118 level=error component=configuration msg="Loading configuration file failed" file=config.yml err="end of input: expected label value"

在过渡期结束时,UTF-8 严格模式将成为 Alertmanager 的默认模式。

经典模式

经典模式等同于 Alertmanager 0.26.0 及更早版本。

alertmanager --config.file=config.yml --enable-feature="classic-mode"

如果怀疑回退模式或 UTF-8 严格模式存在问题,您可以使用此模式。在这种情况下,请在 GitHub 上打开一个问题,并提供尽可能多的信息。

验证

您可以使用 amtool 在 Alertmanager 服务器中启用 UTF-8 严格模式之前,验证 Alertmanager 配置文件是否与 UTF-8 严格模式兼容。您不需要运行 Alertmanager 服务器来进行此操作。

与 Alertmanager 服务器一样,如果配置不兼容或存在不一致,amtool 将会记录警告。

amtool check-config config.yml
Checking 'config.yml'
level=warn msg="Alertmanager is moving to a new parser for labels and matchers, and this input is incompatible. Alertmanager has instead parsed the input using the classic matchers parser as a fallback. To make this input compatible with the UTF-8 matchers parser please make sure all regular expressions and values are double-quoted and backslashes are escaped. If you are still seeing this message please open an issue." input="foo=" origin=config err="end of input: expected label value" suggestion="foo=\"\""
level=warn msg="Matchers input has disagreement" input="qux=\"\\xf0\\x9f\\x99\\x82\"\n" origin=config
  SUCCESS
Found:
 - global config
 - route
 - 2 inhibit rules
 - 2 receivers
 - 0 templates

amtool 未记录任何警告时,您就知道配置与 UTF-8 严格模式兼容。

amtool check-config config.yml
Checking 'config.yml'  SUCCESS
Found:
 - global config
 - route
 - 2 inhibit rules
 - 2 receivers
 - 0 templates

您还可以使用 amtool 的 UTF-8 严格模式作为额外的验证级别。如果命令失败,您就知道配置是无效的。

amtool check-config config.yml --enable-feature="utf8-strict-mode"
level=warn msg="UTF-8 mode enabled"
Checking 'config.yml'  FAILED: end of input: expected label value

amtool: error: failed to validate 1 file(s)

如果命令成功,您就知道配置是有效的。

amtool check-config config.yml --enable-feature="utf8-strict-mode"
level=warn msg="UTF-8 mode enabled"
Checking 'config.yml'  SUCCESS
Found:
 - global config
 - route
 - 2 inhibit rules
 - 2 receivers
 - 0 templates

<matcher>

UTF-8 匹配器

一个 UTF-8 匹配器包含三个令牌:

  • 未加引号的字面量或用于标签名称的双引号字符串。
  • =!==~!~ 之一。= 表示等于,!= 表示不等于,=~ 表示匹配正则表达式,!~ 表示不匹配正则表达式。
  • 未加引号的字面量或用于正则表达式或标签值的双引号字符串。

未加引号的字面量可以包含除保留字符外的所有 UTF-8 字符。保留字符包括空格以及 { } ! = ~ , \ " ' ` 中的所有字符。例如,foo[a-zA-Z]+Προμηθεύς(希腊语的 Prometheus)都是有效的未加引号字面量的示例。但是,foo! 不是一个有效的字面量,因为 ! 是一个保留字符。

双引号字符串可以包含所有 UTF-8 字符。与未加引号的字面量不同,没有保留字符。但是,字面双引号和反斜杠必须用单个反斜杠转义。例如,要匹配正则表达式 \d+,反斜杠必须转义为 "\\d+"。这是因为双引号字符串遵循 Go 的字符串字面量 规则。双引号字符串也支持 UTF-8 代码点。例如,"foo!""bar,baz""\"baz qux\"""\xf0\x9f\x99\x82"

经典匹配器

经典匹配器是一个语法受到 PromQL 和 OpenMetrics 启发的字符串。经典匹配器的语法包含三个令牌:

  • 一个有效的 Prometheus 标签名称。
  • =!==~!~ 之一。= 表示等于,!= 表示字符串不相等,=~ 用于正则表达式的相等性,!~ 用于正则表达式的不相等性。它们的含义与 PromQL 选择器中已知的一致。
  • 一个 UTF-8 字符串,可以包含在双引号中。在每个令牌之前或之后,可以有任意数量的空格。

第三个令牌可以是空字符串。在第三个令牌内,适用 OpenMetrics 转义规则:\" 表示双引号,\n 表示换行符,\\ 表示字面反斜杠。未转义的 " 不得出现在第三个令牌内部(仅作为第一个或最后一个字符)。然而,字面换行符以及后跟 \n" 以外字符的单个 \ 字符是可以容忍的。在这种情况下,它们将作为字面反斜杠处理。

匹配器的组合

您可以组合匹配器来创建复杂的匹配表达式。组合时,所有匹配器都必须匹配,整个表达式才能匹配。例如,表达式 {alertname="Watchdog", severity=~"warning|critical"} 将匹配一个带有标签 alertname=Watchdog, severity=critical 的告警,但不匹配一个带有标签 alertname=Watchdog, severity=none 的告警,因为尽管 alertname 是 Watchdog,但 severity 既不是 warning 也不是 critical。

您可以使用 YAML 列表将匹配器组合成表达式。

matchers:
  - alertname = Watchdog
  - severity =~ "warning|critical"

或作为受 PromQL 启发的表达式,其中每个匹配器都用逗号分隔。

{alertname="Watchdog", severity=~"warning|critical"}

允许使用单个末尾逗号。

{alertname="Watchdog", severity=~"warning|critical",}

开括号 { 和闭括号 } 是可选的。

alertname="Watchdog", severity=~"warning|critical"

但是,两者都必须存在或被省略。您不能有不完整的开括号或闭括号。

{alertname="Watchdog", severity=~"warning|critical"
alertname="Watchdog", severity=~"warning|critical"}

您也不能有重复的开括号或闭括号。

{{alertname="Watchdog", severity=~"warning|critical",}}

在双引号之外允许使用空格(空格、制表符和换行符),它们对匹配器本身没有影响。例如:

{
   alertname = "Watchdog",
   severity =~ "warning|critical",
}

等同于

{alertname="Watchdog",severity=~"warning|critical"}

更多示例

以下是一些更多示例:

  1. 两个等于匹配器组合为 YAML 列表。

    matchers:
      - foo = bar
      - dings != bums
  2. 两个匹配器组合为短格式 YAML 列表。

    matchers: [ foo = bar, dings != bums ]

    如下所示,在短格式中,最好使用双引号以避免逗号等特殊字符引起的问题。

    matchers: [ "foo = \"bar,baz\"", "dings != bums" ]
  3. 您也可以将两个匹配器放入一个类似 PromQL 的字符串中。单引号在这里效果最好。

    matchers: [ '{foo="bar", dings!="bums"}' ]
  4. 为避免 YAML 中转义和引号规则的问题,您也可以使用 YAML 块。

    matchers:
      - |
          {quote=~"She said: \"Hi, all!( How're you…)?\""}

这些接收器设置允许配置通知目标(接收器)以及基于 HTTP 的接收器的 HTTP 客户端选项。

<receiver>

接收器是单个或多个通知集成的一个命名配置。

注意:在解除对新接收器设置的禁令之后,同意除了现有要求之外,新的通知集成还需要一个已提交的维护者,并具有推送权限。

# The unique name of the receiver.
name: <string>

# Configurations for several notification integrations.
discord_configs:
  [ - <discord_config>, ... ]
email_configs:
  [ - <email_config>, ... ]
mattermost_configs:
  [ - <mattermost_config>, ... ]
msteams_configs:
  [ - <msteams_config>, ... ]
msteamsv2_configs:
  [ - <msteamsv2_config>, ... ]
jira_configs:
  [ - <jira_config>, ... ]
opsgenie_configs:
  [ - <opsgenie_config>, ... ]
pagerduty_configs:
  [ - <pagerduty_config>, ... ]
incidentio_configs:
  [ - <incidentio_config>, ... ]
pushover_configs:
  [ - <pushover_config>, ... ]
rocketchat_configs:
  [ - <rocketchat_config>, ... ]
slack_configs:
  [ - <slack_config>, ... ]
sns_configs:
  [ - <sns_config>, ... ]
telegram_configs:
  [ - <telegram_config>, ... ]
victorops_configs:
  [ - <victorops_config>, ... ]
webex_configs:
  [ - <webex_config>, ... ]
webhook_configs:
  [ - <webhook_config>, ... ]
wechat_configs:
  [ - <wechat_config>, ... ]

<http_config>

http_config 允许配置接收器用于与基于 HTTP 的 API 服务通信的 HTTP 客户端。

# Note that `basic_auth` and `authorization` options are mutually exclusive.

# Sets the `Authorization` header with the configured username and password.
# password and password_file are mutually exclusive.
basic_auth:
  [ username: <string> ]
  [ password: <secret> ]
  [ password_file: <string> ]

# Optional the `Authorization` header configuration.
authorization:
  # Sets the authentication type.
  [ type: <string> | default: Bearer ]
  # Sets the credentials. It is mutually exclusive with
  # `credentials_file`.
  [ credentials: <secret> ]
  # Sets the credentials with the credentials read from the configured file.
  # It is mutually exclusive with `credentials`.
  [ credentials_file: <filename> ]

# Optional OAuth 2.0 configuration.
# Cannot be used at the same time as basic_auth or authorization.
oauth2:
  [ <oauth2> ]

# Whether to enable HTTP2.
[ enable_http2: <bool> | default: true ]

# Optional proxy URL.
[ proxy_url: <string> ]
# Comma-separated string that can contain IPs, CIDR notation, domain names
# that should be excluded from proxying. IP and domain names can
# contain port numbers.
[ no_proxy: <string> ]
# Use proxy URL indicated by environment variables (HTTP_PROXY, http_proxy, HTTPS_PROXY, https_proxy, NO_PROXY, and no_proxy)
[ proxy_from_environment: <boolean> | default: false ]
# Specifies headers to send to proxies during CONNECT requests.
[ proxy_connect_header:
  [ <string>: [<secret>, ...] ] ]

# Configure whether HTTP requests follow HTTP 3xx redirects.
[ follow_redirects: <bool> | default = true ]

# Configures the TLS settings.
tls_config:
  [ <tls_config> ]

# Custom HTTP headers to be sent along with each request.
# Headers that are set by Prometheus itself can't be overwritten.
http_headers:
  [ <http_header> ]

<http_header>

# Header name.
<string>:
    # Header values.
    [ values: [<string>, ...] ]
    # Headers values. Hidden in configuration page.
    [ secrets: [<secret>, ...] ]
    # Files to read header values from.
    [ files: [<string>, ...] ]

<oauth2>

使用客户端凭据授权类型的 OAuth 2.0 身份验证。Alertmanager 使用给定的客户端访问密钥和 secret 密钥从指定端点获取访问令牌。

client_id: <string>
[ client_secret: <secret> ]

# Read the client secret from a file.
# It is mutually exclusive with `client_secret`.
[ client_secret_file: <filename> ]

# Scopes for the token request.
scopes:
  [ - <string> ... ]

# The URL to fetch the token from.
token_url: <string>

# Optional parameters to append to the token URL.
endpoint_params:
  [ <string>: <string> ... ]

# Configures the token request's TLS settings.
tls_config:
  [ <tls_config> ]

# Optional proxy URL.
[ proxy_url: <string> ]
# Comma-separated string that can contain IPs, CIDR notation, domain names
# that should be excluded from proxying. IP and domain names can
# contain port numbers.
[ no_proxy: <string> ]
# Use proxy URL indicated by environment variables (HTTP_PROXY, https_proxy, HTTPs_PROXY, https_proxy, and no_proxy)
[ proxy_from_environment: <boolean> | default: false ]
# Specifies headers to send to proxies during CONNECT requests.
[ proxy_connect_header:
  [ <string>: [<secret>, ...] ] ]

<tls_config>

tls_config 允许配置 TLS 连接。

# CA certificate to validate the server certificate with.
[ ca_file: <filepath> ]

# Certificate and key files for client cert authentication to the server.
[ cert_file: <filepath> ]
[ key_file: <filepath> ]

# ServerName extension to indicate the name of the server.
# http://tools.ietf.org/html/rfc4366#section-3.1
[ server_name: <string> ]

# Disable validation of the server certificate.
[ insecure_skip_verify: <boolean> | default = false]

# Minimum acceptable TLS version. Accepted values: TLS10 (TLS 1.0), TLS11 (TLS
# 1.1), TLS12 (TLS 1.2), TLS13 (TLS 1.3).
# If unset, Prometheus will use Go default minimum version, which is TLS 1.2.
# See MinVersion in https://pkg.go.dev/crypto/tls#Config.
[ min_version: <string> ]
# Maximum acceptable TLS version. Accepted values: TLS10 (TLS 1.0), TLS11 (TLS
# 1.1), TLS12 (TLS 1.2), TLS13 (TLS 1.3).
# If unset, Prometheus will use Go default maximum version, which is TLS 1.3.
# See MaxVersion in https://pkg.go.dev/crypto/tls#Config.
[ max_version: <string> ]

接收器集成设置

这些设置允许配置特定的接收器集成。

<discord_config>

Discord 通知通过 Discord webhook API  发送。请参阅 Discord 的 “Webhook 入门”文章 ,了解如何为频道配置 webhook 集成。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The Discord webhook URL.
# webhook_url and webhook_url_file are mutually exclusive.
webhook_url: <secret>
webhook_url_file: <filepath>

# Message title template.
[ title: <tmpl_string> | default = '{{ template "discord.default.title" . }}' ]

# Message body template.
[ message: <tmpl_string> | default = '{{ template "discord.default.message" . }}' ]

# Message content template. Limited to 2000 characters.
[ content: <tmpl_string> | default = '{{ template "discord.default.content" . }}' ]

# Message username.
[ username: <string> | default = '' ]

# Message avatar URL.
[ avatar_url: <string> | default = '' ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<email_config>

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]

# The email address to send notifications to.
# Allows a comma separated list of rfc5322 compliant email addresses.
to: <tmpl_string>

# The sender's address.
[ from: <tmpl_string> | default = global.smtp_from ]

# The SMTP host through which emails are sent.
[ smarthost: <string> | default = global.smtp_smarthost ]

# The hostname to identify to the SMTP server.
[ hello: <string> | default = global.smtp_hello ]

# SMTP authentication information.
# auth_password and auth_password_file are mutually exclusive.
[ auth_username: <string> | default = global.smtp_auth_username ]
[ auth_password: <secret> | default = global.smtp_auth_password ]
[ auth_password_file: <string> | default = global.smtp_auth_password_file ]
[ auth_secret: <secret> | default = global.smtp_auth_secret ]
[ auth_identity: <string> | default = global.smtp_auth_identity ]

# The SMTP TLS requirement.
# Note that Go does not support unencrypted connections to remote SMTP endpoints.
[ require_tls: <bool> | default = global.smtp_require_tls ]

# TLS configuration.
tls_config:
  [ <tls_config> | default = global.smtp_tls_config ]

# The HTML body of the email notification.
[ html: <tmpl_string> | default = '{{ template "email.default.html" . }}' ]
# The text body of the email notification.
[ text: <tmpl_string> ]

# Further headers email header key/value pairs. Overrides any headers
# previously set by the notification implementation.
[ headers: { <string>: <tmpl_string>, ... } ]

# Email threading configuration.
threading:
  # Whether to enable threading, which makes alert notifications in the same
  # alert group show up in the same email thread.
  [ enabled: <boolean> | default = false ]
  # What granularity of current date to thread by. Accepted values: daily, none.
  # (none means group by alert group key, no date).
  [ thread_by_date: <string> | default = daily ]

<mattermost_config>

Mattermost 通知通过 Mattermost webhook API  发送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The Mattermost webhook URL.
# webhook_url and webhook_url_file are mutually exclusive.
webhook_url: <secret>
webhook_url_file: <filepath>

# Overrides the channel the message posts in. Use the channel’s name and not the display name, e.g. use town-square, not Town Square.
[ channel: <string> | default = '' ]

# Overrides the username the message posts as.
# Defaults to the username set during webhook creation; if no username was set during creation, webhook is used.
[ username: <string> | default = '' ]

# Markdown-formatted message to display in the post.
# To trigger notifications, use @<username>, @channel, and @here like you would in other Mattermost messages.
text: <tmpl_string> | default = '{{ template "mattermost.default.text" . }}'

# Overrides the profile picture the message posts with.
[ icon_url: <string> | default = '' ]

# Overrides the profile picture and icon_url parameter.
[ icon_emoji: <string> | default = '' ]

# Message attachments used for richer formatting options.
# It is for compatibility with Slack.
[ attachments: ]
  [ <attachment_config> ... ]

[ props: <prop_config> ]

[ priority: <priority_config> ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<attachment_config>

有关更多信息,请参阅 Mattermost 文档 

[ fallback: <string> | default = '' ]
[ color: <string> | default = '' ]
[ pretext: <string> | default = '' ]
[ text: <string> | default = '' ]
[ author_name: <string> | default = '' ]
[ author_link: <string> | default = '' ]
[ author_icon: <string> | default = '' ]
[ title: <string> | default = '' ]
[ title_link: <string> | default = '' ]
# Same as Slack fields.
[ fields: <string> | default = '' ]
  [ <field_config> ... ]
[ thumb_url: <string> | default = '' ]
[ footer: <string> | default = '' ]
[ footer_icon: <string> | default = '' ]
[ image_url: <string> | default = '' ]

<prop_config>

# Props card allows for extra information (Markdown-formatted text) to be sent to Mattermost that will only be displayed in the RHS panel after a user selects the info icon displayed alongside the post.
[ card: <string> | default = '' ]

<priority_config>

# priority adds label to the message. Possible values are "urgent", "important" and "standard".
[ priority: <string> | default = '' ]

# If set to true, the message will be marked as requiring an acknowledgment from the users by displaying a checkmark icon next to the message. Keep in mind that this requires the message priority to be set to Important or Urgent.
# Only for enterprise version of Mattermost.
[ requested_ack: <bool> | default = false ]

# Only for Urgent messages. If set to true recipients will receive a persistent notification every five minutes until they acknowledge the message.
# Only for enterprise version of Mattermost.
[ persistent_notifications: <bool> | default = false ]

<msteams_config>

Microsoft Teams 通知通过 Incoming Webhooks  API 端点发送。

DEPRECATION NOTICE: Microsoft is deprecating the creation and usage of Microsoft 365 connectors via Microsoft Teams 。请考虑迁移到使用 Workflows  与 msteamsv2 配置。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The incoming webhook URL.
# webhook_url and webhook_url_file are mutually exclusive.
[ webhook_url: <secret> ]
[ webhook_url_file: <filepath> ]

# Message title template.
[ title: <tmpl_string> | default = '{{ template "msteams.default.title" . }}' ]

# Message summary template.
[ summary: <tmpl_string> | default = '{{ template "msteams.default.summary" . }}' ]

# Message body template.
[ text: <tmpl_string> | default = '{{ template "msteams.default.text" . }}' ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<msteamsv2_config>

Microsoft Teams v2 通知使用新的消息格式和自适应卡片,这是 flows  所要求的。请遵循 文档  以了解有关如何设置此集成的更多信息。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The incoming webhook URL.
# webhook_url and webhook_url_file are mutually exclusive.
[ webhook_url: <secret> ]
[ webhook_url_file: <filepath> ]

# Message title template.
[ title: <tmpl_string> | default = '{{ template "msteamsv2.default.title" . }}' ]

# Message body template.
[ text: <tmpl_string> | default = '{{ template "msteamsv2.default.text" . }}' ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<jira_config>

JIRA 通知通过 JIRA Rest API v2 JIRA REST API v3  发送。

注意:此集成仅在 Jira Cloud 实例上进行了测试。Jira Data Center(本地实例)可能可用,但不保证。

两个 API 具有相同的功能集。区别在于 V2 支持 Wiki Markup  来设置问题描述,而 V3 支持 Atlassian Document Format (ADF) 。默认的 jira.default.description 模板仅适用于 V2。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The URL to send API requests to. The full API path must be included.
# Example: https://company.atlassian.net/rest/api/2/
[ api_url: <string> | default = global.jira_api_url ]

# The API Type to use for search requests, can be either auto, cloud or datacenter
# Example: cloud
[ api_type: <string> | default = auto ]

# The project key where issues are created.
project: <string>

# Issue summary configuration.
[ summary:
    # Template for the issue summary.
    [ template: <tmpl_string> | default = '{{ template "jira.default.summary" . }}' ]

    # If set to false, the summary will not be updated when updating an existing issue.
    [ enable_update: <boolean> | default = true ]
]

# Issue description configuration.
[ description:
    # Template for the issue description.
    [ template: <tmpl_string> | default = '{{ template "jira.default.description" . }}' ]

    # If set to false, the description will not be updated when updating an existing issue.
    [ enable_update: <boolean> | default = true ]
]

# Labels to be added to the issue.
labels:
  [ - <tmpl_string> ... ]

# Priority of the issue.
[ priority: <tmpl_string> | default = '{{ template "jira.default.priority" . }}' ]

# Type of the issue (e.g. Bug).
[ issue_type: <string> ]

# Name of the workflow transition to resolve an issue. The target status must have the category "done".
# NOTE: The name of the transition can be localized and depends on the language setting of the service account.
[ resolve_transition: <string> ]

# Name of the workflow transition to reopen an issue. The target status should not have the category "done".
# NOTE: The name of the transition can be localized and depends on the language setting of the service account.
[ reopen_transition: <string> ]

# If reopen_transition is defined, ignore issues with that resolution.
[ wont_fix_resolution: <string> ]

# If reopen_transition is defined, reopen the issue when it is not older than this value (rounded down to the nearest minute).
# The resolutiondate field is used to determine the age of the issue.
[ reopen_duration: <duration> ]

# Other issue and custom fields.
fields:
  [ <string>: <jira_field> ... ]


# The HTTP client's configuration. You must use this configuration to supply the personal access token (PAT) as part of the HTTP `Authorization` header.
# For Jira Cloud, use basic_auth with the email address as the username and the PAT as the password.
# For Jira Data Center, use the 'authorization' field with 'credentials: <PAT value>'.
[ http_config: <http_config> | default = global.http_config ]

labels 字段是添加到问题中的标签列表。支持模板表达式。例如:

labels:
  - 'alertmanager'
  - '{{ .CommonLabels.severity }}'

<jira_field>

Jira 问题字段可以有多种类型。根据字段类型,必须以不同方式提供值。有关更多示例,请参阅 https://developer.atlassian.com/server/jira/platform/jira-rest-api-examples/#setting-custom-field-data-for-other-field-types 

fields:
    # Components
    components: { name: "Monitoring" }
    # Custom Field TextField
    customfield_10001: "Random text"
    # Custom Field SelectList
    customfield_10002: {"value": "red"}
    # Custom Field MultiSelect
    customfield_10003: [{"value": "red"}, {"value": "blue"}, {"value": "green"}]

<opsgenie_config>

OpsGenie 通知通过 OpsGenie API  发送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The API key to use when talking to the OpsGenie API.
[ api_key: <secret> | default = global.opsgenie_api_key ]

# The filepath to API key to use when talking to the OpsGenie API. Conflicts with api_key.
[ api_key_file: <filepath> | default = global.opsgenie_api_key_file ]

# The host to send OpsGenie API requests to.
[ api_url: <string> | default = global.opsgenie_api_url ]

# Alert text limited to 130 characters.
[ message: <tmpl_string> | default = '{{ template "opsgenie.default.message" . }}' ]

# A description of the alert.
[ description: <tmpl_string> | default = '{{ template "opsgenie.default.description" . }}' ]

# A backlink to the sender of the notification.
[ source: <tmpl_string> | default = '{{ template "opsgenie.default.source" . }}' ]

# A set of arbitrary key/value pairs that provide further detail
# about the alert.
# All common labels are included as details by default.
[ details: { <string>: <tmpl_string>, ... } ]

# List of responders responsible for notifications.
responders:
  [ - <responder> ... ]

# Comma separated list of tags attached to the notifications.
[ tags: <tmpl_string> ]

# Additional alert note.
[ note: <tmpl_string> ]

# Priority level of alert. Possible values are P1, P2, P3, P4, and P5.
[ priority: <tmpl_string> ]

# Whether to update message and description of the alert in OpsGenie if it already exists
# By default, the alert is never updated in OpsGenie, the new message only appears in activity log.
[ update_alerts: <boolean> | default = false ]

# Optional field that can be used to specify which domain alert is related to.
[ entity: <tmpl_string> ]

# Comma separated list of actions that will be available for the alert.
[ actions: <tmpl_string> ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<responder>

# Exactly one of these fields should be defined.
[ id: <tmpl_string> ]
[ name: <tmpl_string> ]
[ username: <tmpl_string> ]

# One of `team`, `teams`, `user`, `escalation` or `schedule`.
#
# The `teams` responder is configured using the `name` field above.
# This field can contain a comma-separated list of team names.
# If the list is empty, no responders are configured.
type: <tmpl_string>

<pagerduty_config>

PagerDuty 通知通过 PagerDuty API  发送。PagerDuty 提供了 集成文档 。Alertmanager 的 v0.11 及更高版本对 PagerDuty 的 Events API v2 的支持存在重要差异。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The routing and service keys are mutually exclusive.
# The PagerDuty integration key (when using PagerDuty integration type `Events API v2`).
# It is mutually exclusive with `routing_key_file`.
routing_key: <tmpl_secret>
# Read the Pager Duty routing key from a file.
# It is mutually exclusive with `routing_key`.
routing_key_file: <filepath>
# The PagerDuty integration key (when using PagerDuty integration type `Prometheus`).
# It is mutually exclusive with `service_key_file`.
service_key: <tmpl_secret>
# Read the Pager Duty service key from a file.
# It is mutually exclusive with `service_key`.
service_key_file: <filepath>

# The URL to send API requests to
[ url: <string> | default = global.pagerduty_url ]

# The client identification of the Alertmanager.
[ client:  <tmpl_string> | default = '{{ template "pagerduty.default.client" . }}' ]
# A backlink to the sender of the notification.
[ client_url:  <tmpl_string> | default = '{{ template "pagerduty.default.clientURL" . }}' ]

# A description of the incident.
[ description: <tmpl_string> | default = '{{ template "pagerduty.default.description" .}}' ]

# Severity of the incident.
[ severity: <tmpl_string> | default = 'error' ]

# Unique location of the affected system.
[ source: <tmpl_string> | default = client ]

# A set of arbitrary key/value pairs that provide further detail about the incident.
# Nested key/value pairs are accepted when using PagerDuty integration type `Events API v2`.
[ details: { <string>: <tmpl_string>, ... } | default = {
  firing:       '{{ .Alerts.Firing | toJSON }}'
  resolved:     '{{ .Alerts.Resolved | toJSON }}'
  num_firing:   '{{ .Alerts.Firing | len }}'
  num_resolved: '{{ .Alerts.Resolved | len }}'
} ]

# Images to attach to the incident.
images:
  [ <image_config> ... ]

# Links to attach to the incident.
links:
  [ <link_config> ... ]

# The part or component of the affected system that is broken.
[ component: <tmpl_string> ]

# A cluster or grouping of sources.
[ group: <tmpl_string> ]

# The class/type of the event.
[ class: <tmpl_string> ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

# The maximum time to wait for a pagerduty request to complete, before failing the
# request and allowing it to be retried. The default value of 0s indicates that
# no timeout should be applied.
# NOTE: This will have no effect if set higher than the group_interval.
[ timeout: <duration> | default = 0s ]

<image_config>

字段在 PagerDuty API 文档  中进行了记录。

href: <tmpl_string>
src: <tmpl_string>
alt: <tmpl_string>

字段在 PagerDuty API 文档  中进行了记录。

href: <tmpl_string>
text: <tmpl_string>

<pushover_config>

Pushover 通知通过 Pushover API  发送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The recipient user's key.
# user_key and user_key_file are mutually exclusive.
user_key: <secret>
user_key_file: <filepath>

# Your registered application's API token, see https://pushover.net/apps
# You can also register a token by cloning this Prometheus app:
# https://pushover.net/apps/clone/prometheus
# token and token_file are mutually exclusive.
token: <secret>
token_file: <filepath>

# Notification title.
[ title: <tmpl_string> | default = '{{ template "pushover.default.title" . }}' ]

# Notification message.
[ message: <tmpl_string> | default = '{{ template "pushover.default.message" . }}' ]

# A supplementary URL shown alongside the message.
[ url: <tmpl_string> | default = '{{ template "pushover.default.url" . }}' ]

# Optional device to send notification to, see https://pushover.net/api#device
[ device: <string> ]

# Optional sound to use for notification, see https://pushover.net/api#sound
[ sound: <string> ]

# Priority, see https://pushover.net/api#priority
[ priority: <tmpl_string> | default = '{{ if eq .Status "firing" }}2{{ else }}0{{ end }}' ]

# How often the Pushover servers will send the same notification to the user.
# Must be at least 30 seconds.
[ retry: <duration> | default = 1m ]

# How long your notification will continue to be retried for, unless the user
# acknowledges the notification.
[ expire: <duration> | default = 1h ]

# Optional time to live (TTL) to use for notification, see https://pushover.net/api#ttl
[ ttl: <duration> ]

# Optional HTML/monospace formatting for the message, see https://pushover.net/api#html
# html and monospace formatting are mutually exclusive.
[ html: <boolean> | default = false ]
[ monospace: <boolean> | default = false ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<rocketchat_config>

Rocketchat 通知通过 Rocketchat REST API  发送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]
[ api_url: <string> | default = global.rocketchat_api_url ]
[ channel: <tmpl_string> | default = global.rocketchat_api_url ]

# The sender token and token_id
# See https://docs.rocket.chat/use-rocket.chat/user-guides/user-panel/my-account#personal-access-tokens
# token and token_file are mutually exclusive.
# token_id and token_id_file are mutually exclusive.
token: <secret>
token_file: <filepath>
token_id: <secret>
token_id_file: <filepath>


[ color: <tmpl_string | default '{{ if eq .Status "firing" }}red{{ else }}green{{ end }}' ]
[ emoji <tmpl_string | default = '{{ template "rocketchat.default.emoji" . }}'
[ icon_url <tmpl_string | default = '{{ template "rocketchat.default.iconurl" . }}'
[ text <tmpl_string | default = '{{ template "rocketchat.default.text" . }}'
[ title <tmpl_string | default = '{{ template "rocketchat.default.title" . }}'
[ title_link <tmpl_string | default = '{{ template "rocketchat.default.titlelink" . }}'
fields:
  [ <rocketchat_field_config> ... ]
[ image_url <tmpl_string> ]
[ thumb_url <tmpl_string> ]
[ link_names <tmpl_string> ]
[ short_fields: <boolean> | default = false ]
actions:
  [ <rocketchat_action_config> ... ]

<rocketchat_field_config>

字段在 Rocketchat API 文档  中进行了记录。

[ title: <tmpl_string> ]
[ value: <tmpl_string> ]
[ short: <boolean> | default = rocketchat_config.short_fields ]

<rocketchat_action_config>

字段在 Rocketchat API api models  中进行了记录。

[ type: <tmpl_string> | ignored, only "button" is supported ]
[ text: <tmpl_string> ]
[ url: <tmpl_string> ]
[ msg: <tmpl_string> ]

<slack_config>

Slack 通知可以通过 Incoming webhooks Bot tokens  发送。

如果使用 incoming webhook,则 api_url 必须设置为 incoming webhook 的 URL,或写入 api_url_file 引用的文件。

如果使用 Bot Token,则 api_url 必须设置为 https://slack.com/api/chat.postMessage,bot token 必须在 http_config 中设置为授权凭证,并且 channel 必须包含要发送通知的频道的名称或频道 ID。如果使用频道名称,则 '#' 是可选的。

通知包含一个 附件 

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]

# The Slack webhook URL. Either api_url or api_url_file should be set.
# Defaults to global settings if none are set here.
[ api_url: <secret> | default = global.slack_api_url ]
[ api_url_file: <filepath> | default = global.slack_api_url_file ]

# The channel or user to send notifications to.
channel: <tmpl_string>

# API request data as defined by the Slack webhook API.
[ icon_emoji: <tmpl_string> ]
[ icon_url: <tmpl_string> ]
[ link_names: <boolean> | default = false ]
[ username: <tmpl_string> | default = '{{ template "slack.default.username" . }}' ]
# The following parameters define the attachment.
actions:
  [ <action_config> ... ]
[ callback_id: <tmpl_string> | default = '{{ template "slack.default.callbackid" . }}' ]
[ color: <tmpl_string> | default = '{{ if eq .Status "firing" }}danger{{ else }}good{{ end }}' ]
[ fallback: <tmpl_string> | default = '{{ template "slack.default.fallback" . }}' ]
fields:
  [ <field_config> ... ]
[ footer: <tmpl_string> | default = '{{ template "slack.default.footer" . }}' ]
[ mrkdwn_in: '[' <string>, ... ']' | default = ["fallback", "pretext", "text"] ]
[ pretext: <tmpl_string> | default = '{{ template "slack.default.pretext" . }}' ]
[ short_fields: <boolean> | default = false ]
[ text: <tmpl_string> | default = '{{ template "slack.default.text" . }}' ]
[ title: <tmpl_string> | default = '{{ template "slack.default.title" . }}' ]
[ title_link: <tmpl_string> | default = '{{ template "slack.default.titlelink" . }}' ]
[ image_url: <tmpl_string> ]
[ thumb_url: <tmpl_string> ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

# The maximum time to wait for a slack request to complete, before failing the
# request and allowing it to be retried. The default value of 0s indicates that
# no timeout should be applied.
# NOTE: This will have no effect if set higher than the group_interval.
[ timeout: <duration> | default = 0s ]

<action_config>

字段记录在 Slack API 文档的 消息附件 交互式消息  中。

text: <tmpl_string>
type: <tmpl_string>
# Either url or name and value are mandatory.
[ url: <tmpl_string> ]
[ name: <tmpl_string> ]
[ value: <tmpl_string> ]

[ confirm: <action_confirm_field_config> ]
[ style: <tmpl_string> | default = '' ]
<action_confirm_field_config>

字段记录在 Slack API 文档  中。

text: <tmpl_string>
[ dismiss_text: <tmpl_string> | default '' ]
[ ok_text: <tmpl_string> | default '' ]
[ title: <tmpl_string> | default '' ]

<field_config>

字段记录在 Slack API 文档  中。

title: <tmpl_string>
value: <tmpl_string>
[ short: <boolean> | default = slack_config.short_fields ]

<sns_config>

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The SNS API URL i.e. https://sns.us-east-2.amazonaws.com.
#  If not specified, the SNS API URL from the SNS SDK will be used.
[ api_url: <tmpl_string> ]

# Configures AWS's Signature Verification 4 signing process to sign requests.
sigv4:
  [ <sigv4_config> ]

# SNS topic ARN, i.e. arn:aws:sns:us-east-2:698519295917:My-Topic
# If you don't specify this value, you must specify a value for the phone_number or target_arn.
# If you are using a FIFO SNS topic you should set a message group interval longer than 5 minutes
# to prevent messages with the same group key being deduplicated by the SNS default deduplication window
[ topic_arn: <tmpl_string> ]

# Subject line when the message is delivered to email endpoints.
[ subject: <tmpl_string> | default = '{{ template "sns.default.subject" .}}' ]

# Phone number if message is delivered via SMS in E.164 format.
# If you don't specify this value, you must specify a value for the topic_arn or target_arn.
[ phone_number: <tmpl_string> ]

# The  mobile platform endpoint ARN if message is delivered via mobile notifications.
# If you don't specify this value, you must specify a value for the topic_arn or phone_number.
[ target_arn: <tmpl_string> ]

# The message content of the SNS notification.
[ message: <tmpl_string> | default = '{{ template "sns.default.message" .}}' ]

# SNS message attributes.
attributes:
  [ <string>: <string> ... ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<sigv4_config>

# The AWS region. If blank, the region from the default credentials chain is used.
[ region: <string> ]

# The AWS API keys. Both access_key and secret_key must be supplied or both must be blank.
# If blank the environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are used.
[ access_key: <string> ]
[ secret_key: <secret> ]

# Named AWS profile used to authenticate.
[ profile: <string> ]

# AWS Role ARN, an alternative to using AWS API keys.
[ role_arn: <string> ]

<telegram_config>

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The Telegram API URL i.e. https://api.telegram.org.
# If not specified, default API URL will be used.
[ api_url: <string> | default = global.telegram_api_url ]

# Telegram bot token. It is mutually exclusive with `bot_token_file`.
[ bot_token: <secret> ]

# Read the Telegram bot token from a file. It is mutually exclusive with `bot_token`.
[ bot_token_file: <filepath> ]

# ID of the chat where to send the messages.
[ chat_id: <int> ]

# Optional ID of the message thread where to send the messages.
[ message_thread_id: <int> ]

# Message template.
[ message: <tmpl_string> default = '{{ template "telegram.default.message" .}}' ]

# Disable telegram notifications
[ disable_notifications: <boolean> | default = false ]

# Parse mode for telegram message, supported values are MarkdownV2, Markdown, HTML and empty string for plain text.
[ parse_mode: <string> | default = "HTML" ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<victorops_config>

VictorOps 通知通过 VictorOps API  发送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The API key to use when talking to the VictorOps API.
# It is mutually exclusive with `api_key_file`.
[ api_key: <secret> | default = global.victorops_api_key ]

# Reads the API key to use when talking to the VictorOps API from a file.
# It is mutually exclusive with `api_key`.
[ api_key_file: <filepath> | default = global.victorops_api_key_file ]

# The VictorOps API URL.
[ api_url: <string> | default = global.victorops_api_url ]

# A key used to map the alert to a team.
routing_key: <tmpl_string>

# Describes the behavior of the alert (CRITICAL, WARNING, INFO).
[ message_type: <tmpl_string> | default = 'CRITICAL' ]

# Contains summary of the alerted problem.
[ entity_display_name: <tmpl_string> | default = '{{ template "victorops.default.entity_display_name" . }}' ]

# Contains long explanation of the alerted problem.
[ state_message: <tmpl_string> | default = '{{ template "victorops.default.state_message" . }}' ]

# The monitoring tool the state message is from.
[ monitoring_tool: <tmpl_string> | default = '{{ template "victorops.default.monitoring_tool" . }}' ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

<webhook_config>

Webhook 接收器允许配置一个通用接收器。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The endpoint to send HTTP POST requests to.
# url and url_file are mutually exclusive.
url: <secret>
url_file: <filepath>

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

# The maximum number of alerts to include in a single webhook message. Alerts
# above this threshold are truncated. When leaving this at its default value of
# 0, all alerts are included.
[ max_alerts: <int> | default = 0 ]

# The maximum time to wait for a webhook request to complete, before failing the
# request and allowing it to be retried. The default value of 0s indicates that
# no timeout should be applied.
# NOTE: This will have no effect if set higher than the group_interval.
[ timeout: <duration> | default = 0s ]

Alertmanager 将以以下 JSON 格式向配置的端点发送 HTTP POST 请求。

{
  "version": "4",
  "groupKey": <string>,              // key identifying the group of alerts (e.g. to deduplicate)
  "truncatedAlerts": <int>,          // how many alerts have been truncated due to "max_alerts"
  "status": "<resolved|firing>",
  "receiver": <string>,
  "groupLabels": <object>,
  "commonLabels": <object>,
  "commonAnnotations": <object>,
  "externalURL": <string>,           // backlink to the Alertmanager.
  "alerts": [
    {
      "status": "<resolved|firing>",
      "labels": <object>,
      "annotations": <object>,
      "startsAt": "<rfc3339>",
      "endsAt": "<rfc3339>",
      "generatorURL": <string>,      // identifies the entity that caused the alert
      "fingerprint": <string>        // fingerprint to identify the alert
    },
    ...
  ]
}

此功能有一系列 集成

<incidentio_config>

incident.io 通知通过 incident.io Alert Sources API  发送。

配置此集成时,可以通过 http_config 来设置 authorization,或者使用 alert_source_tokenalert_source_token_filealert_source_tokenalert_source_token_file 的配置会覆盖 http_config

请注意,如果负载超过 incident.io 的 API 限制(512KB),集成将自动截断除第一个之外的所有警报。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The HTTP client's configuration.
[ http_config: <http_config> | default = global.http_config ]

# The URL to send the incident.io alert. This would typically be provided by the
# incident.io team when setting up an alert source.
# URL and URL_file are mutually exclusive.
url: <string>
url_file: <filepath>

# The alert source token is used to authenticate with incident.io.
# alert_source_token and alert_source_token_file are mutually exclusive.
[ alert_source_token: <secret> ]
[ alert_source_token_file: <filepath> ]

# The maximum number of alerts to be sent per incident.io message.
# Alerts exceeding this threshold will be truncated. Setting this to 0
# allows an unlimited number of alerts. Note that if the payload exceeds
# incident.io's size limits (512KB), the notifier will automatically drop
# all alerts except the first one. If the payload is still too
# large after this truncation, you will receive a 429 response and alerts
# will not be ingested.
[ max_alerts: <int> | default = 0 ]

# Timeout is the maximum time allowed to invoke incident.io. Setting this to 0
# does not impose a timeout.
[ timeout: <duration> | default = 0s ]

<wechat_config>

微信通知通过 微信 API  发送。

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = false ]

# The API key to use when talking to the WeChat API.
[ api_secret: <secret> | default = global.wechat_api_secret ]

# The WeChat API URL.
[ api_url: <string> | default = global.wechat_api_url ]

# The corp id for authentication.
[ corp_id: <string> | default = global.wechat_api_corp_id ]

# API request data as defined by the WeChat API.
[ message: <tmpl_string> | default = '{{ template "wechat.default.message" . }}' ]
# Type of the message type, supported values are `text` and `markdown`.
[ message_type: <string> | default = 'text' ]
[ agent_id: <string> | default = '{{ template "wechat.default.agent_id" . }}' ]
[ to_user: <string> | default = '{{ template "wechat.default.to_user" . }}' ]
[ to_party: <string> | default = '{{ template "wechat.default.to_party" . }}' ]
[ to_tag: <string> | default = '{{ template "wechat.default.to_tag" . }}' ]

<webex_config>

# Whether to notify about resolved alerts.
[ send_resolved: <boolean> | default = true ]

# The Webex Teams API URL i.e. https://webexapis.com/v1/messages
# If not specified, default API URL will be used.
[ api_url: <string> | default = global.webex_api_url ]

# ID of the Webex Teams room where to send the messages.
room_id: <tmpl_string>

# Message template.
[ message: <tmpl_string> default = '{{ template "webex.default.message" .}}' ]

# The HTTP client's configuration. You must use this configuration to supply the bot token as part of the HTTP `Authorization` header.
[ http_config: <http_config> | default = global.http_config ]

追踪配置

<tracing_config>

# The tracing client type, supported values are `http` and `grpc`.
[ client_type: <tracing_client_type> | default = "grpc" ]

# The tracing endpoint.
[ endpoint: <string> | default = "" ]

# The sampling fraction.
[ sampling_fraction: <float> | default = 0.0 ]

# Whether to disable TLS.
[ insecure: <boolean> | default = false ]

# The HTTP client's configuration.
[ tls_config: <tls_config> ]

# Custom HTTP headers.
[ http_headers:
  [ <http_header> ] ]

# The tracing compression.
[ compression: <string> | default = "gzip" ]

# The tracing timeout.
[ timeout: <duration> | default = 0s ]

本页内容