Alerting

Search…
Alerting
Flagger can be configured to send alerts to various chat platforms. You can define a global alert provider at install time or configure alerts on a per canary basis.
Global configuration
Slack
Slack Configuration
Flagger requires a custom webhook integration from slack, instead of the new slack app system.
The webhook can be generated by following the legacy slack documentation​
Flagger configuration
Once the webhook has been generated. Flagger can be configured to send Slack notifications:
1
helm upgrade -i flagger flagger/flagger \
2
--set slack.url=https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK \
3
--set slack.proxy=my-http-proxy.com \ # optional http/s proxy
4
--set slack.channel=general \
5
--set slack.user=flagger \
6
--set clusterName=my-cluster
Copied!
Once configured with a Slack incoming webhook, Flagger will post messages when a canary deployment has been initialised, when a new revision has been detected and if the canary analysis failed or succeeded.
Slack Notifications
A canary deployment will be rolled back if the progress deadline exceeded or if the analysis reached the maximum number of failed checks:
Slack Notifications
Microsoft Teams
Flagger can be configured to send notifications to Microsoft Teams:
1
helm upgrade -i flagger flagger/flagger \
2
--set msteams.url=https://outlook.office.com/webhook/YOUR/TEAMS/WEBHOOK \
3
--set msteams.proxy-url=my-http-proxy.com # optional http/s proxy
Copied!
Similar to Slack, Flagger alerts on canary analysis events:
MS Teams Notifications
MS Teams Notifications
Canary configuration
Configuring alerting globally has several limitations as it's not possible to specify different channels or configure the verbosity on a per canary basis. To make the alerting move flexible, the canary analysis can be extended with a list of alerts that reference an alert provider. For each alert, users can configure the severity level. The alerts section overrides the global setting.
Slack example:
1
apiVersion: flagger.app/v1beta1
2
kind: AlertProvider
3
metadata:
4
  name: on-call
5
  namespace: flagger
6
spec:
7
  type: slack
8
  channel: on-call-alerts
9
  username: flagger
10
  # webhook address (ignored if secretRef is specified)
11
  address: https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK
12
  # optional http/s proxy
13
  proxy: http://my-http-proxy.com
14
  # secret containing the webhook address (optional)
15
  secretRef:
16
    name: on-call-url
17
---
18
apiVersion: v1
19
kind: Secret
20
metadata:
21
  name: on-call-url
22
  namespace: flagger
23
data:
24
  address: <encoded-url>
Copied!
The alert provider type can be: slack, msteams, rocket or discord. When set to discord, Flagger will use Slack formatting and will append /slack to the Discord address.
When not specified, channel defaults to general and username defaults to flagger.
When secretRef is specified, the Kubernetes secret must contain a data field named address, the address in the secret will take precedence over the address field in the provider spec.
The canary analysis can have a list of alerts, each alert referencing an alert provider:
1
  analysis:
2
    alerts:
3
      - name: "on-call Slack"
4
        severity: error
5
        providerRef:
6
          name: on-call
7
          namespace: flagger
8
      - name: "qa Discord"
9
        severity: warn
10
        providerRef:
11
          name: qa-discord
12
      - name: "dev MS Teams"
13
        severity: info
14
        providerRef:
15
          name: dev-msteams
Copied!
Alert fields:
name (required)
severity levels: info, warn, error (default info)
providerRef.name alert provider name (required)
providerRef.namespace alert provider namespace (defaults to the canary namespace)
When the severity is set to warn, Flagger will alert when waiting on manual confirmation or if the analysis fails. When the severity is set to error, Flagger will alert only if the canary analysis fails.
To differentiate alerts based on the cluster name, you can configure Flagger with the -cluster-name=my-cluster command flag, or with Helm --set clusterName=my-cluster.
Prometheus Alert Manager
You can use Alertmanager to trigger alerts when a canary deployment failed:
1
  - alert: canary_rollback
2
    expr: flagger_canary_status > 1
3
    for: 1m
4
    labels:
5
      severity: warning
6
    annotations:
7
      summary: "Canary failed"
8
      description: "Workload {{ $labels.name }} namespace {{ $labels.namespace }}"
Copied!
Last modified 5mo ago
Copy link
Contents
Prometheus Alert Manager