Notification Policies

Notification policies provide you with a flexible way of designing how to handle notifications and minimize alert noise. For a complete explanation on notification policies, see the upstream Grafana documentation.

Simple Notification Policy

The following snippet shows an example notification policy routing to the operations or security team based on the team label.

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaNotificationPolicy
metadata:
  name: grafananotificationpolicy-sample
spec:
  instanceSelector:
    matchLabels:
      dashboards: "grafana"
  route:
    receiver: grafana-email-default
    group_by:
      - grafana_folder
      - alertname
    routes:
      - receiver: grafana-email-operations
        object_matchers:
          - - team
            - =
            - operations
      - receiver: grafana-email-security
        object_matchers:
          - - team
            - =
            - security

Dynamic Notification Policy Routes

There might be scenarios where you can not define the entire notification policy in a single place and you have to assemble it from multiple resouces. In this case, you can use the routeSelector field in combination with multiple GrafanaNotificationPolicyRoute resources.

Both GrafanaNotificationPolicy and GrafanaNotificationPolicyRoute objects support the routeSelector field.

All GrafanaNotificationPolicyRoute resources will then be discovered based on the label selector defined in spec.route.routeSelector. In case spec.allowCrossNamespaceImport is enabled, matching routes will be fetched from all namespaces. Otherwise only routes from the same namespace as the GrafanaNotificationPolicy will be discovered.

The discovered routes are then used when applying the notification policy.

The following shows an example of how dynamic routes will get merged.

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaNotificationPolicy
metadata:
  name: grafananotificationpolicy-sample
spec:
  allowCrossNamespaceImport: true
  instanceSelector:
    matchLabels:
      dashboards: "grafana"
  route:
    receiver: grafana-default-email
    group_by:
      - grafana_folder
      - alertname
    routes:
      - receiver: grafana-default-email
        object_matchers:
          - - team
            - =
            - a
          - - inline
            - =
            - first
        routeSelector:
          matchLabels:
            team-a: "child"
      - receiver: grafana-default-email
        object_matchers:
          - - team
            - =
            - b
          - - inline
            - =
            - second
---
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaNotificationPolicyRoute
metadata:
  name: dynamic-c
  namespace: grafana-crds
  labels:
    team-a: "child"
spec:
  receiver: grafana-default-email
  object_matchers:
    - - crossNamespace
      - =
      - "true"
    - - dynamic
      - =
      - c
  routeSelector:
    matchLabels:
      team-c: "child"
---
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaNotificationPolicyRoute
metadata:
  name: dynamic-d
  labels:
    team-c: "child"
spec:
  receiver: grafana-default-email
  object_matchers:
    - - dynamic
      - =
      - d
---
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaNotificationPolicyRoute
metadata:
  name: dynamic-e
  labels:
    team-a: "child"
spec:
  receiver: grafana-default-email
  object_matchers:
    - - dynamic
      - =
      - e

The resulting Notification Policy will be the following:

Dynamic notification policy tree after applying the example routes

Last modified February 19, 2025: docs: update issue template (#1867) (dafc0e9)