Kubernetes ServiceAccount JWT Auth
Among the auth mechanisms supported by Grafana, [auth.jwt] stands out as it is uniquely compatible with Kubernetes!
By using this method, it is possible to use Kubernetes ServiceAccount tokens (JWTs) to authenticate with Grafana.
Since version v5.21.0 of the Grafana-Operator, we support authentication using the projected Kubernetes ServiceAccount JWT when [auth.jwt] is configured.
Configuration
Enable JWT auth for a Grafana instance with .spec.client.useKubeAuth=true and configure Grafana to trust JWTs issued by Kubernetes:
apiVersion: grafana.integreatly.org/v1beta1
kind: Grafana
metadata:
name: jwt-grafana-ca
labels:
dashboards: "grafana"
spec:
# version: 12.2.0 # or newer
client:
useKubeAuth: true # <- enables authentication using the kubernetes service account
disableDefaultAdminSecret: true # Prevents the creation of the secret containing admin credentials
config:
auth.jwt:
enabled: "true"
header_name: Authorization
expect_claims: '{"aud": ["operator.grafana.com"]}' # Optional security: Rejects default ServiceAccount tokens
username_claim: sub
email_claim: sub
auto_sign_up: "true"
role_attribute_strict: "true" # Disables auto_assign_org_role
# Assigns the Admin role to the operator service account, replace 'default' with the namespace of the operator
role_attribute_path: "contains(sub, 'system:serviceaccount:default:grafana-operator') && 'GrafanaAdmin' || 'None'"
jwk_set_url: https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT_HTTPS}/openid/v1/jwks
jwk_set_bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
tls_client_ca: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
# Optional: Disable basic auth and server admin creation, disables local accounts using username and passwords
# auth.basic:
# enabled: "false" # Disables BASIC auth in Grafana
# security:
# disable_initial_admin_creation: "true" # Disables the creation of the default admin user in Grafana
Note
The example assumes the Grafana operator is installed in the default namespace and using the grafana-operator ServiceAccount.
Both being the default when installing via the official Helm chart.
Remember to update the role_attribute_path accordingly.
role_attribute_path determines the assigned role by the claims in the JWT body:
The example assigns Admin, or GrafanaAdmin if allow_assign_grafana_admin: "true", to the grafana-operator ServiceAccount in the default namespace.
If you require more flexibility, you can customize the role_attribute_path.
By default, you have the following claims available from the kubernetes service account token:
{
"aud": [
"operator.grafana.com"
],
"exp": 1754331475,
"iat": 1754327875,
"iss": "https://kubernetes.default.svc.cluster.local",
"jti": "924ed6e1-e3e1-4d5d-8ef3-5fa9531af0fe",
"kubernetes.io": {
"namespace": "default",
"node": {
"name": "kind-grafana-control-plane",
"uid": "4a792812-ae7b-44c4-88d8-ec68bd928f3e"
},
"pod": {
"name": "grafana-operator-5b77b77989-rdd9p",
"uid": "1130fdf8-347e-4b2a-9343-dd53932d8a91"
},
"serviceaccount": {
"name": "grafana-operator",
"uid": "86d5de01-31da-4b97-9228-00b2ff0fcc7c"
}
},
"nbf": 1754327875,
"sub": "system:serviceaccount:default:grafana-operator"
}
The below configuration will assign GrafanaAdmin to the main ServiceAccount, but Editor to any Service account in the grafana namespace.
This can be used when other workloads in the cluster need access to Grafana through the same JWT authentication but with less permissions.
role_attribute_path: "contains(sub, 'system:serviceaccount:default:grafana-operator') && 'GrafanaAdmin' || contains(\"kubernetes.io\".namespace, 'grafana') && 'Editor' || 'None'"
If you intend to use ServiceAccount tokens with the default audience (aud) claim, remember to remove the expect_claims config from the examples.
Grafana versions prior to 12.2.0
Older versions of Grafana cannot authenticate with a JWKS endpoint, which is necessary to retrieve the JWKSet from Kubernetes.
Users instead need to mount in the JWKSet as a file from either a ConfigMap or Secret.
kubectl create configmap kube-root-jwks --from-literal=jwks.json="$(kubectl get --raw /openid/v1/jwks)"
apiVersion: grafana.integreatly.org/v1beta1
kind: Grafana
metadata:
name: older-versions
labels:
dashboards: "grafana"
spec:
version: 12.1.3 # Relevant for 12.1.X or lower as jwk_set_bearer_token_file and tls_client_ca don't exist
client:
useKubeAuth: true # <- enables authentication using the kubernetes service account
disableDefaultAdminSecret: true # Prevents the creation of the secret containing admin credentials
config:
auth.jwt:
enabled: "true"
header_name: Authorization
expect_claims: '{"aud": ["operator.grafana.com"]}' # Optional security: Rejects default ServiceAccount tokens
username_claim: sub
email_claim: sub
auto_sign_up: "true"
jwk_set_file: /var/run/kube-root-jwks.json
role_attribute_strict: "true" # Disables auto_assign_org_role
# Assigns the Admin role to the operator service account, replace 'default' with the namespace of the operator
role_attribute_path: "contains(sub, 'system:serviceaccount:default:grafana-operator') && 'GrafanaAdmin' || 'None'"
# Optional: Disable basic auth and server admin creation, disables local accounts using username and passwords
# auth.basic:
# enabled: "false" # Disables BASIC auth in Grafana
# security:
# disable_initial_admin_creation: "true" # Disables the creation of the default admin user in Grafana
deployment:
spec:
template:
spec:
containers:
- name: grafana
volumeMounts:
- mountPath: /var/run/kube-root-jwks.json
name: cluster-jwks
readOnly: true
subPath: kube-root-jwks.json
volumes:
- name: cluster-jwks
configMap:
name: kube-root-jwks
defaultMode: 0444
items:
- key: jwks.json
path: kube-root-jwks.json
Issuing Tokens for ServiceAccounts
Warning
Always be vary of tokens being leaked, especially when using long-lived Kubernetes ServiceAccount tokens for other purposes than authenticating with the Kubernetes API.Tokens can be issued for a service account ad hoc with kubectl.
This could be used for testing or just an easy way to create short lived JWTs for a ServiceAccount with access to Grafana
# Create serviceaccount JWT and store it in ./token
kubectl create token -n grafana grafana-operator-controller-manager --audience 'operator.grafana.com' --duration 1h >token
# Expose a port
kubectl port-forward svc/jwt-grafana-ca-service 3000:3000 &>/dev/null &
# curl the instance using the token
curl 'http://127.0.0.1:3000/api/folders' -H "Authorization: Bearer $(cat token)"
# An array, even empty `[]`, is a successful response!
Custom token audience
If the default token at /var/run/secrets/kubernetes.io/serviceaccount/token is leaked, whatever permissions assigned to the ServiceAccount can be abused by whoever obtains it.
To prevent this, it’s highly recommended to create tokens with custom audience claims ("aud": ["..."]) that invalidates the token from being used with the Kubernets API.
By default, the grafana-operator ServiceAccount can create, Update, and Delete various resources on the cluster level.
To avoid accidental exposure of this service account, the operator mounts a second token at /var/run/secrets/grafana.com/serviceaccount/token that cannot be used with the Kubernetes API.
This is the token used for JWT authentication.
# The majority of the manifest is omitted for brevity
apiVersion: apps/v1
kind: Deployment
metadata:
name: grafana-operator
spec:
replicas: 1
template:
spec:
containers:
- name: automation
image: ghcr.io/grafana/grafana-operator:v5.21.0
imagePullPolicy: IfNotPresent
volumeMounts: # Where to mount the serviceaccount
- name: kubeauth-token-volume
mountPath: /var/run/secrets/grafana.com/serviceaccount
readOnly: true
volumes:
- name: kubeauth-token-volume
projected:
sources:
- serviceAccountToken:
audience: operator.grafana.com # Ensures the token cannot authenticate with the Kubernetes API
path: token