We can modify grafana k8s deployment manifest by volumeMounts to feed grafana.ini on our host to grafana running in a pod.

First, prepare your grafana.ini with SMTP settings. E.g.

[smtp]
enabled = true
host = smtp.gmail.com:465
# Please change user and password to your ones.
user = foo@bar.com
password = your-password

Then, you can place this file on your host. E.g. /home/mydir/grafana.ini

Modify the loaded grafana k8s deployment manifest:

kubectl edit deployments.apps -n monitoring grafana

Add a new mount to volumeMounts (not the one in kubectl.kubernetes.io/last-applied-configuration):

        volumeMounts:
        - mountPath: /etc/grafana/grafana.ini
          name: mydir
          subPath: grafana.ini

Add a new hostPath to volumes:

      volumes:
      - hostPath:
          path: /home/mydir
          type: ""
        name: mydir

Finally, restart the deployment:

kubectl rollout restart -n monitoring deployment grafana

Run this command and use a web browser on your host to navigate to http://localhost:8080 to grafana web app:

kubectl port-forward -n monitoring svc/grafana 8080:3000

Then, you can navigate to Alerting / Notification channels / Add channel to add an Email notification channel and test it!