I have a Prometheus server that runs on my cluster in Google Cloud which I have successfully used to collect metrics from my cluster and other sources into Grafana. Recently I tried to configure a scrap that will collect data from Aiven's Redis and Mysql. I used this config map:
apiVersion: v1
kind: ConfigMap
metadata:
name: prometheus-server
labels:
name: prometheus-server
namespace: my namespace
data:
prometheus.yml: |-
global:
scrape_interval: 5s
evaluation_interval: 5s
scrape_configs:
- job_name: aivenmetrics
scheme: https
basic_auth:
username: my user
password: my password
dns_sd_configs:
- names:
- redis-**********.aivencloud.com
- mysql-**********.aivencloud.com
type: A
port: 9273
tls_config:
insecure_skip_verify: true Here is my Prometheus deployment YAML.
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "3"
creationTimestamp: "2020-05-25T11:17:06Z"
generation: 3
labels:
app: prometheus
chart: prometheus-11.3.0
component: server
heritage: Tiller
release: prometheus
name: prometheus-server
namespace: my namespace
resourceVersion: "41496623"
selfLink: /apis/apps/v1/namespaces/integration/deployments/prometheus-server
uid: 7dabe25a-7a8f-499d-99aa-b332be4d9f5c
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: prometheus
component: server
release: prometheus
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: prometheus
chart: prometheus-11.3.0
component: server
heritage: Tiller
release: prometheus
spec:
containers:
- args:
- --volume-dir=/etc/config
- --webhook-url=http://127.0.0.1:9090/-/reload
image: jimmidyson/configmap-reload:v0.3.0
imagePullPolicy: IfNotPresent
name: prometheus-server-configmap-reload
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/config
name: config-volume
readOnly: true
- args:
- --storage.tsdb.retention.time=15d
- --config.file=/etc/config/prometheus.yml
- --storage.tsdb.path=/data
- --web.console.libraries=/etc/prometheus/console_libraries
- --web.console.templates=/etc/prometheus/consoles
- --web.enable-lifecycle
image: prom/prometheus:v2.18.1
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /-/healthy
port: 9090
scheme: HTTP
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
name: prometheus-server
ports:
- containerPort: 9090
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /-/ready
port: 9090
scheme: HTTP
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/config
name: config-volume
- mountPath: /data
name: storage-volume
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 65534
runAsGroup: 65534
runAsNonRoot: true
runAsUser: 65534
serviceAccount: prometheus-server
serviceAccountName: prometheus-server
terminationGracePeriodSeconds: 300
volumes:
- configMap:
defaultMode: 420
name: prometheus-server
name: config-volume
- name: storage-volume
persistentVolumeClaim:
claimName: prometheus-server
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2020-05-25T11:18:12Z"
lastUpdateTime: "2020-05-25T11:18:12Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
- lastTransitionTime: "2020-05-25T11:17:06Z"
lastUpdateTime: "2020-05-26T14:26:13Z"
message: ReplicaSet "prometheus-server-86688c7bb6" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
observedGeneration: 3
readyReplicas: 1
replicas: 1
updatedReplicas: 1My Redis dashboards aren't able to collect any data but the same Prometheus is able to provide data related to my cluster. Grafana works with other sources like Elastic for example. I have also run a container_scrap_error in query history using web access to Prometheus but there wasn't any error related to failed scrap.
Thank you