I have setup prometheus to monitor kubernetes cluster and also setup a job to probe service using blackbox-exporter.

- job_name: 'kubernetes-services'
  scheme: http
  metrics_path: /probe
  params:
    module: [http_2xx]
  kubernetes_sd_configs:
    - role: service

  relabel_configs:
  - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_probe]
    action: keep
    regex: true
  - source_labels: [__address__]
    target_label: __param_target
  - target_label: __address__
    replacement:  blackbox:9115
  - source_labels: [__param_target]
    target_label: instance
  - action: labelmap
    regex: __meta_kubernetes_service_label_(.+)
  - source_labels: [__meta_kubernetes_service_namespace]
    target_label: kubernetes_namespace
  - source_labels: [__meta_kubernetes_service_name]
    target_label: kubernetes_name

Added annotation to service file - prometheus.io/probe: "true"

So its showing status as UP

But its not showing status as DOWN when deployment(Pod) related to this service is down/has some error