The livenessProbe in conjunction with the readynessProbe are two parameters that you need to put in your own yaml definition as a container spec, the example files are only to exemplify it.

An example of livenessProbe for a grafana service colud be:

livenessProbe:
  failureThreshold: 10
    httpGet:
      path: /api/health
      port: 3000
      scheme: HTTP
    initialDelaySeconds: 60
    periodSeconds: 10
    successThreshold: 1
    timeoutSeconds: 30

Not all the parameters are needed you have to think about how to do it better for your services.

The "apply" command from the Kubernetes Rest client(kubectl) can have either a local or remote file. Liveness is proof of availability and readness is proof of pod readiness. From the moment the pod is created livenessProbe runs every 5 seconds as per periodSeconds configured, the "rm -rf / tmp / healthy" command runs after 30 seconds of sleep, so there will come a time that the proof of Life will not find the file and will generate an alert.

Hope this helps.

1) When I apply liveness-exec do I need mention any particular namespace for example kubectl apply -f https://k8s.io/examples/pods/probe/exec-liveness.yaml -n kube-system

If you specify a namespace in the apply command the pod will be created in that namespace, provided the namespace exist. If you don't specify a namespace (by command line or in the pod yaml definition) then the pod will be created in the default namespace

2) Can I use yaml file directly from k8s.io/example or I need to clone in my local

You can apply it direcly from k8s.io/example.

kubectl apply -f https://k8s.io/examples/pods/probe/exec-liveness.yaml

3) How these whole things works? What I understand so far is

a) When we deploy the pod, it will create a container and create /tmp/healthy file and then delete the container

b) It will run after 30 seconds and redo the same thing from step 3a

I think that the idea of the example is just to show how liveness probes work and how your pod will be restarted after the test fails, so the commands executed are a bit artificial. I think the error you see is simply the probe failing the check "cat /tmp/healthy"