K
Q

K8S - Docker - ARM unbutu 64bits - qemu: uncaught target signal 11 (Segmentation fault) - core dumped

7/20/2020

I installed a K8S cluster on my 4 raspberry pi 4 with 2gb of RAM and 32gb SD cards.

On my master and slave nodes I installed qemu binfmt-support qemu-user-static in order to enable the support of ARM CPU for the docker images.

I tried to install Grafana and Prometheus in order to monitor the cluster by doing:

kubectl apply --filename https://raw.githubusercontent.com/giantswarm/prometheus/master/manifests-all.yaml

But the pods prometheus-node-exporter failed with the status CrashLoopBackOff

When I check the logs kubectl logs prometheus-node-exporter-dn8t9 -n monitoring I got the error qemu: uncaught target signal 11 (Segmentation fault) - core dumped

And I do not know how to solve this issue or where to start looking.

Is there anyone that could help with that?

-- Gabi
arm
docker
kubernetes
raspberry-pi

Similar Questions

AWS and kubernetes volumes - how to share configuration across multiple Pods?
Azure terraform deployment issue - unknown addon_profile
Running mongodb stateful set on Kubernetes with istio
Transfer file from kubernetes cluster to other ec2 machine
kubectl get nodes shows NotReady

1 Answer

7/20/2020

I looked at this 👀 and see 👀 since you are running on a raspberry pi 4, your architecture is probably aarch64 (arm64). So, it seems like the node-exporter DaemonSet K8s manifest is pulling the following image: prom/node-exporter:v0.14.0 and I looked at dockerhub and that image tag doesn't have the aarch64 architecture tag, so it's most likely pulling the amd64 version causing qemu on the node to crash in your case.

You can also see that there's an arm64 image starting with prom/node-exporter:v0.18.0. So, you can try downloading the file editing the node-exporter container to use v0.18.0 and that should fix the issue with that container. You may also need to update other containers that have matching arm64 architecture.

🏃‍♀️🏃‍♂️🏃

$ wget https://raw.githubusercontent.com/giantswarm/prometheus/master/manifests-all.yaml

Then change:

...
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: prometheus-node-exporter
  namespace: monitoring
  labels:
    app: prometheus
    component: node-exporter
spec:
  selector:
    matchLabels:
      app: prometheus
  template:
    metadata:
      name: prometheus-node-exporter
      labels:
        app: prometheus
        component: node-exporter
    spec:
      containers:
      - image: prom/node-exporter:v0.18.0 👈 Change here
        name: prometheus-node-exporter
        ports:
        - name: prom-node-exp
          #^ must be an IANA_SVC_NAME (at most 15 characters, ..)
          containerPort: 9100
          hostPort: 9100
      hostNetwork: true
      hostPID: true
...

✌️

-- Rico
Source: StackOverflow