Istio not reporting https traces to Jaeger

10/5/2018

Describe the bug We have a container running with envoy sidecar proxy with service/deployment for port 443 using Istio's own example: sample/https/nginx. We can curl the container to get nginx page just fine but see absolutely no traces in Jaeger for https calls. We see HTTP calls in Jaeger as soon as we switch the port to 80 in deployment/service

Expected behavior We should see traces for both HTTP/HTTPS calls to the container.

Steps to reproduce the bug:

  • create nginx config:

    https://github.com/istio/istio/blob/master/samples/https/nginx-app.yaml
  • Create nginx deployment :

    kubectl apply -f <(istioctl kube-inject -f nginx.yaml)
  • curl -kv https://service-ip gives 200

  • no traces in Jaeger for HTTPS

Version

istioctl version
Version: 1.0.2
GitRevision: d639408fded355fb906ef2a1f9e8ffddc24c3d64
User: root@66ce69d4a51e
Hub: gcr.io/istio-release
GolangVersion: go1.10.1
BuildStatus: Clean

[root@ip-10-147-5-150 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.2", GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239", GitTreeState:"clean", BuildDate:"2018-08-07T23:17:28Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-28T20:13:43Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

Installation

- kubectl apply -f install/kubernetes/helm/helm-service-account.yaml
- helm init --service-account tiller
- helm install install/kubernetes/helm/istio --name istio --namespace istio-system
- helm upgrade --set tracing.enabled=true --set servicegraph.enabled=true --set global.configValidation=false istio install/kubernetes/helm/istio

Environment - Running this within AWS EKS

Cluster state - Attached archite.tar.gz

Edit 1

yaml for service - jaeger-query :

apiVersion: v1 kind: Service metadata: creationTimestamp: 2018-10-02T02:32:23Z labels: app: jaeger chart: tracing-1.0.1 heritage: Tiller jaeger-infra: jaeger-service release: istio name: jaeger-query namespace: istio-system resourceVersion: "5259733" selfLink: /api/v1/namespaces/istio-system/services/jaeger-query uid: 6513eded-c5eb-11e8-860c-12504ba0df7c spec: clusterIP: 172.20.14.251 ports: - name: query-http port: 16686 protocol: TCP targetPort: 16686 selector: app: jaeger sessionAffinity: None type: ClusterIP status: loadBalancer: {}

Deployment : istio-tracing :

apiVersion: extensions/v1beta1 kind: Deployment metadata: annotations: deployment.kubernetes.io/revision: "1" creationTimestamp: 2018-10-02T02:32:23Z generation: 1 labels: app: istio-tracing chart: tracing-1.0.1 heritage: Tiller release: istio name: istio-tracing namespace: istio-system resourceVersion: "5259783" selfLink: /apis/extensions/v1beta1/namespaces/istio-system/deployments/istio-tracing uid: 65056099-c5eb-11e8-860c-12504ba0df7c spec: progressDeadlineSeconds: 600 replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: app: jaeger strategy: rollingUpdate: maxSurge: 1 maxUnavailable: 1 type: RollingUpdate template: metadata: annotations: scheduler.alpha.kubernetes.io/critical-pod: "" sidecar.istio.io/inject: "false" creationTimestamp: null labels: app: jaeger spec: affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution: - preference: matchExpressions: - key: beta.kubernetes.io/arch operator: In values: - amd64 weight: 2 - preference: matchExpressions: - key: beta.kubernetes.io/arch operator: In values: - ppc64le weight: 2 - preference: matchExpressions: - key: beta.kubernetes.io/arch operator: In values: - s390x weight: 2 requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: beta.kubernetes.io/arch operator: In values: - amd64 - ppc64le - s390x containers: - env: - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace - name: COLLECTOR_ZIPKIN_HTTP_PORT value: "9411" - name: MEMORY_MAX_TRACES value: "50000" image: docker.io/jaegertracing/all-in-one:1.5 imagePullPolicy: IfNotPresent livenessProbe: failureThreshold: 3 httpGet: path: / port: 16686 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 name: jaeger ports: - containerPort: 9411 protocol: TCP - containerPort: 16686 protocol: TCP - containerPort: 5775 protocol: UDP - containerPort: 6831 protocol: UDP - containerPort: 6832 protocol: UDP readinessProbe: failureThreshold: 3 httpGet: path: / port: 16686 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 resources: requests: cpu: 10m terminationMessagePath: /dev/termination-log terminationMessagePolicy: File dnsPolicy: ClusterFirst restartPolicy: Always schedulerName: default-scheduler securityContext: {} terminationGracePeriodSeconds: 30 status: availableReplicas: 1 conditions: - lastTransitionTime: 2018-10-02T02:32:23Z lastUpdateTime: 2018-10-02T02:32:23Z message: Deployment has minimum availability. reason: MinimumReplicasAvailable status: "True" type: Available - lastTransitionTime: 2018-10-02T02:32:23Z lastUpdateTime: 2018-10-02T02:32:27Z message: ReplicaSet "istio-tracing-ff94688bb" has successfully progressed. reason: NewReplicaSetAvailable status: "True" type: Progressing observedGeneration: 1 readyReplicas: 1 replicas: 1 updatedReplicas: 1

-- Scooby
amazon-eks
envoyproxy
istio
kubernetes

0 Answers