K
Q

Problem to access Deployment with service

4/4/2020

I'm trying to run Zipkin on Kubernetes cluster. This is my Deployment file:

apiVersion: v1
kind: Service
metadata:
  name: zipkin
  namespace: default
  labels:
    app: zipkin
spec:
  type: NodePort
  selector:
    app: zipkin
  ports:
  - name: http
    port: 9411
    protocol: "TCP"

---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    fabric8.io/git-commit: "78bd65851c7d9e22652957107fc339a55f80b606"
    fabric8.io/git-branch: "release-v0.1.5"
    fabric8.io/metrics-path: "dashboard/file/kubernetes-pods.json/?var-project=zipkin&var-version=0.1.5"
    fabric8.io/build-id: "3"
    fabric8.io/git-url: "http://gogs.ux.fabric8.io/gogsadmin/oss-parent/commit/78bd65851c7d9e22652957107fc339a55f80b606"
    fabric8.io/build-url: "http://jenkins.ux.fabric8.io/job/oss-parent/3"
  labels:
    app: zipkin
  name: zipkin
spec:
  replicas: 1
  selector:
    matchLabels:
      app: zipkin
  template:
    metadata:
      annotations:
        fabric8.io/git-commit: "78bd65851c7d9e22652957107fc339a55f80b606"
        fabric8.io/metrics-path: "dashboard/file/kubernetes-pods.json/?var-project=zipkin&var-version=0.1.5"
        fabric8.io/build-id: "3"
        fabric8.io/build-url: "http://jenkins.ux.fabric8.io/job/oss-parent/3"
        fabric8.io/git-branch: "release-v0.1.5"
        fabric8.io/git-url: "http://gogs.ux.fabric8.io/gogsadmin/oss-parent/commit/78bd65851c7d9e22652957107fc339a55f80b606"
      labels:
        app: zipkin
    spec:
      containers:
      - env:
        - name: COLLECTOR_KAFKA_ENABLED
          value: "true"
        - name: TRANSPORT_TYPE
          value: http
        - name: KAFKA_BOOTSTRAP_SERVERS
          value: http://kafka-headless:9092
        - name: HTTP_COLLECTOR_ENABLED
          value: "true"
        - name: COLLECTOR_ACTIVEMQ_ENABLED
          value: "false"
        - name: COLLECTOR_RABBITMQ_ENABLED
          value: "false"
        - name: QUERY_LOG_LEVEL
          value: DEBUG
        - name: logging.level.zipkin2
          value: DEBUG
        - name: "KUBERNETES_NAMESPACE"
          value: default
        image: "openzipkin/zipkin:latest"
        name: zipkin
        ports:
        - name: http
          containerPort: 9411
        readinessProbe:
          httpGet:
            path: /api/v1/services
            port: 9411
          initialDelaySeconds: 20
          periodSeconds: 15
          failureThreshold: 6
        livenessProbe:
          httpGet:
            path: /health
            port: 9411
          initialDelaySeconds: 20
        resources:
          limits:
            cpu: "0"
            memory: "0"
          requests:
            cpu: "0"
            memory: "0"

when I call /healthendpoint from other deployment inside of my cluster with pod's ip, I can access to it and this is my output:

curl 10.42.4.17:9411/health
{
  "status" : "UP",
  "zipkin" : {
    "status" : "UP",
    "details" : {
      "InMemoryStorage{}" : {
        "status" : "UP"
      },
      "KafkaCollector{bootstrapServers=http://kafka-headless:9092, topic=zipkin}" : {
        "status" : "UP"
      }
    }
  }

but when I try to access with service name, I can't.

curl http://zipkin:9411/health
curl: (7) Failed to connect to zipkin port 9411: Connection refused

this is the output of kubectl describe svc zipkin

kubectl describe svc zipkin
Name:                     zipkin
Namespace:                default
Labels:                   app=zipkin
Annotations:              field.cattle.io/publicEndpoints:
                            [{"addresses":["192.168.164.42"],"port":30562,"protocol":"TCP","serviceName":"default:zipkin","allNodes":true}]
                          kubectl.kubernetes.io/last-applied-configuration:
                            {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"zipkin"},"name":"zipkin","namespace":"default"},"spec":{...
Selector:                 app=zipkin
Type:                     NodePort
IP:                       10.43.186.82
Port:                     http  9411/TCP
TargetPort:               9411/TCP
NodePort:                 http  30562/TCP
Endpoints:                
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

and: kubectl logs zipkin-9898fcf7f-2nmkx

                  oo
                 oooo
                oooooo
               oooooooo
              oooooooooo
             oooooooooooo
           ooooooo  ooooooo
          oooooo     ooooooo
         oooooo       ooooooo
        oooooo   o  o   oooooo
       oooooo   oo  oo   oooooo
     ooooooo  oooo  oooo  ooooooo
    oooooo   ooooo  ooooo  ooooooo
   oooooo   oooooo  oooooo  ooooooo
  oooooooo      oo  oo      oooooooo
  ooooooooooooo oo  oo ooooooooooooo
      oooooooooooo  oooooooooooo
          oooooooo  oooooooo
              oooo  oooo

     ________ ____  _  _____ _   _
    |__  /_ _|  _ \| |/ /_ _| \ | |
      / / | || |_) | ' / | ||  \| |
     / /_ | ||  __/| . \ | || |\  |
    |____|___|_|   |_|\_\___|_| \_|

:: version 2.21.0 :: commit 36b7d32 ::

2020-04-04 12:38:52.851 DEBUG 1 --- [           main] z.s.i.ZipkinModuleImporter               : no modules found under path zipkin.internal.module
2020-04-04 12:38:52.862 DEBUG 1 --- [           main] z.s.i.ZipkinActuatorImporter             : attempting to load actuator configuration: [org.springframework.boot.actuate.autoconfigure.beans.BeansEndpointAutoConfiguration, org.springframework.boot.actuate.autoconfigure.condition.ConditionsReportEndpointAutoConfiguration, org.springframework.boot.actuate.autoconfigure.context.properties.ConfigurationPropertiesReportEndpointAutoConfiguration, org.springframework.boot.actuate.autoconfigure.endpoint.EndpointAutoConfiguration, org.springframework.boot.actuate.autoconfigure.env.EnvironmentEndpointAutoConfiguration, org.springframework.boot.actuate.autoconfigure.management.HeapDumpWebEndpointAutoConfiguration, org.springframework.boot.actuate.autoconfigure.logging.LoggersEndpointAutoConfiguration, org.springframework.boot.actuate.autoconfigure.management.ThreadDumpEndpointAutoConfiguration]
2020-04-04 12:38:53.941  INFO 1 --- [           main] z.s.ZipkinServer                         : Starting ZipkinServer on zipkin-9898fcf7f-2nmkx with PID 1 (/zipkin/BOOT-INF/classes started by zipkin in /zipkin)
2020-04-04 12:38:53.942  INFO 1 --- [           main] z.s.ZipkinServer                         : The following profiles are active: shared
2020-04-04 12:38:56.034  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.verboseExceptions: rate-limit=10 (default)
2020-04-04 12:38:56.056  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.verboseSocketExceptions: false (default)
2020-04-04 12:38:56.056  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.verboseResponses: false (default)
2020-04-04 12:38:56.099  INFO 1 --- [pool-2-thread-1] o.a.k.c.c.ConsumerConfig                 : ConsumerConfig values: 
        allow.auto.create.topics = true
        auto.commit.interval.ms = 5000
        auto.offset.reset = earliest
        bootstrap.servers = [http://kafka-headless:9092]
        check.crcs = true
        client.dns.lookup = default
        client.id = 
        client.rack = 
        connections.max.idle.ms = 540000
        default.api.timeout.ms = 60000
        enable.auto.commit = true
        exclude.internal.topics = true
        fetch.max.bytes = 52428800
        fetch.max.wait.ms = 500
        fetch.min.bytes = 1
        group.id = zipkin
        group.instance.id = null
        heartbeat.interval.ms = 3000
        interceptor.classes = []
        internal.leave.group.on.close = true
        isolation.level = read_uncommitted
        key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
        max.partition.fetch.bytes = 1048576
        max.poll.interval.ms = 300000
        max.poll.records = 500
        metadata.max.age.ms = 300000
        metric.reporters = []
        metrics.num.samples = 2
        metrics.recording.level = INFO
        metrics.sample.window.ms = 30000
        partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
        receive.buffer.bytes = 65536
        reconnect.backoff.max.ms = 1000
        reconnect.backoff.ms = 50
        request.timeout.ms = 30000
        retry.backoff.ms = 100
        sasl.client.callback.handler.class = null
        sasl.jaas.config = null
        sasl.kerberos.kinit.cmd = /usr/bin/kinit
        sasl.kerberos.min.time.before.relogin = 60000
        sasl.kerberos.service.name = null
        sasl.kerberos.ticket.renew.jitter = 0.05
        sasl.kerberos.ticket.renew.window.factor = 0.8
        sasl.login.callback.handler.class = null
        sasl.login.class = null
        sasl.login.refresh.buffer.seconds = 300
        sasl.login.refresh.min.period.seconds = 60
        sasl.login.refresh.window.factor = 0.8
        sasl.login.refresh.window.jitter = 0.05
        sasl.mechanism = GSSAPI
        security.protocol = PLAINTEXT
        security.providers = null
        send.buffer.bytes = 131072
        session.timeout.ms = 10000
        ssl.cipher.suites = null
        ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
        ssl.endpoint.identification.algorithm = https
        ssl.key.password = null
        ssl.keymanager.algorithm = SunX509
        ssl.keystore.location = null
        ssl.keystore.password = null
        ssl.keystore.type = JKS
        ssl.protocol = TLS
        ssl.provider = null
        ssl.secure.random.implementation = null
        ssl.trustmanager.algorithm = PKIX
        ssl.truststore.location = null
        ssl.truststore.password = null
        ssl.truststore.type = JKS
        value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer

2020-04-04 12:38:56.208  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.useEpoll: true (default)
2020-04-04 12:38:56.210  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.maxNumConnections: 2147483647 (default)
2020-04-04 12:38:56.211  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.numCommonWorkers: 2 (default)
2020-04-04 12:38:56.212  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.numCommonBlockingTaskThreads: 200 (default)
2020-04-04 12:38:56.215  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultMaxRequestLength: 10485760 (default)
2020-04-04 12:38:56.216  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultMaxResponseLength: 10485760 (default)
2020-04-04 12:38:56.217  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultRequestTimeoutMillis: 10000 (default)
2020-04-04 12:38:56.218  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultResponseTimeoutMillis: 15000 (default)
2020-04-04 12:38:56.218  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultConnectTimeoutMillis: 3200 (default)
2020-04-04 12:38:56.219  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultWriteTimeoutMillis: 1000 (default)
2020-04-04 12:38:56.220  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultServerIdleTimeoutMillis: 15000 (default)
2020-04-04 12:38:56.221  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultClientIdleTimeoutMillis: 10000 (default)
2020-04-04 12:38:56.222  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultHttp2PingTimeoutMillis: 0 (default)
2020-04-04 12:38:56.222  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultHttp2InitialConnectionWindowSize: 1048576 (default)
2020-04-04 12:38:56.223  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultHttp2InitialStreamWindowSize: 1048576 (default)
2020-04-04 12:38:56.224  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultHttp2MaxFrameSize: 16384 (default)
2020-04-04 12:38:56.225  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultHttp2MaxStreamsPerConnection: 2147483647 (default)
2020-04-04 12:38:56.226  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultHttp2MaxHeaderListSize: 8192 (default)
2020-04-04 12:38:56.227  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultHttp1MaxInitialLineLength: 4096 (default)
2020-04-04 12:38:56.228  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultHttp1MaxHeaderSize: 8192 (default)
2020-04-04 12:38:56.229  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultHttp1MaxChunkSize: 8192 (default)
2020-04-04 12:38:56.230  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultUseHttp2Preface: true (default)
2020-04-04 12:38:56.230  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultUseHttp1Pipelining: false (default)
2020-04-04 12:38:56.230  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultUseHttp2PingWhenNoActiveStreams: false (default)
2020-04-04 12:38:56.231  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultBackoffSpec: exponential=200:10000,jitter=0.2 (default)
2020-04-04 12:38:56.232  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.defaultMaxTotalAttempts: 10 (default)
2020-04-04 12:38:56.233  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.routeCache: maximumSize=4096 (default)
2020-04-04 12:38:56.233  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.routeDecoratorCache: maximumSize=4096 (default)
2020-04-04 12:38:56.233  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.compositeServiceCache: maximumSize=256 (default)
2020-04-04 12:38:56.234  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.parsedPathCache: maximumSize=4096 (default)
2020-04-04 12:38:56.234  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.headerValueCache: maximumSize=4096 (default)
2020-04-04 12:38:56.234  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.fileServiceCache: maximumSize=1024 (default)
2020-04-04 12:38:56.240  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.cachedHeaders: :authority,:scheme,:method,accept-encoding,content-type (default)
2020-04-04 12:38:56.243  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.annotatedServiceExceptionVerbosity: unhandled (default)
2020-04-04 12:38:56.246  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.useJdkDnsResolver: false (default)
2020-04-04 12:38:56.246  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.reportBlockedEventLoop: true (default)
2020-04-04 12:38:56.247  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.validateHeaders: true (default)
2020-04-04 12:38:56.247  INFO 1 --- [           main] c.l.a.c.Flags                            : com.linecorp.armeria.useLegacyMeterNames: false (default)
2020-04-04 12:38:56.248  INFO 1 --- [           main] c.l.a.c.Flags                            : Using /dev/epoll
2020-04-04 12:38:56.547  INFO 1 --- [pool-2-thread-1] o.a.k.c.u.AppInfoParser                  : Kafka version: 2.4.1
2020-04-04 12:38:56.547  INFO 1 --- [pool-2-thread-1] o.a.k.c.u.AppInfoParser                  : Kafka commitId: c57222ae8cd7866b
2020-04-04 12:38:56.548  INFO 1 --- [pool-2-thread-1] o.a.k.c.u.AppInfoParser                  : Kafka startTimeMs: 1586003936543
2020-04-04 12:38:56.557  INFO 1 --- [pool-2-thread-1] o.a.k.c.c.KafkaConsumer                  : [Consumer clientId=consumer-zipkin-1, groupId=zipkin] Subscribed to topic(s): zipkin
2020-04-04 12:38:56.558  INFO 1 --- [pool-2-thread-1] z.c.k.KafkaCollectorWorker               : Kafka consumer starting polling loop.
2020-04-04 12:38:57.002  INFO 1 --- [           main] c.l.a.s.d.DocStringExtractor             : Using com.linecorp.armeria.thrift.jsonDir: META-INF/armeria/thrift
2020-04-04 12:38:57.224  INFO 1 --- [pool-2-thread-1] o.a.k.c.Metadata                         : [Consumer clientId=consumer-zipkin-1, groupId=zipkin] Cluster ID: NxPNZeYPRLChEBVHfUiuAA
2020-04-04 12:38:57.227  INFO 1 --- [pool-2-thread-1] o.a.k.c.c.i.AbstractCoordinator          : [Consumer clientId=consumer-zipkin-1, groupId=zipkin] Discovered group coordinator 10.42.6.12:9092 (id: 2147483647 rack: null)
2020-04-04 12:38:57.235  INFO 1 --- [pool-2-thread-1] o.a.k.c.c.i.AbstractCoordinator          : [Consumer clientId=consumer-zipkin-1, groupId=zipkin] (Re-)joining group
2020-04-04 12:38:57.255  INFO 1 --- [pool-2-thread-1] o.a.k.c.c.i.AbstractCoordinator          : [Consumer clientId=consumer-zipkin-1, groupId=zipkin] (Re-)joining group
2020-04-04 12:38:57.268  INFO 1 --- [           main] c.l.a.c.u.SystemInfo                     : Hostname: zipkin-9898fcf7f-2nmkx (from /proc/sys/kernel/hostname)
2020-04-04 12:38:57.560 DEBUG 1 --- [pool-2-thread-1] z.c.k.KafkaCollectorWorker               : Kafka polling returned batch of 0 messages.
2020-04-04 12:38:58.200  INFO 1 --- [oss-http-*:9411] c.l.a.s.Server                           : Serving HTTP at /0.0.0.0:9411 - http://127.0.0.1:9411/
2020-04-04 12:38:58.207  INFO 1 --- [           main] c.l.a.s.ArmeriaAutoConfiguration         : Armeria server started at ports: {/0.0.0.0:9411=ServerPort(/0.0.0.0:9411, [http])}
2020-04-04 12:38:58.268  INFO 1 --- [           main] z.s.ZipkinServer                         : Started ZipkinServer in 7.288 seconds (JVM running for 12.716)
2020-04-04 12:38:58.566 DEBUG 1 --- [pool-2-thread-1] z.c.k.KafkaCollectorWorker               : Kafka polling returned batch of 0 messages.
2020-04-04 12:38:59.568 DEBUG 1 --- [pool-2-thread-1] z.c.k.KafkaCollectorWorker               : Kafka polling returned batch of 0 messages.
2020-04-04 12:39:00.265  INFO 1 --- [pool-2-thread-1] o.a.k.c.c.i.ConsumerCoordinator          : [Consumer clientId=consumer-zipkin-1, groupId=zipkin] Finished assignment for group at generation 117: {consumer-zipkin-1-fb98f8ee-4b85-4616-b60b-35ef4b41706d=Assignment(partitions=[zipkin-0])}
2020-04-04 12:39:00.282  INFO 1 --- [pool-2-thread-1] o.a.k.c.c.i.AbstractCoordinator          : [Consumer clientId=consumer-zipkin-1, groupId=zipkin] Successfully joined group with generation 117
2020-04-04 12:39:00.294  INFO 1 --- [pool-2-thread-1] o.a.k.c.c.i.ConsumerCoordinator          : [Consumer clientId=consumer-zipkin-1, groupId=zipkin] Adding newly assigned partitions: zipkin-0
2020-04-04 12:39:00.316  INFO 1 --- [pool-2-thread-1] o.a.k.c.c.i.ConsumerCoordinator          : [Consumer clientId=consumer-zipkin-1, groupId=zipkin] Setting offset for partition zipkin-0 to the committed offset FetchPosition{offset=0, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=10.42.4.102:9092 (id: 1 rack: null), epoch=-1}}
2020-04-04 12:39:00.570 DEBUG 1 --- [pool-2-thread-1] z.c.k.KafkaCollectorWorker               : Kafka polling returned batch of 0 messages.

can anybody help me?

-- Amir Soleimani
kubernetes
zipkin

Similar Questions

How many threads for a kubernetes application?
Playing with Raspberry Pi and kubernettes
How does K8S auto-start the Deployments and DaemonSets?
x509: certificate signed by unknown authority (possibly because of "crypto/rsa:...authority certificate "osmacbook")

1 Answer

4/4/2020
>> kubectl get deployment zipkin 
NAME READY UP-TO-DATE AVAILABLE AGE 
zipkin 0/1 1 0 14m

This indicates that the pod is not ready, hence the service will not add that pod's IP in the list of endpoints. Check the readiness probe of your pod by describing it and debug the issue that's making it non-ready. Once this if fixed, you'll start seeing some endpoints populated when you describe the service & that will enable you to access the service by DNS name.

-- Abhishek Jaisingh
Source: StackOverflow