I'm currently trying to start spinnaker on our internal kubernetes cluster. The current state is:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
data-cassandra-v000-1azd8 1/1 Running 0 3h
data-redis-master-v000-68jzu 1/1 Running 0 4h
spkr-clouddriver-v000-0xpdl 1/1 Running 0 3h
spkr-deck-v000-xsl4o 1/1 Running 0 3h
spkr-echo-v000-metx4 0/1 CrashLoopBackOff 44 3h
spkr-front50-v000-nqfrx 0/1 Running 29 3h
spkr-gate-v000-ep56r 1/1 Running 0 3h
spkr-igor-v000-svkwb 0/1 CrashLoopBackOff 44 3h
spkr-orca-v000-bdeig 0/1 CrashLoopBackOff 44 3h
I currently have 2.5 issues I'm not sure how to get around:
Any hints are welcome :)
Thanks a lot!
Update: After @lwander pointed me to check the spinnaker-local.yml file I noticed that file was probably a copy of the one created from default-spinnaker-local.yml. I reverted that and tried again:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
data-cassandra-v000-gxgtv 1/1 Running 0 19m
data-redis-master-v000-y4iug 1/1 Running 0 20m
spkr-clouddriver-v000-xe3pe 1/1 Running 0 13m
spkr-deck-v000-mvs88 1/1 Running 0 13m
spkr-echo-v000-tjnk9 0/1 CrashLoopBackOff 6 13m
spkr-front50-v000-s4r61 0/1 Running 3 13m
spkr-gate-v000-5wz0f 0/1 Running 0 13m
spkr-igor-v000-xqv9g 1/1 Running 0 13m
spkr-orca-v000-ahz47 0/1 CrashLoopBackOff 6 13m
But I still get connection errors, just the hostname is no longer localhost but ending with .spinnaker.svc.cluster.local - not sure if thats also a value I would need to change or one that should work?
$ kubectl logs spkr-echo-v000-tjnk9
Caused by: com.netflix.astyanax.connectionpool.exceptions.PoolTimeoutException: PoolTimeoutException: [host=data-cassandra.spinnaker.svc.cluster.local(data-cassandra.spinnaker.svc.cluster.local):9160, latency=2000(2000), attempts=1]Timed out waiting for connection
at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.waitForConnection(SimpleHostConnectionPool.java:231)
at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.borrowConnection(SimpleHostConnectionPool.java:198)
at com.netflix.astyanax.connectionpool.impl.RoundRobinExecuteWithFailover.borrowConnection(RoundRobinExecuteWithFailover.java:84)
at com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:117)
at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:352)
at com.netflix.astyanax.thrift.ThriftKeyspaceImpl.executeOperation(ThriftKeyspaceImpl.java:517)
at com.netflix.astyanax.thrift.ThriftKeyspaceImpl.internalDescribeKeyspace(ThriftKeyspaceImpl.java:268)
at com.netflix.astyanax.thrift.ThriftKeyspaceImpl.describeKeyspace(ThriftKeyspaceImpl.java:264)
at com.netflix.scheduledactions.persistence.cassandra.ThriftCassandraDao.createColumnFamily(ThriftCassandraDao.java:81)
... 70 more
$ kubectl logs spkr-orca-v000-ahz47
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
... 21 more
Caused by: java.net.UnknownHostException: data-redis-server.spinnaker.svc.cluster.local
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at redis.clients.jedis.Connection.connect(Connection.java:158)
... 44 more
$ kubectl logs spkr-igor-v000-xqv9g
Caused by: java.net.UnknownHostException: spkr-clouddriver.spinnaker.svc.cluster.local: unknown error
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
at java.net.InetAddress.getAllByName(InetAddress.java:1192)
at java.net.InetAddress.getAllByName(InetAddress.java:1126)
at com.squareup.okhttp.internal.Network$1.resolveInetAddresses(Network.java:29)
Update 2: it seems it not too far away to connect the parts. The fabric8 ui for the kubernetes cluster seems to recognize most of it.