I've followed spinnaker HTTPS configuration as per https://www.spinnaker.io/setup/security/ssl/#server-terminated-ssl
After that spinnaker services echo, front50 and igor are not starting up properly, getting below output.
kubectl get po -n spinnaker
NAME READY STATUS RESTARTS AGE
spin-clouddriver-5cd79d98d6-bhd4s 0/1 Evicted 0 3h34m
spin-clouddriver-5cd79d98d6-srnwl 1/1 Running 0 9m54s
spin-deck-5bd984bd9c-cx6xn 1/1 Running 0 3h34m
spin-echo-84848bfc46-2hwml 0/1 Running 219 17h
spin-echo-866c54bb8d-p4qbs 0/1 CrashLoopBackOff 37 3h34m
spin-front50-686f577f79-xld6d 0/1 Running 218 17h
spin-front50-988ccc789-662kn 0/1 Running 39 3h34m
spin-gate-5954b6cd88-rzqmd 1/1 Running 0 3h34m
spin-igor-5c8bd8c694-lr7z6 0/1 Running 226 17h
spin-igor-fd97cd9f6-q5wfs 0/1 Running 41 3h34m
spin-orca-546d5685dc-wswkj 1/1 Running 8 3h34m
spin-redis-58d4d69d77-st4bk 1/1 Running 0 29d
spin-rosco-fd6f6596d-2ht9r 1/1 Running 0 3h34m
Echo service
kubectl describe po -n spinnaker spin-echo-84848bfc46-2hwml
Warning Unhealthy 5m35s (x1648 over 18h) kubelet, aks-agentpool-30093549-vmss000015 Readiness probe failed:
Front50 service
kubectl describe po -n spinnaker spin-front50-988ccc789-662kn
Warning Unhealthy 77s (x866 over 3h51m) kubelet, aks-agentpool-30093549-vmss000014 Readiness probe failed: wget: can't connect to remote host (127.0.0.1): Connection refused
Igor service
kubectl describe po -n spinnaker spin-igor-fd97cd9f6-q5wfs
Warning Unhealthy 17m (x558 over 3h52m) kubelet, aks-agentpool-30093549-vmss000015 Readiness probe failed: wget: can't connect to remote host (127.0.0.1): Connection refused
Front50 logs stops after "Scanning for api listing references" and then restarts
2020-01-09 09:29:11.974 INFO 1 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'
2020-01-09 09:29:20.526 INFO 1 --- [ main] d.s.w.p.DocumentationPluginsBootstrapper : Context refreshed
2020-01-09 09:29:20.894 INFO 1 --- [ main] d.s.w.p.DocumentationPluginsBootstrapper : Found 1 custom documentation plugin(s)
2020-01-09 09:29:22.149 INFO 1 --- [ main] s.d.s.w.s.ApiListingReferenceScanner : Scanning for api listing references
Echo service logs shows, it fails because can't connect to front50
service
Caused by: java.net.ConnectException: Failed to connect to spin-front50.spinnaker/ipAddress:8080
Similarly Igor service logs shows, it fails because can't connect to echo
service
Caused by: java.net.ConnectException: Failed to connect to spin-echo.spinnaker/ipAddress:8089
I tried to increase liveness probe hal config deploy edit --liveness-probe-enabled true --liveness-probe-initial-delay-seconds 180
but still same.
Other front50 service logs are, not sure if related
2020-01-09 09:37:32.066 INFO 1 --- [ main] i.g.r.utils.RxJava2OnClasspathCondition : RxJava2 related Aspect extensions are not activated, because RxJava2 is not on the classpath.
2020-01-09 09:37:32.067 INFO 1 --- [ main] i.g.r.utils.ReactorOnClasspathCondition : Reactor related Aspect extensions are not activated because Reactor is not on the classpath.
Spinnaker is connected to Azure storage properly as it requires for front50 service. When I do hal config storage azs
I get the response of configured storage
+ Get current deployment
Success
+ Get persistent store
Success
+ Successfully got persistent store "azs".
AzsPersistentStore(storageAccountName=xxx, storageAccountKey=xxx, storageContainerName=xxx)
Please advise. Thanks.
I've again increased --liveness-probe-initial-delay-seconds to 600 sec and then it started working.