I was using fabric8io to write a service function, relevant code is as below
KubernetesClient fabricClient = new DefaultKubernetesClient();
fabricClient.pods().inNamespace("xxxnamespace").withLabel("somekey", somevalue).list().getItems()It was working fine when I'm doing unit test. But when I was deploying the whole application, and trigger the service funtion, it throws an error as below
java.lang.RuntimeException: Error processing bean
at org.springframework.statemachine.processor.MethodInvokingStateMachineRuntimeProcessor.process(MethodInvokingStateMachineRuntimeProcessor.java:70) ~[spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.processor.StateMachineHandler.handle(StateMachineHandler.java:135) ~[spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.processor.StateMachineHandlerCallHelper.getStateMachineHandlerResults(StateMachineHandlerCallHelper.java:438) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.processor.StateMachineHandlerCallHelper.callOnTransition(StateMachineHandlerCallHelper.java:237) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.support.StateMachineObjectSupport.notifyTransition(StateMachineObjectSupport.java:225) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.support.AbstractStateMachine$3.transit(AbstractStateMachine.java:329) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.support.DefaultStateMachineExecutor.handleTriggerTrans(DefaultStateMachineExecutor.java:287) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.support.DefaultStateMachineExecutor.handleTriggerTrans(DefaultStateMachineExecutor.java:210) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.support.DefaultStateMachineExecutor.processTriggerQueue(DefaultStateMachineExecutor.java:450) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.support.DefaultStateMachineExecutor.access$200(DefaultStateMachineExecutor.java:64) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.support.DefaultStateMachineExecutor$1.run(DefaultStateMachineExecutor.java:330) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.statemachine.support.DefaultStateMachineExecutor.scheduleEventQueueProcessing(DefaultStateMachineExecutor.java:353) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.support.DefaultStateMachineExecutor.access$500(DefaultStateMachineExecutor.java:64) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.support.DefaultStateMachineExecutor$2.triggered(DefaultStateMachineExecutor.java:540) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.trigger.CompositeTriggerListener.triggered(CompositeTriggerListener.java:34) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.trigger.TimerTrigger.notifyTriggered(TimerTrigger.java:123) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.trigger.TimerTrigger.access$000(TimerTrigger.java:33) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.trigger.TimerTrigger$1.run(TimerTrigger.java:117) [spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93) [spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_181]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_181]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_181]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: java.lang.RuntimeException: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [list] for kind: [Pod] with name: [null] in namespace: [biztech-bos] failed.
at com.xxxx.saveBuildLog(BosBuildLogAop.java:56) ~[bos-coordinator-server-1.30.0-SNAPSHOT.jar:?]
at com.xxxx.BosBuildLogAop.logAround(BosBuildLogAop.java:39) ~[bos-coordinator-server-1.30.0-SNAPSHOT.jar:?]
at sun.reflect.GeneratedMethodAccessor689.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at com.xxxx.statemachine.TransitionActionConfig$EnhancerBySpringCGLIB$165e28b8.checkingIncrementDtsDeploy(<generated>) ~[bos-coordinator-server-1.30.0-SNAPSHOT.jar:?]
at sun.reflect.GeneratedMethodAccessor823.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]
at org.springframework.expression.spel.support.ReflectiveMethodExecutor.execute(ReflectiveMethodExecutor.java:130) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.expression.spel.ast.MethodReference.getValueInternal(MethodReference.java:111) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.expression.spel.ast.MethodReference.access$000(MethodReference.java:54) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.expression.spel.ast.MethodReference$MethodValueRef.getValue(MethodReference.java:390) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:90) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:116) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:365) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.statemachine.support.AbstractExpressionEvaluator.evaluateExpression(AbstractExpressionEvaluator.java:126) ~[spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.processor.StateMachineMethodInvokerHelper.processInternal(StateMachineMethodInvokerHelper.java:243) ~[spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.processor.StateMachineMethodInvokerHelper.process(StateMachineMethodInvokerHelper.java:119) ~[spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.processor.MethodInvokingStateMachineRuntimeProcessor.process(MethodInvokingStateMachineRuntimeProcessor.java:68) ~[spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
... 27 more
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [list] for kind: [Pod] with name: [null] in namespace: [biztech-bos] failed.
at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) ~[kubernetes-client-5.2.1.jar:?]
at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72) ~[kubernetes-client-5.2.1.jar:?]
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:168) ~[kubernetes-client-5.2.1.jar:?]
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:664) ~[kubernetes-client-5.2.1.jar:?]
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:86) ~[kubernetes-client-5.2.1.jar:?]
at com.xxxx.service.impl.DTSServiceImpl.checkDeployIncrementDts(DTSServiceImpl.java:217) ~[bos-coordinator-server-1.30.0-SNAPSHOT.jar:?]
at com.xxxx.configuration.statemachine.TransitionActionConfig.checkingIncrementDtsDeploy(TransitionActionConfig.java:96) ~[bos-coordinator-server-1.30.0-SNAPSHOT.jar:?]
at com.xxxx.configuration.statemachine.TransitionActionConfig$FastClassBySpringCGLIB$f83ad12a.invoke(<generated>) ~[bos-coordinator-server-1.30.0-SNAPSHOT.jar:?]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at com.xxxx.server.aop.BosBuildLogAop.saveBuildLog(BosBuildLogAop.java:52) ~[bos-coordinator-server-1.30.0-SNAPSHOT.jar:?]
at com.xxxx.server.aop.BosBuildLogAop.logAround(BosBuildLogAop.java:39) ~[bos-coordinator-server-1.30.0-SNAPSHOT.jar:?]
at sun.reflect.GeneratedMethodAccessor689.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) ~[spring-aop-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at com.xxxx.server.configuration.statemachine.TransitionActionConfig$EnhancerBySpringCGLIB$165e28b8.checkingIncrementDtsDeploy(<generated>) ~[bos-coordinator-server-1.30.0-SNAPSHOT.jar:?]
at sun.reflect.GeneratedMethodAccessor823.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]
at org.springframework.expression.spel.support.ReflectiveMethodExecutor.execute(ReflectiveMethodExecutor.java:130) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.expression.spel.ast.MethodReference.getValueInternal(MethodReference.java:111) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.expression.spel.ast.MethodReference.access$000(MethodReference.java:54) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.expression.spel.ast.MethodReference$MethodValueRef.getValue(MethodReference.java:390) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:90) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:116) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:365) ~[spring-expression-5.1.3.RELEASE.jar:5.1.3.RELEASE]
at org.springframework.statemachine.support.AbstractExpressionEvaluator.evaluateExpression(AbstractExpressionEvaluator.java:126) ~[spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.processor.StateMachineMethodInvokerHelper.processInternal(StateMachineMethodInvokerHelper.java:243) ~[spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.processor.StateMachineMethodInvokerHelper.process(StateMachineMethodInvokerHelper.java:119) ~[spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
at org.springframework.statemachine.processor.MethodInvokingStateMachineRuntimeProcessor.process(MethodInvokingStateMachineRuntimeProcessor.java:68) ~[spring-statemachine-core-2.2.0.RELEASE.jar:2.2.0.RELEASE]
... 27 more
Caused by: java.net.UnknownHostException: kubernetes.default.svc
at java.net.InetAddress.getAllByName0(InetAddress.java:1280) ~[?:1.8.0_181]
at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[?:1.8.0_181]
at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[?:1.8.0_181]
...
in which DTSServiceImplis my service implementation class.<br>
On the deployment machine, I was putting kubeconfig file on ~/.kube/config, and kubectlcommand line is woking fine. <br>
I have no clue how to handle this problem. Any idea how should I find out the exception reason？<br>
Thanks in advance!
Thanks to @Yayotron and my colleagues, it turns out that I was setting the wrong path to kubeconfig system property.<br>
According to this question, I was setting kubeconfig system property to some specific path, so that fabric8 client can get the kubeconfig file from that path and get clusters informations / authentication tokens, etc.<br>
So if anyone meet this exception in the future, you can consider whether if the client configuration has been configured right.
Fabric8 uses Kubernetes REST API to perform their operations, by default their HTTP client is assuming that it's running inside kubernetes cluster which is why it's trying to reach kubernetes.default.svc
Since you're calling it outside the cluster then you must tell Fabric8 the address of your cluster, this can be done by specifying the host during creation of the client
new DefaultKubernetesClient("https://my-cluster");If you're able to SSH to the server which is running your application and ping the cluster address, most likely fabric8 will work.