I am new to Ignite and Kubernetes.
Trying to integrate Ignite Version 2.6.0 data grid application in AKS. Client jar file is in VM, where kubectl is exposed. Using kubectl executed all .yaml files, in the client jar VM, as per the below url
https://apacheignite.readme.io/docs/stateful-deployment
Able to telnet master url of the Kubernetes cluster from the VM where Client jar is present.
ignite-stateful-set.yaml is configured with ‘example-kube-persistence.xml’ as mentioned below. - name: selfLink value: file:///opt/example-kube-persistence.xml
Ignite client is reading the configuration like ... try (Ignite ignite = Ignition.start("/opt/apache-ignite-fabric-2.6.0-bin/examples/config/example-ignite.xml"))
example-ignite.xml configuration ...
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
<property name="namespace" value="ignite"/>
<property name="serviceName" value="ignite"/>
<property name="masterUrl" value="https://xxxxxxxxx.azmk8s.io:443"/>
<property name="AccountToken" value="/data/ignite"/>
</bean>Started running client and getting the following exception...
class org.apache.ignite.spi.IgniteSpiException: Failed to retrieve Ignite pods IP addresses.
at org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:172)
at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:1828)
at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.resolvedAddresses(TcpDiscoverySpi.java:1776)
at org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1029)
at org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:890)
at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:373)
at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:1948)
at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:915)
at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1721)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1028)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2014)
at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1723)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1151)
at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1069)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:955)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:854)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:724)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:693)
at org.apache.ignite.Ignition.start(Ignition.java:352)
at com.ecoenergy.cortix.ignite.IgniteFutureAsync.main(IgniteFutureAsync.java:54)
Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to load services account token [setAccountToken= /var/run/secrets/kubernetes.io/serviceaccount/token]
at org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.serviceAccountToken(TcpDiscoveryKubernetesIpFinder.java:287)
at org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:148)
... 20 more
Caused by: java.nio.file.NoSuchFileException: /var/run/secrets/kubernetes.io/serviceaccount/token
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
at java.nio.file.Files.newByteChannel(Files.java:361)
at java.nio.file.Files.newByteChannel(Files.java:407)
at java.nio.file.Files.readAllBytes(Files.java:3152)
at org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.serviceAccountToken(TcpDiscoveryKubernetesIpFinder.java:285)
... 21 morePlease help me out how to get rid of exception. Thanks in advance.
It's looking for the service account token in /var/run/secrets/kubernetes.io/serviceaccount/token but that file does not exist. The question is why?
Some commands:
kubectl get serviceaccountsYou should find one called "ignite". If not, that's your problem.
kubectl get serviceaccounts/ignite -o yamlIf there's a property called automountServiceAccountToken set to false that's your problem.
At the bottom it should list the secrets. If not, that's your problem.
kubectl get secrets/default-token-wnfp5 -o yaml(Obviously replace with your secret name.)
You should find a token in the output.
kubectl get pods
kubectl get pods/ignite-0 -o yamlYou're looking for the local name of the secret. And then, later in the file, where it's mounted. If it's not mounted, that's your problem. If it's pointing somewhere weird, that's your problem.
Also, have a look at:
kubectl get clusterrolebinding/igniteThe roleref and subjects should all point to sensible values.
Kubernetes has so many levels of indirections that it's easy to miss a link somewhere. If you didn't accidentally miss a step (or there are some Azure quirks) you can also look at the service account documentation.