Can't get my .jar starting Ignite to work on GKE

8/10/2018

I've set up correctly my RBAC, and my services as described in Ignite doc with the fix described here >> Failed to retrieve Ignite pods IP addresses I managed to run the docker image "apacheignite/ignite" with no problem after that.

How ever, when I can't get to run this code :

    IgniteConfiguration cfg = new IgniteConfiguration();
    cfg.setExecutorConfiguration(new ExecutorConfiguration("myPool").setSize(1));

    DataStorageConfiguration dsCfg = new DataStorageConfiguration();
    DataRegionConfiguration dataRegionConfigurations = new DataRegionConfiguration();
    dataRegionConfigurations.setPersistenceEnabled(true);

    dsCfg.getDefaultDataRegionConfiguration();
    dsCfg.setDefaultDataRegionConfiguration(dataRegionConfigurations);
    cfg.setDataStorageConfiguration(dsCfg);

    TcpDiscoverySpi tcpDS = new TcpDiscoverySpi();
    TcpDiscoveryKubernetesIpFinder ipFinder = new TcpDiscoveryKubernetesIpFinder();
    ipFinder.setNamespace("ignite");
    tcpDS = tcpDS.setIpFinder(ipFinder);        
    cfg.setDiscoverySpi(tcpDS);

    Ignite ignite = Ignition.start(cfg);

I get the following error : class org.apache.ignite.spi.IgniteSpiException: Failed to retrieve Ignite pods IP addresses.

It seems to be caused by the following exception : java.io.FileNotFoundException: https://kubernetes.default.svc.cluster.local:443/api/v1/namespaces/default/endpoints/ignite-worker

My docker file is the following :

# Use an official Python runtime as a parent image
FROM openjdk:8

# Set the working directory to /app
WORKDIR /app

# Copy the current directory contents into the container at /app
ADD . /app

EXPOSE 47100
EXPOSE 47500
EXPOSE 49112
EXPOSE 10900
EXPOSE 10800
EXPOSE 11211

# Run app.py when the container launches
CMD ["java", "-Xms3g", "-Xmx3g", "-jar", "ignite-app.jar"]

and the yaml file for my deployment looks like this :

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  # Custom Ignite cluster's name.
  name: ignite-cluster
spec:
  # A number of Ignite pods to be started by Kubernetes initially.
  replicas: 1
  template:
    metadata:
      labels:
        app: ignite
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - ignite
            topologyKey: kubernetes.io/hostname
      serviceAccount: ignite
      containers:
        # Custom Ignite pod name.
      - name: ignite-node
        image: gcr.io/my-project/my-image:v1
        env:
        - name: OPTION_LIBS
          value: ignite-kubernetes
        - name: CONFIG_URI
          value: https://raw.githubusercontent.com/apache/ignite/master/modules/kubernetes/config/example-kube-persistence.xml
        ports:
        # Ports to open.
        # Might be optional depending on your Kubernetes environment.
        - containerPort: 11211 # REST port number.
        - containerPort: 47100 # communication SPI port number.
        - containerPort: 47500 # discovery SPI port number.
        - containerPort: 49112 # JMX port number.
        - containerPort: 10800 # SQL port number.
        - containerPort: 10900 # Thin clients port number.

I can't manage to understand how to solve my "java.io.FileNotFoundException: https://kubernetes.default.svc.cluster.local:443/api/v1/namespaces/default/endpoints/ignite-worker" exception.

The stack is the following :

org.apache.ignite.spi.IgniteSpiException: Failed to retrieve Ignite pods IP addresses.
    at org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:172)
    at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:1828)
    at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.resolvedAddresses(TcpDiscoverySpi.java:1776)
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1029)
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:890)
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:373)
    at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:1948)
    at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
    at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:915)
    at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1721)
    at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1028)
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2014)
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1723)
    at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1151)
    at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1069)
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:955)
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:854)
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:724)
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:693)
    at org.apache.ignite.Ignition.start(Ignition.java:352)
    at Initiator.main(Initiator.java:17)
Caused by: java.io.FileNotFoundException: https://kubernetes.default.svc.cluster.local:443/api/v1/namespaces/default/endpoints/ignite-worker
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1890)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263)
    at org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:153)

Does anybody have any clue of what's going on please ?

-- Ab. C.
google-kubernetes-engine
ignite
java
kubernetes

1 Answer

8/10/2018

Looking at the URL from the stack trace, I can tell that IP finder uses default namespace and ignite-worker service name. However, that does not correspond to the configuration you provided. You should check what configuration is actually used and where ignite-worker service name is coming from. Once all correct settings are properly set on the IP finder, it should work as expceted.

-- Valentin Kulichenko
Source: StackOverflow