I have create a monolithic application, using JHipster, and I want to run several replicas of it on a GCP Kubernetes Engine cluster.

As I would like to share data between replicas (or even services that are on that cluster), I've decided to use the built-int Hazelcast cache, but instead of using the recommended Eureka discovery-client I want to use K8 dns-discovery add-on.

So I've added to my pom.xml:

<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast-kubernetes</artifactId>
    <version>1.1.0</version>
</dependency>

And played a bit with the generated CacheConfiguration.java file to support the hazelcast-kubernetes configuration (based on their github repo):

@Bean
public HazelcastInstance hazelcastInstance(JHipsterProperties jHipsterProperties) {
    log.debug("Configuring Hazelcast");
    HazelcastInstance hazelCastInstance = Hazelcast.getHazelcastInstanceByName("hazelcastK8");
    if (hazelCastInstance != null) {
        log.debug("Hazelcast already initialized");
        return hazelCastInstance;
    }
    Config config = new Config();
    config.setInstanceName("hazelcastK8");
    config.getNetworkConfig().setPort(5701);
    config.getNetworkConfig().setPortAutoIncrement(true);

    config.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false);
    config.getNetworkConfig().getJoin().getTcpIpConfig().setEnabled(false);

    if (env.acceptsProfiles(JHipsterConstants.SPRING_PROFILE_DEVELOPMENT)) {
        System.setProperty("hazelcast.local.localAddress", "127.0.0.1");
        config.getNetworkConfig().getJoin().getAwsConfig().setEnabled(false);
    } else { // In production we want to use hazelcast k8's dns discovery service
        log.info("Configuring hazelcast DNS discovery by K8");

        DiscoveryStrategyConfig strategyConfig =
            new DiscoveryStrategyConfig(
                new HazelcastKubernetesDiscoveryStrategyFactory()
            );

        // Default namespace
        strategyConfig.addProperty("service-dns", "hazelcastk8.default.svc.cluster.local");
        strategyConfig.addProperty("service-dns-timeout", 10);

        config.getNetworkConfig().getJoin().getDiscoveryConfig().addDiscoveryStrategyConfig(strategyConfig);
    }

    config.getMapConfigs().put("default", initializeDefaultMapConfig());
    config.setManagementCenterConfig(initializeDefaultManagementCenterConfig(jHipsterProperties));
    config.getMapConfigs().put("com.company.distributed.domain.*", initializeDomainMapConfig(jHipsterProperties));

    return Hazelcast.newHazelcastInstance(config);
}

To check if the data-grid is working as expected, I've written a simple scheduled task that prints the current value of an atomic-long:

@Service
@Profile(JHipsterConstants.SPRING_PROFILE_PRODUCTION)
public class FetchHazelcastClusterSequence {

    private final HazelcastInstance hazelcastInstance;

    private static final Logger log = LoggerFactory.getLogger(FetchHazelcastClusterSequence.class);

    public FetchHazelcastClusterSequence(HazelcastInstance hazelcastInstance) {
        this.hazelcastInstance = hazelcastInstance;
    }

    @Timed
    @Scheduled(fixedDelay = 5000)
    public void start() {
        log.info("Fetching cluster sequence, the current value is: {}", this.hazelcastInstance.getAtomicLong("sequence").getAndIncrement());
    }
}

But as I've started the application, with a default configuration of 3 replicas, 3 pods were created, but each pod log showed as if it is the only member in the cluster and thus printing the same values of the "sequence".

Am I missing some configuration?