The hazelcast members are able to communicate with each other in non-kube environments. However, the same is not happening in Kube environments. The pod is not able to resolve its own domain.

I am working with Spring Boot 2.1.8 and Hazelcast 3.11.4 (TcpIp config)

Hazelcast configuration:

Config config = new Config();
config.setInstanceName("hazelcast-instance")
    .setGroupConfig(new GroupConfig(hazelcastGroupName, hazelcastGroupPassword))
    .setProperties(hzProps)
    .setNetworkConfig(
        new NetworkConfig()
            .setPort(hazelcastNetworkPort)
            .setPortAutoIncrement(hazelcastNetworkPortAutoIncrement)
            .setJoin(
                new JoinConfig()
                    .setMulticastConfig(new MulticastConfig().setEnabled(false))
                    .setAwsConfig(new AwsConfig().setEnabled(false))
                    .setTcpIpConfig(new TcpIpConfig().setEnabled(true).setMembers(memberList))))
    .addMapConfig(initReferenceDataMapConfig());

return config;

Members definition in Statefulset config file:

- name: HC_NETWORK_MEMBERS
  value: project-0.project-ha, project-1.project-ha

Headless service config:

---
apiVersion: v1
kind: Service
metadata:
  namespace: {{ kuber_namespace }}
  labels:
    app: project
    name: project-ha
  spec:
    clusterIP: None
  selector:
    app: project

Errors:

Resolving domain name 'project-0.project-ha' to address(es): [10.42.30.215]
Cannot resolve hostname: 'project-1.project-ha'
Members {size:1, ver:1} [ 
  Member [10.42.11.173]:5001 - d928de2c-b4ff-4f6d-a324-5487e33ca037 this
] 

The other pod has a similar error.

Fun facts:

• It works fine when I set the full hostname in the HC_NETWORK_MEMBERS var. For example: project-0.project-ha.namespace.svc.cluster.local, project-1.project-ha.namespace.svc.cluster.local