Hazecast-Kubernetes Springboot application embedded cluster warning

10/29/2020

i have deploy embedded hazelcast cluster on kubernetes, i am getting below warning. disabled auto increment flag and portCount still getting error.

Config :

Config config = new Config();
config.setClusterName("cache-cluster");
JoinConfig joinConfig = config.getNetworkConfig().getJoin();
joinConfig.getMulticastConfig().setEnabled(false);
joinConfig.getTcpIpConfig().setEnabled(false);
joinConfig.getKubernetesConfig().setEnabled(true)
            .setProperty("namespace", "<Namespace>")
            .setProperty("service-name", "<Service-name>");

i have opened container port in kubernetes. services are able to create the cluster but getting the below warning continuously.

2020-10-29 14:04:40.092  WARN  [hz.silly_lalande.IO.thread-in-1    ]  c.h.i.n.t.TcpIpConnection                    - [22.249.47.234]:5701 [cache-cluster] [4.0.1] Connection[id=2, /22.249.47.234:5701->/22.249.5.1:52152, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Exception in Connection[id=2, /22.249.47.234:5701->/22.249.5.1:52152, qualifier=null, endpoint=null, alive=true, connectionType=NONE], thread=hz.silly_lalande.IO.thread-in-1 
java.lang.IllegalStateException: Unknown protocol: ?
	at com.hazelcast.internal.nio.tcp.UnifiedProtocolDecoder.onRead(UnifiedProtocolDecoder.java:116)
	at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:137)
	at com.hazelcast.internal.networking.nio.NioPipeline.lambda$start$0(NioPipeline.java:127)
	at com.hazelcast.internal.networking.nio.NioThread.processTaskQueue(NioThread.java:354)
	at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:289)
	at com.hazelcast.internal.networking.nio.NioThread.run(NioThread.java:248)
2020-10-29 14:13:10.038  WARN  [hz.silly_lalande.IO.thread-in-2    ]  c.h.i.n.t.TcpIpConnection                    - [22.249.47.234]:5701 [cache-cluster] [4.0.1] Connection[id=3, /22.249.47.234:5701->/22.249.55.1:52730, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Exception in Connection[id=3, /22.249.47.234:5701->/22.249.55.1:52730, qualifier=null, endpoint=null, alive=true, connectionType=NONE], thread=hz.silly_lalande.IO.thread-in-2 
java.lang.IllegalStateException: TLS handshake header detected, but plain protocol header was expected.
	at com.hazelcast.internal.nio.tcp.UnifiedProtocolDecoder.loadProtocol(UnifiedProtocolDecoder.java:134)
	at com.hazelcast.internal.nio.tcp.UnifiedProtocolDecoder.onRead(UnifiedProtocolDecoder.java:90)
	at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:137)
	at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:382)
	at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:367)
	at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:293)
	at com.hazelcast.internal.networking.nio.NioThread.run(NioThread.java:248)
2020-10-29 14:13:10.039  WARN  [hz.silly_lalande.IO.thread-in-0    ]  c.h.i.n.t.TcpIpConnection                    - [22.249.47.234]:5701 [cache-cluster] [4.0.1] Connection[id=4, /22.249.47.234:5701->/22.249.55.1:52736, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Exception in Connection[id=4, /22.249.47.234:5701->/22.249.55.1:52736, qualifier=null, endpoint=null, alive=true, connectionType=NONE], thread=hz.silly_lalande.IO.thread-in-0 
java.lang.IllegalStateException: TLS handshake header detected, but plain protocol header was expected.
	at com.hazelcast.internal.nio.tcp.UnifiedProtocolDecoder.loadProtocol(UnifiedProtocolDecoder.java:134)
	at com.hazelcast.internal.nio.tcp.UnifiedProtocolDecoder.onRead(UnifiedProtocolDecoder.java:90)
	at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:137)
	at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:382)
	at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:367)
	at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:293)
	at com.hazelcast.internal.networking.nio.NioThread.run(NioThread.java:248)

Not sure why connection is happing here

[22.249.47.234]:5701 [cache-cluster] [4.0.1] Connection[id=4, **/22.249.47.234:5701->/22.249.55.1:52736**, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Exception in Connection[id=4, **/22.249.47.234:5701->/22.249.55.1:52736**, qualifier=null, endpoint=null, alive=true, connectionType=NONE], thread=hz.silly_lalande.IO.thread-in-0 

is there any configuration missing?

-- MAYUR KUMBHAR
hazelcast
kubernetes
spring-boot
spring-cache

1 Answer

11/13/2020

This looks similar to https://github.com/hazelcast/hazelcast-kubernetes/issues/186 Adding containerPort: 5701 to your HZ service manifest should work.

It should work irrespective of the type of service whether it is LoadBalancer or ClusterIP

Can you confirm clustering is working fine too even in the case of warnings?

-- vivek_vara
Source: StackOverflow