Kubernetes Service, Redis Sentinel , is not returning Redis Master IP & Port consistently and all the pods are in docker container. I have used both https://github.com/kubernetes/kubernetes/tree/master/examples/redis & https://github.com/helm/helm - redis cluster

When I connect to the Kubernetes Service (10.3.251.152 , 26379/TCP) it should determine the Redis Sentinel master (ip: '10.0.0.68', port: '6379') and then make a connection with it. But sometimes, Kubernetes service returns (ip: '10.0.0.68', port: '6379') and some other times, it returns Kubernetes Service (10.3.251.152 , 26379/TCP) as sentinel master IP. This much we can see in the program Log to connect with sentinel master

In both ( kubernetes sentinel example , helm.sh's redis cluster) cases this inconsistency happens.

And you can see the service and pod details here ..

How do I do the troubleshooting?

useRedisSentinel  true
redisSentinelHost  10.3.251.152 redisSentinelPort  26379
Queueing psubscribe for next server connection.
sentinelClient  { domain: null,
  _events: 
   { 'sentinel disconnected': [Function],
     'failover start': [Function],
     'switch master': [Function] },
  _maxListeners: undefined,
  options: 
   { master_debug: true,
     port: '26379',
     host: '10.3.251.152',
    and so on .. 
   }
  }

More details about the master / slave / sentinel details

masters  [ { name: 'mymaster',
    ip: '10.0.0.68',
    port: '6379',
    runid: '33798857c71ebf2c85c7095291d453cb143cea7e',
    flags: 'master',
    'pending-commands': '0',
    'last-ok-ping-reply': '0',
    'last-ping-reply': '518',
    'info-refresh': '518',
    'num-slaves': '60000',
    'num-other-sentinels': '9021',
    quorum: 'master' } ]
mymaster slaves  [ { name: '10.0.0.70:6379',
    ip: '10.0.0.70',
    port: '6379',
    runid: 'ce2330d9071e091e8d808017d761cbdae1d286b9',
    flags: 'slave',
    'pending-commands': '0',
    'last-ok-ping-reply': '0',
    'last-ping-reply': '518',
    'info-refresh': '518',
    'master-link-down-time': '60000',
    'master-link-status': '9021',
    'master-host': 'slave',
    'master-port': '33297880',
    'slave-priority': '0' },
  { name: '10.0.0.69:6379',
    ip: '10.0.0.69',
    port: '6379',
    runid: 'a1e063c71a4ef93a8c5769aa29cb1d8dbf669579',
    flags: 'slave',
    'pending-commands': '0',
    'last-ok-ping-reply': '0',
    'last-ping-reply': '518',
    'info-refresh': '518',
    'master-link-down-time': '60000',
    'master-link-status': '9021',
    'master-host': 'slave',
    'master-port': '33297888',
    'slave-priority': '0' } ]
mymaster sentinels  [ { name: '10.0.0.71:26379',
    ip: '10.0.0.71',
    port: '26379',
    runid: '6f787783753fa446c6ecb118d07339d51f1fda14',
    flags: 'sentinel',
    'pending-commands': '0',
    'last-ok-ping-reply': '0',
    'last-ping-reply': '518',
    'last-hello-message': '518',
    'can-failover-its-master': '60000' },
  { name: '10.0.0.68:26379',
    ip: '10.0.0.68',
    port: '26379',
    runid: '7542fbbb0c36c4d171c10b492f67a416d794fc96',
    flags: 'sentinel',
    'pending-commands': '0',
    'last-ok-ping-reply': '0',
    'last-ping-reply': '518',
    'last-hello-message': '518',
    'can-failover-its-master': '60000' } ]
mymaster getMasterAddress  { ip: '10.0.0.68', port: '6379' }

How do I do the troubleshooting?