I'm setting up a new k8s environment with multiple pods running microservices written in node.js. Several of these services connect to a redis cache.

This is all working most of the time, but I am receiving intermittent errors when accessing redis that make me believe that I'm not connecting correctly, most commonly:

RedisServerException: READONLY You can't write against a read only slave.

If I try again, I am often successful after two or three attempts.

This is my redis demployment:

RESOURCES:

==> v1/Service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE cache-redis-ha-announce-0 ClusterIP 100.xxx.xxx.xxx <none> 6379/TCP,26379/TCP 163m cache-redis-ha-announce-1 ClusterIP 100.xxx.xxx.xxx <none> 6379/TCP,26379/TCP 163m cache-redis-ha-announce-2 ClusterIP 100.xxx.xxx.xxx <none> 6379/TCP,26379/TCP 163m cache-redis-ha ClusterIP None <none> 6379/TCP,26379/TCP 163m

==> v1/StatefulSet NAME DESIRED CURRENT AGE cache-redis-ha-server 3 3 94s

==> v1/Pod(related) NAME READY STATUS RESTARTS AGE cache-redis-ha-server-0 2/2 Running 0 94s cache-redis-ha-server-1 2/2 Running 0 64s cache-redis-ha-server-2 2/2 Running 0 36s

==> v1/ConfigMap NAME DATA AGE cache-redis-ha-configmap 3 163m cache-redis-ha-probes 2 163m

NOTES: Redis can be accessed via port 6379 and Sentinel can be accessed via port 26379 on the following DNS name from within your cluster: cache-redis-ha.devtest.svc.cluster.local

In my service, I connect to redis thus:

this.client = redis.createClient(6379, "cache-redis-ha.devtest.svc.cluster.local");


this.delAsync = promisify(this.client.del).bind(this.client);


async flush(matchPattern: string): Promise<CacheResult> {
    let result: CacheResult = { matchPattern: matchPattern, reply: true };
    return await this.keysAsync(matchPattern).then(async (keys) => {
        result.matchedKeys = keys;
        if (keys.length) {
            return await this.delAsync(keys).then((reply) => {
                result.reply = reply;
                return result;
            });
        }
        return result;
    });
}

I tried to connecting to the Sentinel port in createClient, but this didn't work.

Is there something obviously wrong in my implementation?