TCP socket connection interference on send message in Kubernetes

7/24/2020

I'm creating an application in C that intermediates and manages scheduled applications (NodeJs) to collect information into MongoDB.

This makes possible the application's scalability and avoids replicated values.

Problem

  • Local tests: OK
  • Container tests: Ok
  • K8s Minikube tests: scheduled application error in bson parse
  • EKS Production tests: scheduled application error in bson parse

Code of C application to send socket message:

static void
ssq_connection_server_handler()
{
    ...

    for (i = 0; i < ssq_max_sockets; i++) {
        sd = ssq_connection->clients[i];
        if (FD_ISSET(sd, &master)) {
            if ((ret = ssq_connection->recv(sd, buffer, SSQ_SERVER_MAX_BUFFER_SIZE)) <= 0) {
                continue;
            }

            buffer[ret] = '\0';
            token = ssq_strtok(buffer, ":");
            opts = ssq_strtok(NULL, ":");

            limit = ssq_atoi(opts, ssq_strlen(opts));
            if (limit == SSQ_ERROR) {
                limit = 10;
            }

            if (ssq_strcmp("next", token) == 0) {
                u_char *tasks;
                size_t len;

                tasks = ssq_mongo_collection_find(limit, ssq_mongo->collection, &len);
                ssq_connection->send(sd, tasks, len);
            }
        }
    }

    ...
}

static ssize_t
ssq_connection_send_handler(int fd, u_char *buf, size_t size)
{
    ssize_t n;
    ssq_err_t err;

    for (;;) {
        n = send(fd, buf, size, 0);

        ssq_log_debug("send: fd:%d %z of %uz", fd, n, size);

        if (n > 0) {
            return n;
        }

        err = ssq_socket_errno;

        if (n == 0) {
            ssq_log_error(SSQ_LOG_ALERT, err, "send() returned zero");
            return n;
        }

        if (err == SSQ_EAGAIN || err == SSQ_EINTR) {
            ssq_log_debug_error(err, "send() not ready");

            if (err == SSQ_EAGAIN) {
                return SSQ_AGAIN;
            }
        } else {
            return SSQ_ERROR;
        }
    }
}

Code in NodeJs to receive message:

const net = require('net');
const { EJSON } = require('bson');

const sock = new net.Socket();

sock.connect(process.env.SSQ_PORT || 3145, process.env.SSQ_HOST || '127.0.0.1', () => {
    console.log('socket connection established');
});

sock.on('data', (data) => {
    try {
        const tasks = EJSON.parse(data);
        console.log(tasks);
        process.exit(0);
    } catch (err) {
        console.error(err);
        process.exit(1);
    }
});

sock.on('error', () => {
    /* void */
});

sock.on('close', () => {
    console.warn('socket connection has been closed');
});

sock.write(Buffer.from('next:10'));

I think that something coming from K8s is intercepting and modifying the message.

-- Placidina
c
kubernetes
sockets

0 Answers