nodejs http client stuck on high traffic

3/27/2020

i am encountering weird behavior which i cant explain, i have GKE cluster of 1 instance (2 cpu, 4gb memory) which contain 1 service of nodeJS application with 2 replicas which contain 1 function:

        const req = client.get("https://myextrenalapi.com", opts, (res) => {
            let responseBody = ''
            res.socket.setNoDelay();
            res.on('data', (chunk) => { responseBody += chunk })
            res.on('end', () => {
                return responseBody;
            })
        })

now after starting to loadtest this server with approx 130 RPS it starts to get slow and http client request takes 10+ seconds (300ms idle).

BUT when i try to CURL from within one of my nodes (while doing the stress test) to: https://myextrenalapi.com the requests takes 300MS, so how come curl is doing it in 300MS while to nodeJS http client it takes 10+ seconds?

i added logging to the http client while running the stress test:

{"dnsLookup":194,"tcpConnection":6875,"tlsHandshake":9206,"firstByte":10746,"end":10746}

any ideas why its so slow campers to CURL on highload? i tried almost everything in nodeJS side, i changed maxTcpConnection, keep-alive, removed agent completely but results are almost the same in all senarios

-- roy
http
kubernetes
linux
networking
node.js

0 Answers