http-proxy (reverse proxy) server hangs up in a random time

1/27/2020

Problem: When I send a request after this log (memory), it shows "socket hang up (connResetException)" error. And the NodeJS server stops working.

[1:0x330e8a0]    22724 ms: Mark-sweep 16.4 (25.2) -> 10.6 (29.9) MB,
 3.3 / 0.0 ms  (+ 0.1 ms in 2 steps since start of marking, biggest step 0.1 
ms, walltime since start of marking 7 ms) (average mu
 = 0.999, current mu = 1.000) finalize incremental marking
 via task GC in old space requested

Architecture: I use http-proxy as a reverse proxy. It enables the SSO(Single-Sign-On) and sends its traffics to the application.

Error Log in Proxy Server(Only happens in the production - more traffics)

/node_modules/http-proxy/lib/http-proxy/index.js:120
    throw err;
    ^

Error: socket hang up
    at connResetException (internal/errors.js:561:14)
    at Socket.socketCloseListener (_http_client.js:380:25)
    at Socket.emit (events.js:214:15)
    at TCP.<anonymous> (net.js:658:12) {
  code: 'ECONNRESET'
}

Since socket hangs up happens with MANY cases, I researched and experimented with various cases. And I'm thinking the memory leak issue from my node can be the issue. node --trace_gc src/index.js commands print 2~3 allocation failures per request.

Weird logs

 [1:0x449d780]    20364 ms: Scavenge 11.9 (13.2) -> 11.3 (13.2) MB, 3.2 / 0.0 ms \
     (average mu = 0.961, current mu = 0.961) allocation failure 

Source Code

var apiProxy = httpProxy.createProxyServer();
app.use(cookieParser());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json())

app.use(passport.initialize({}));
app.use(passport.session({}));
app.use(session({
    secret: 'secret',
    resave: false,
    saveUninitialized: true,}
));

app.get('/source*',
    function(req, res, next) {
        req.query.RelayState = req.url;
        if(req.user) {
            apiProxy.web(req, res, {target: xx});

Problem: [http-proxy expressjs] hangs up in a random time. It usually works for two days and the server goes 502 Error. I need to manually restart the container to restore it.

/node_modules/http-proxy/lib/http-proxy/index.js:120
    throw err;
    ^

Error: socket hang up
    at connResetException (internal/errors.js:561:14)
    at Socket.socketCloseListener (_http_client.js:380:25)
    at Socket.emit (events.js:214:15)
    at TCP.<anonymous> (net.js:658:12) {
  code: 'ECONNRESET'
}

Any idea is welcome... please help

-- hellofanengineer
express
http-proxy
http-proxy-middleware
kubernetes
node.js

0 Answers