Mongo Memory Usage Is Growing Up In Kubernetes

2/13/2020

I have a kubernetes cluster and i am running one instance of mongodb as a statefulset.I've written a tool that raplicates some collection to a remote mongodb(because mongodb does not have collection replication).I know mongo sometimes can use too much memory and release it when it does not need that memory.I think it uses swap for achieving this release(In a kubernetes cluster swap is always disabled.).But in kubernetes the memory usage of mongo is always growing up and it is not release any memory.I set memory limit to 16 Gi but no chance.I try to use limit wiredtiger to 4 Gi but nothing has changed.Any help will be appreciated.

When i look malloc it shows(but inside of container i can see mongod is using 7 Gi Memory):

var mem = db.serverStatus().tcmalloc;
mem.tcmalloc.formattedString

------------------------------------------------
MALLOC:     3507618112 ( 3345.1 MiB) Bytes in use by application
MALLOC: +    636588032 (  607.1 MiB) Bytes in page heap freelist
MALLOC: +     14434728 (   13.8 MiB) Bytes in central cache freelist
MALLOC: +      2499328 (    2.4 MiB) Bytes in transfer cache freelist
MALLOC: +     35990040 (   34.3 MiB) Bytes in thread cache freelists
MALLOC: +     15204352 (   14.5 MiB) Bytes in malloc metadata
MALLOC:   ------------
MALLOC: =   4212334592 ( 4017.2 MiB) Actual memory used (physical + swap)
MALLOC: +    296247296 (  282.5 MiB) Bytes released to OS (aka unmapped)
MALLOC:   ------------
MALLOC: =   4508581888 ( 4299.7 MiB) Virtual address space used
MALLOC:
MALLOC:          52538              Spans in use
MALLOC:            124              Thread heaps in use
MALLOC:           4096              Tcmalloc page size
------------------------------------------------
Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()).
Bytes released to the OS take up virtual address space but no physical memory.

When i look to monitoring it gives me all of the slowest queries are from my replication tool but when i stop replication,situation is the same memory is growing up.Also my replication tool is using mongo watch:

https://docs.mongodb.com/manual/reference/method/db.collection.watch/

https://docs.mongodb.com/manual/reference/method/Mongo.watch/

-- akuscu
kubernetes
memory-management
mongodb

0 Answers