I'm using Cloud Composer on GCP and having trouble diagnosing what's causing a trend I see below.

The first screenshot is of my memory utilization per node in my Kubernetes cluster from the Composer Environments Monitoring tab.

I'm confused, because when I run kubectl top nodes it doesn't look like the nodes are using 20+gb of memory.

Can anyone help me understand what the first screenshot is telling me in relation to the top output? Things that I've checked so far to help understand what might be causing the slow increase in memory:

• Confirmed remote logs are being written to GCS and not locally
• No tasks are writing files to the local file system, they're using the FUSE mounted buckets for storage.

Thanks for your thoughts