Sometimes when running certain builds via self-hosted GitLab CI in a Kubernetes cluster the runners fail with the K8s event The node was low on resource: inodes. Upon checking the respective nodes inodes I don't see any problem. This occurs only on specific CI runs which seem to install a lot of files with npm. Sadly the Node Exporter always reports zero free inodes with its metric container_fs_inodes_free. Since the inode-usage on the node itself is very low I don't know how to further debug this problem. Do you have ideas/hints I can follow?

All nodes are set up with VMWare.

Here is the build-log:

$ npx lerna run typecheck --scope=myproduct --stream
lerna notice cli v3.22.1
lerna info versioning independent
lerna info ci enabled
lerna notice filter including "myproduct"
lerna info filter [ 'myproduct' ]
lerna info Executing command in 1 package: "yarn run typecheck"
myproduct: yarn run v1.22.5
myproduct: $ tsc --noEmit
Cleaning up file based variables
ERROR: Job failed: command terminated with exit code 137

Here is the output of df -i

Filesystem                   Inodes  IUsed    IFree IUse% Mounted on
devtmpfs                    4079711    454  4079257    1% /dev
tmpfs                       4083138      1  4083137    1% /dev/shm
tmpfs                       4083138   1738  4081400    1% /run
tmpfs                       4083138     17  4083121    1% /sys/fs/cgroup
/dev/mapper/vg00-lv_root     327680  93629   234051   29% /
/dev/mapper/vg00-lv_opt      131072  18148   112924   14% /opt
/dev/mapper/vg00-lv_var      327680  16972   310708    6% /var
/dev/sda1                    128016    351   127665    1% /boot
/dev/mapper/vg00-lv_home      32768     73    32695    1% /home
/dev/mapper/vg00-lv_tmp       65536     53    65483    1% /tmp
/dev/mapper/vg01-lv_docker  3211264 409723  2801541   13% /var/lib/docker
/dev/mapper/vg00-lv_varlog    65536    283    65253    1% /var/log