perl 5.26.2 crashes with SIGSEGV (exit code 139) in ubuntu container

1/23/2020

I have a confusing perl behaviour. From time to time (several times a day) it crashes when running inside ubuntu 18.04 kubernetes container with SIGSEGV (exit code 139). A coredump reveals some strange error like below:

Type "apropos word" to search for commands related to "word"...
Reading symbols from /opt/mettools/bin/perl...done.
[New LWP 2218]
[New LWP 1]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `perl /opt/mettools/bin/s4p-server-http.pl -a'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  __GI_getenv (name=0x7f34b8d9b226 "", name@entry=0x7f34b8d9b224 "TZ") at getenv.c:84
84      getenv.c: No such file or directory.
[Current thread is 1 (Thread 0x7f34b79ab700 (LWP 2218))]
(gdb) bt
#0  __GI_getenv (name=0x7f34b8d9b226 "", name@entry=0x7f34b8d9b224 "TZ") at getenv.c:84
#1  0x00007f34b8cba0db in tzset_internal (always=1) at tzset.c:378
#2  __tzset () at tzset.c:552
#3  0x00005650223b41ef in Perl_localtime64_r ()
#4  0x00005650223698c5 in Perl_pp_gmtime ()
#5  0x000056502230c5b6 in Perl_runops_standard ()
#6  0x000056502228586f in Perl_call_sv ()
#7  0x00007f34b9aba93b in S_jmpenv_run () from /opt/mettools/lib/5.26.2/x86_64-linux-thread-multi/auto/threads/threads.so
#8  0x00007f34b9abab9d in S_ithread_run () from /opt/mettools/lib/5.26.2/x86_64-linux-thread-multi/auto/threads/threads.so
#9  0x00007f34b97b86db in start_thread (arg=0x7f34b79ab700) at pthread_create.c:463
#10 0x00007f34b8d0788f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Does anybody have an idea where to look at for that kind of error? The perl script is an API server running in the container. Thanks to kubernetes nature it restarts the container every time the perl crashes (5-6 failures a day), but of course it's annoying to see such a buggy system. :)

The system is:

  • ubuntu 18.04
  • perl 5.26.2 (from anaconda conda-forge channel)
  • base module for the server is HTTP::Server::Simple::CGI 0.52

I don't know if it relates or just a coincidence, but running the same docker container in a similar docker swarm infrastructure helps to increase the life time of the container up to several weeks and sometimes months. Pls share your thoughts!

-- novak Li
crash
docker
kubernetes
perl
ubuntu

0 Answers