19:28 <agentzh> fche: happy new year!

19:28 <agentzh> i wonder if stap is supposed to support those "RT" linux kernels.

19:28 <agentzh> seems like a lot of problems there.

19:40 <fche> hi

19:41 <fche> yeah "supposed to" ... well, I never liked that phrase

19:41 <fche> it would be nice

19:49 <agentzh> we've been regressing all those debian/ubuntu/rhel/centos/oracle/amazon/sles/opensuse/etc kernels with our stap fuzzer.

19:49 <agentzh> some are known kernel regressions. some are stap bugs.

19:49 <agentzh> some are still being investigated.

19:49 <fche> neat

19:49 <agentzh> rt kernels have so many errors.

19:50 <agentzh> hence my question above. sorry if it sounds offensive ;)

19:50 <fche> not at all

19:51 <fche> I just fight with passive-voice directives like "supposed to" all the time :)

19:51 <agentzh> heh, okay, understood.

19:52 <agentzh> we're just completed the first round of fuzz testing, covered 7k+ kernel binaries.

19:52 <agentzh> *we've

19:53 <agentzh> now investigating all the failures.

19:53 <fche> neat

20:13 <irker060> systemtap: alizhang systemtap.git:pr13838/fpsupport * release-4.4-41-gf9e223393 / runtime/loc2c-runtime.h: runtme/loc2c-runtime.h: extended x64_64 register to 24 to allow access of xmm0-xmm7 register.

20:37 <kerneltoast> fche, bonjour

20:38 <kerneltoast> how do you feel about hiding these messages for NULL frees: "ERROR: Free of unallocated memory (null) type=vfree"

20:38 <kerneltoast> since vfree(NULL) and kfree(NULL) are safe

20:38 <kerneltoast> and in fact some of the code i punted into stap relies on this

20:39 <fche> what prints those messages?

20:39 <kerneltoast> i don't know which particular piece of code produced that message we found, but i know of one piece of code that will do that

20:39 <kerneltoast> _stp_runtime_contexts_alloc() failing an allocation

20:40 <irker060> systemtap: alizhang systemtap.git:pr13838/fpsupport * release-4.4-42-g040e99332 / main.cxx testsuite/systemtap.examples/general/floatingpoint.stp: main.cxx: delete parameter i_temp, which was used for debugging; testsuite/systemtap.examples/general: update examples, using $fp instead of user_long($fpointer)

20:40 <kerneltoast> then _stp_runtime_contexts_free() will be called to free a bunch of NULL pointers

20:42 <fche> _stp_mem_debug_free << y'all using this stuff?

20:43 <fche> could hard-code it or its caller whereever that is to quietly permit addr==NULL

20:43 <kerneltoast> yeah we're using it

20:43 <irker060> systemtap: scox systemtap.git:master * release-4.4-51-g5114b474f / buildrun.cxx elaborate.cxx loc2stap.cxx runtime/dyninst/print.c runtime/dyninst/sym.c tapset/dyninst/errno.stpm tapset/linux/tls.stp tapset/tls.stp testsuite/systemtap.base/externalvar.exp testsuite/systemtap.base/tls.exp: Add stapdyn VMA-tracking.

20:44 <fche> yeah given the parallels to kfree() vfree() setc. in _stp_*free(), yeah would suggest teaching the _debug_free to just accept & ignore the NULL

20:44 <kerneltoast> k

20:44 <kerneltoast> time to teach _stp_*free() a lesson

20:44 <kerneltoast> "how many times do we need to teach you this lesson, old man?!"

20:45 <kerneltoast> https://youtu.be/sVxJ016xb4Q

20:49 <fche> nice

20:57 <kerneltoast> fche, how's this: https://paste.centos.org/view/1e2e8dcc

20:57 <kerneltoast> i made it explicit for each allocation type in case the mem debug code gets support for kmem caches in the future or something

20:58 <kerneltoast> since kmem_cache_free() won't tolerate NULL pointers

21:02 <fche> sure

21:03 <kerneltoast> noice

21:10 <irker060> systemtap: sultan systemtap.git:master * release-4.4-52-g798ec0f9c / runtime/linux/alloc.c: Don't warn about freeing a NULL pointer for functions that tolerate it

21:43 <kerneltoast> fche, i see you are familiar with fs code

21:44 <kerneltoast> our fuzzer hit a crash on some ubuntu kernels where __stp_debugfs_root_dir->d_inode was NULL

21:44 <kerneltoast> and i have no clue why

21:45 <kerneltoast> this is the crash: https://gist.github.com/kerneltoast/0adfb2f3e178faf8fd0ff7a66914c2e8

21:51 <fche> no prior error messages in dmesg?

21:51 <kerneltoast> just the ones i already included

21:54 <kerneltoast> this is the source tree for one of the kernels in question: https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic/tree/?h=Ubuntu-hwe-edge-5.3.0-21.22_18.04.1

21:55 <kerneltoast> it matches with that dmesg output in my gist

22:30 <fche> kerneltoast, is this problem reproducible enough to try a different stap configuration quickly?

22:30 <fche> -DSTAP_TRANS_PROCFS ?

23:58 <kerneltoast> fche, yeah it's very reproducible, though I'm not the one reproducing it. i can relay that on to see if it fixes it though