10:31 <hassan64> I need to convert my systemtap script to instrumentation module i.e *.ko . And then transfer this ELF binary to my target machine i.e ARRCH64.

10:31 <hassan64> I developed my target machine via buildroot

10:34 <hassan64> It is successfully compiled on my target machine, However, I used to create *.ko modules, I got the error . Please visit this link i.e https://gist.github.com/hassan113/9509989fc58bae3264e5795bd86b8cdd

11:23 <fche> hassan64, hm lots of things seem to have gone wrong there

11:23 <fche> run again with a bunch more -v

11:23 <fche> am suspicious that e.g. the -B CROSS_COMPILE= setting didn't work due to incomplete path names ("output/...")

11:34 <hassan64> fche: please visit the link again: https://gist.github.com/hassan113/9509989fc58bae3264e5795bd86b8cdd

11:35 <hassan64> Seems that "DebugFS " is required and is not found in the kernel

11:57 <hassan64> fche: Please help me

12:14 <fche> that setting is of your target cross kernel

12:17 <fche> can you get that CONFIG_DEBUG_FS or CONFIG_DEBUG_FS_MODULE enabled?

12:31 <hassan64> @fche: It compiled to *.ko now after enabling DEBUG FS. But getting warning. Please see my link. https://gist.github.com/hassan113/9509989fc58bae3264e5795bd86b8cdd

12:33 <fche> hm CONFIG_RELAY is also needed

12:33 <fche> (the README includes that list)

12:39 <hassan64> @fche: thanks a lot. Complied with no warning. and module loaded at target end successfullu

12:39 <fche> ah neat!

12:39 <fche> impressive how fast your rebuild was

12:39 <hassan64> well, in a few seconds

14:16 <hassan64> @fche: Now I am getting an error while using new script file. Previous on eis just to print "hello world". please llok at my github file. https://gist.github.com/hassan113/9509989fc58bae3264e5795bd86b8cdd

14:16 <hassan64> Thanks

14:17 <fche> what version of stap are you running? arm64 kernel syscall machinery changed not too long ago and stap had to play catch-up

14:20 <hassan64> Systemtap translator/driver (version 3.1/0.170, Debian version 3.1-3ubuntu0.1 (bionic-proposed))

14:21 <hassan64> I just install stap on my host machine via sudo apt

14:21 <hassan64> "sudo apt install systemtap"

14:56 <hassan64> I have updated my systemtap on my host machine Ubuntu:

14:56 <hassan64> Systemtap translator/driver (version 4.2/0.177, non-git sources)

14:58 <hassan64> fche: still the same error. Please visit my guthub: https://gist.github.com/hassan113/9509989fc58bae3264e5795bd86b8cdd

14:59 <hassan64> what should I do to have it compiled.

15:03 <fche> hmmmmm

15:10 <hassan64> got solution. recompile the kernel. Kernel hacking > Tracers > enable kprobes, ubrobes, gpio events

15:11 <hassan64> fche: But how to remove warning. Seems irritating

15:12 <fche> the 'cross-file ... ' ones?

15:12 <fche> that was just fixed in git-master stap a few days back

15:13 <hassan64> I checked out the systemTAP (4.2) from https://sourceware.org/systemtap/wiki/SystemTapReleases

15:13 <hassan64> seems a latest version

15:13 <fche> yeah, this fix is not yet released, but should be in a week or so

15:14 <hassan64> secondly, with latest elfutils ver (0.179), systemtap (4.2) wasn't compiled; showing some dependency check.

15:16 <mjw> could you show the dependency check failure?

15:18 <hassan64> mjw: ok just wait for while

15:47 <hassan64> mjw: configuration process ran fine i.e ./configure --with-elfutils=$(pwd)/../elfutils-0.179 . But when do "make",there is error. please see my github " https://gist.github.com/hassan113/9509989fc58bae3264e5795bd86b8cdd

15:48 <hassan64> systemtap ver = 4.2

15:52 <mjw> hassan64, that error is "correct" libebl was removed in the latest elfutils. It is an internal only "library"

15:53 <mjw> Let me see why systemtap tries to use it. That isn't really supported.

15:54 <mjw> aha, for the stringtab functions

15:54 <mjw> which are part of libdw since... 0.167

15:55 <fche> the configure check looks for ebl_get_elfmachine

15:56 <fche> maybe it finds it in -ldw first and assumes it came from -lebl ?

15:56 <fche> hassan64, can you paste your config.log ?

15:57 <fche> here it says:

15:57 <fche> configure:11503: checking for ebl_get_elfmachine in -lebl

15:57 <fche> configure:11528: gcc -o conftest -fstack-protector-all -D_FORTIFY_SOURCE=2 -g -O2 -I/home/fche/Private/DEVEL/DEVEL-sourceware/elfutils2-BUILD/INST/include -L/home/fche/Private/DEVEL/DEVEL-sourceware/elfutils2-BUILD/INST/lib conftest.c -lebl >&5

15:57 <fche> /usr/bin/ld: cannot find -lebl

15:57 <fche> which is good news

15:58 <fche> maybe hassan64 didn't rerun a full configure?

15:58 <mjw> fche, I think it should check for ebl_strtabinit instead

15:59 <mjw> Since that is the function it really needs (when using elfutils < 0.167)

15:59 <fche> suppose that could be

15:59 <fche> but I don't see that test failing here

16:00 <mjw> fche, the configure check might be trying the system libebl... :{

16:00 <fche> as in failing to behave as expected

16:00 <fche> hm lemme find the other one

16:00 <fche> configure:11503: checking for ebl_get_elfmachine in -lebl

16:00 <fche> configure:11528: gcc -o conftest -fstack-protector-all -D_FORTIFY_SOURCE=2 -g -O2 conftest.c -lebl >&5

16:00 <fche> /usr/bin/ld: cannot find -lebl

16:00 <fche> collect2: error: ld returned 1 exit status

16:00 <fche> configure:11528: $? = 1

16:02 <mjw> Let me try on a RHEL7 setup, that still has a system libebl installed

16:05 <mjw> configure:11424: checking for ebl_get_elfmachine in -lebl

16:05 <mjw> configure:11449: gcc -o conftest -fstack-protector-all -D_FORTIFY_SOURCE=2 -g -O2 conftest.c -lebl >&5

16:05 <mjw> configure:11458: result: yes

16:05 <mjw> configure:11449: $? = 0

16:05 <mjw> boo!

16:05 <fche> ok why is that a problem?

16:06 <mjw> because it will then... hmmm... link with -lebl

16:06 <fche> on rhel7, ok, is that a problem?

16:06 <mjw> even though I gave it --with-elfutils=/tmp/elfutils-0.179

16:07 <fche> new stap doesn't do bundled elfutils any more

16:07 <mjw> but you are right, that -lebl should work on this setup

16:07 <fche> so --with-elfutils is ignored

16:07 <mjw> fche, aha!

16:07 <fche> if you want to use it, LDFLAGS= etc. env vars for configure

16:08 <mjw> but that is how hassan64 configures

16:08 <mjw> and the configure check is still wrong :)

16:08 <fche> "that is how hassan64 configure" ah yes, didn't spot that

16:08 <fche> README advises

16:09 <fche> % .../configure [other autoconf options]

16:09 <fche> to locate libraries in non-system directories.

16:09 <fche> Add env LDFLAGS=-L/path/ CPPFLAGS=-I/path/ before configure

16:09 <mjw> yes, that is sane. But I must admit I also missed that.

16:11 <mjw> BTW. The configure check is there for a bug in the debian packages from 2007...

16:11 <mjw> It should have been resolved with their packaging of 0.141 in 2009

16:11 <fche> then heck nuke the autoconf clause

16:12 <mjw> I am not sure systemtap even supports compiling against that old elfutils

16:12 <mjw> fche, it is still needed for working against elfutils < 0.167 when you compile in BPF support

16:13 <mjw> sorry, dinner time

16:14 <mjw> But I think the check is still needed, but against ebl_strtabinit instead

16:15 <fche> ok, will take your word for it and adjust

16:17 <irker456> systemtap: fche systemtap.git:master * release-4.2-106-ga1bbefd79 / configure configure.ac: configury: adjust -lebl check to ebl_strtabinit

17:16 <mjw> :)

17:46 <hassan64> fche: Please visit the link to see config.log https://gist.github.com/hassan113/9509989fc58bae3264e5795bd86b8cdd

17:49 <fche> hassan64, suggest building this version of elfutils separately

17:49 <hassan64> fche & mjw Any fix against

17:49 <fche> so build it first, install it somewhere

17:50 <fche> then run the systemtap configury with LDFLAGS=-L/path/to/elfutils/libs CPPFLAGS=-I/path/to/elfutils/include

17:50 <fche> in the environment

17:50 <fche> i.e., don't use --with-elfutils

17:51 <hassan64> fche: can you paste the procedure of compiling latest system tap version with latets elfutils files

17:51 <hassan64> it is btw, common practice to compile systemtap through elfutilts by doing ./configure --with-elfutils=<ELFUTILS_DIR>

17:53 <fche> you can encounter problems that way; we have not tested this method in a while, and have removed it altogether in the master version

17:57 <hassan64> Can this issue be resolved in upcoming systemtap version with latest elfutils. Plus I encounter the WARNING messages in doing "stap". Will it be solved in upcoming versions

17:57 <fche> depends on which warning; that cross-file global reference one, yes.

17:58 <fche> the issue you're encountering with --with-elfutils builds will go away by virtue of that option going away

17:58 <fche> but even with the 4.2 released copy of systemtap, you can use other ways to build against a private installation of elfutils:

17:58 <fche> <fche> so build it first, install it somewhere

17:58 <fche> <fche> in the environment

17:58 <fche> <fche> then run the systemtap configury with LDFLAGS=-L/path/to/elfutils/libs CPPFLAGS=-I/path/to/elfutils/include

17:58 <fche> <fche> i.e., don't use --with-elfutils

18:07 <hassan64> fche Thanks

18:48 <agentzh> hi guys, is it a known issue that the stap's userland stack unwinder may result in incorrect backtraces? some times it shows a function calls another but it actually never does as per the source code. gdb produces the correct one when i verified some of these cases myself.

18:49 <agentzh> it happens quite often.

18:49 <agentzh> against different target C applications.

18:49 <fche> not familiar

18:51 <agentzh> k, we'll dig deeper into the unwinder.

19:39 <irker456> systemtap: smakarov systemtap.git:master * release-4.2-107-g1eaeb16df / buildrun.cxx runtime/linux/autoconf-kallsyms-on-each-symbol.c runtime/linux/kprobes.c runtime/linux/runtime.h runtime/sym.c runtime/sym.h runtime/transport/symbols.c runtime/transport/transport.c runtime/transport/transport.h runtime/transport/transport_msgs.h staprun/staprun.c translate.cxx: PR26074: pass kallsyms_lookup_name as a relocation to the stap runtime