09:00 <invano> morning everyone. I got a staprun related question for you guys. I'm using a stap script to monitor specific processes. I run staprun on a VM through ssh and then, again by using ssh, I run the process I want to monitor. The problem is that sometimes I cannot log anything at all. I found out (thanks -vv) that the stap module is ready after printing "stapio:stp_main_loop:... systemtap_module_init() returned 0". However this is not enough. 20% of the times

09:00 <invano> staprun still misses the execution of the new process. I'm wondering if there is something I'm missing...and hope I've been clear enough

10:45 <fche> invano, hi

10:46 <fche> doesn't sound familiar to stap versions in the last year or two; before then we had an occasional bug where not all target processes would be found

10:46 <fche> did stap report anything about missed probes in the 20% cases, at shutdown?

10:47 <fche> (by the way, stap --remote=HOST can help cross-compile, send, and execute a script via ssh

12:53 <invano> fche: hi. no, no missed probes in that case. It's like systemtap doesn't see at all the process.

12:53 <fche> sounds odd. it shouldn't matter, but how soon after the stap script startup do you start the target process?

12:53 <invano> As I told you, if I run the process after "stapio:stp_main_loop:... systemtap_module_init() returned 0" it gets better and better

12:54 <invano> I tried to wait 1, 5, or even 30 secs. What's really important is to wait for that string logged by staprun and then start the process

12:56 <invano> For reference, I'm using a modified version of strace.stp in which I still have the forked set concept and I filter also per uid of the process.

13:00 <fche> ok. how old is this version of stap? (stap -V )

13:00 <invano> ah, of course when I can't get a trace, if I restart the stap over and over I'll finally get what I need

13:01 <invano> I'm using stap 2.9

13:02 <invano> If I remember correctly I had the same issue even with latest stap 3.1 but I can give it a new try

13:08 <fche> ok, a few more things one can try to gather info for a bug report

13:08 <fche> stap -DDEBUG_PROBES

13:09 <fche> -DDEBUG_TASK_FINDER_VMA

13:09 <fche> -DDEBUG_UPROBES

13:09 <fche> -DDEBUG_TASK_FINDER

13:10 <fche> stap-report's output also helpful

14:08 <sona> hi

14:08 <sona> i want to probe a userspace function taken at run time from user

14:09 <sona> how to do this

14:11 <fche> no easy way - resolution of probe locations requires symbol/debuginfo processing, which is not really available at stap run time

14:11 <fche> you can pass a function name as a command line argument and sub it into a stap script with $@ etc.

14:12 <fche> or you could insert probes everywhere, disabled with an on-the-fly global variable, and then selectively activate it in response to a procfs write

14:12 <fche> that might be a reasonable compromise method

14:17 <sona> fche, oky

14:17 <fche> I can try talking you through that last one, it's an advanced (but fun) technique

14:22 <sona> sona, which one is more efficient among the two

14:22 <sona> fhce,

14:22 <sona> fche,

14:24 <fche> the former case requires one to rerun stap with every new function

14:24 <fche> the latter case requires one to probably create the stap script with another script, and may involve runtime overheads for the inactivated probes

14:24 <fche> what -action- (probe handler) did you have in mind for all these?

14:26 <sona> fche, so probe handler just takes traces of functions who triggerd it

14:28 <sona> fche, trying to go by approach 1 as it will put probes on the required function, thus avoiding resouce overhead as in later case

14:29 <fche> so approach 1 is the trivial % stap -e 'probe process.function(@1) { ... } ' foo_bar

14:35 <sona> fche, foo_bar need to be applied after running the command?

14:36 <sona> i meant if running stap -e 'probe process.function(@1)' {printf("data")}' give parse error

14:37 <fche> yes.

14:37 <fche> so approach 1 does not work at script run time.

14:37 <fche> approach 2 cheats by computing all possible probe points at script processing time, but disabling them on-the-fly at run-time, until commanded

14:41 <sona> fche, oky

17:11 <irker394> systemtap: dsmith systemtap.git:refs/heads/master * release-3.1-61-gbccff21 / configure configure.ac httpd/Makefile.am httpd/Makefile.in httpd/main.cxx httpd/server.cxx httpd/server.h: Make changes for RHEL6 in the http web service server code. http://tinyurl.com/lumn4ed