fche changed the topic of #systemtap to: http://sourceware.org/systemtap; email systemtap@sourceware.org if answers here not timely, conversations may be logged
derek0883 has quit [Ping timeout: 260 seconds]
derek0883 has joined #systemtap
derek0883 has quit [Remote host closed the connection]
derek0883 has joined #systemtap
derek0883 has quit [Remote host closed the connection]
orivej has joined #systemtap
derek0883 has joined #systemtap
_whitelogger has joined #systemtap
orivej has quit [Ping timeout: 240 seconds]
sscox has quit [Ping timeout: 264 seconds]
zodbot has quit [Remote host closed the connection]
derek0883 has quit [Remote host closed the connection]
derek0883 has joined #systemtap
zodbot has joined #systemtap
derek0883 has quit [Remote host closed the connection]
derek0883 has joined #systemtap
zodbot has quit [Ping timeout: 272 seconds]
zodbot has joined #systemtap
derek0883 has quit [Remote host closed the connection]
_whitelogger has joined #systemtap
derek0883 has joined #systemtap
derek0883 has quit [Remote host closed the connection]
<agentzh>
i'm also worried about "'__global_a' lock contention occurred 906 times", which is another issue. maybe we can use RCU locks to make it lock-free as well?
<fche>
no
<fche>
but such a probe should probably use stats type variables
<fche>
timer.profile { a <<< 1 }
<fche>
probe end { println(@count(a)) }
<agentzh>
so, stats is faster than a scalar global huh?
<fche>
and yeah the report certainly means something: the hits: / cycles: counts most
<agentzh>
but it does not hit the probe timer.profile in another parallel stap ko.
<agentzh>
not a single time.
<agentzh>
only its own probe handlers.
<agentzh>
fche: is it possible to apply similar optimizations to the global numerical variables as well?
<agentzh>
it seems wasteful to use stats everywhere.
orivej has quit [Ping timeout: 265 seconds]
<agentzh>
the lock contention is troublesome for such globals.
<fche>
no, stats are more efficient when arithmetic ops are limited, and concurrency across cpus exists
<fche>
this is specifically what they were designed for.
<agentzh>
okay
<agentzh>
it's just a bit weird to use stats for a boolean global var.
<fche>
looks like a counter, not boolean
<agentzh>
i mean other use cases.
<agentzh>
we also see global boolean variables lead to a lot of lock contention according to stap -t
<agentzh>
*leading to
<agentzh>
the counter is my example above is logically natural to use stats.
<agentzh>
*in my
<fche>
in highly concurrent probes, one should use stats as much as possible to avoid contention
<fche>
sometimes that's not possible
<agentzh>
oh i see another issue here. stats globals cannot be set initial values via staprun's cmdline?
<agentzh>
like staprun foo.ko FOO=32
<agentzh>
something like this.
<fche>
I suppose you could seed it with some values in a begin probe, but <<< is not assignment, it's statistics-accumulation
<agentzh>
yeah seems like we cannot use stats for everything.
<fche>
yup.
<agentzh>
maybe for read-most global vars, we can use more efficient lock variant?
<agentzh>
sometimes it's just a flag set in probe begin only.
<agentzh>
but it's read everywhere frequently.
<fche>
read-mostly variables should suffer much less contention
<fche>
(rwlocks of some type used here.)
<agentzh>
okay, i
<agentzh>
will check it out.
<agentzh>
fche: i tried harder, stap ko A indeed cannot see ko B's backtraces at all when both of them are running hot.
<agentzh>
i collected all the kernel backtraces in a stats of ko A and print all the unique kernel backtraces out in the end.
<agentzh>
grep shows nothing about ko B.
<agentzh>
ko B is named "test"
<agentzh>
and i used -d test when generating ko A.
<agentzh>
as well as '-d kernel'
<agentzh>
'perl top' shows ko B's "test" module name and frame properly: