fche changed the topic of #systemtap to: http://sourceware.org/systemtap; email systemtap@sourceware.org if answers here not timely, conversations may be logged
nkambo has quit [Read error: Connection reset by peer]
ph7 has joined #systemtap
ego has joined #systemtap
nkambo has joined #systemtap
hpt has quit [Ping timeout: 244 seconds]
ph7 has quit [Quit: Leaving.]
nkambo has quit [Ping timeout: 260 seconds]
ph7 has joined #systemtap
scox has quit [Ping timeout: 250 seconds]
nkambo has joined #systemtap
nkambo has quit [Ping timeout: 260 seconds]
ghone has joined #systemtap
<ghone>
Hi all,
<ghone>
a few years ago I abandoned dtrace camp (solaris) and joined systemtap camp (linux).
<ghone>
So far I haven't used systemtap so I am total beginner
<ghone>
on one of our red hat servers (kernel 2.6.32) there are some network problems
<ghone>
server A is pushing tcp data to server B
<ghone>
when you watch server A (tshark ?) you can see constant, big stream of network packets going to server B
<ghone>
whereas on server B (again tshark) you can see some bursts of network traffic
<ghone>
so from time to time it slows down a lot and then after a period of time (10-20 secs) it is at full speed
<ghone>
what would be the first systemtap-based approach to investigate that ?
ananth has quit [Quit: Leaving]
naveen has quit [Quit: Leaving.]
mjw has joined #systemtap
ph7 has quit [Quit: Leaving.]
ph7 has joined #systemtap
ph7 has quit [Client Quit]
ph7 has joined #systemtap
nkambo has joined #systemtap
nkambo1 has joined #systemtap
nkambo has quit [Ping timeout: 264 seconds]
nkambo1 has quit [Ping timeout: 246 seconds]
srikar is now known as srikar_away
scox has joined #systemtap
mbenitez has joined #systemtap
mbenitez has quit [Changing host]
mbenitez has joined #systemtap
przemoc86 has quit [Quit: reload]
przemoc has joined #systemtap
przemoc has quit [Client Quit]
srikar_away is now known as srikar
ego has quit [Ping timeout: 252 seconds]
ego has joined #systemtap
drsmith has joined #systemtap
ph7 has quit [Quit: Leaving.]
naveen has joined #systemtap
ph7 has joined #systemtap
ph7 has quit [Quit: Leaving.]
ph7 has joined #systemtap
ph7 has quit [Client Quit]
ego has quit [Ping timeout: 246 seconds]
ph7 has joined #systemtap
ph7 has quit [Client Quit]
ph7 has joined #systemtap
brolley has joined #systemtap
<fche>
hi ghone
<ghone>
fche: Hi fche
<fche>
(you don't have to be in one camp vs. another to get some use out of the tools! use whatever works for you!)
<fche>
for general network traffic monitoring, I'd suggest looking into netfilter.* probes
<fche>
you can catch an event for every packet sent out from one side, and on the other, catch an event for every packet that comes in
<fche>
you can filter on peer ip addresses
<ghone>
thanks for that
<fche>
compute rates or timestamps
<ghone>
but for example when I have a lokk at netstat -s you can see a lot of events
<fche>
lot of counts accumulating there, yeah
<ghone>
how to I find a mapping between all these statistics and kernel events ?
<fche>
that's not as easy as it should be
<fche>
the kernel netstat-s counters are incremented in a variety of places, and they don't usually have a marker by which stap can hang a probe point on it
<fche>
do you have a particular counter in mind?
<fche>
then I'd look at the kernel sources to find the variable that corresponds to it; thence the sites where the variable is incremented
<fche>
then - if I really needed to catch those counts increasing as they're happening, live - put a .function or .statement probe into the vicinity
<ghone>
take any of netstat -s counters, like any tcp related
<ghone>
do you have any stap script
<ghone>
which would show how to use these ststaistics ?
<ghone>
All systemtap script I had found so far are kind of net_top showing who is using network the most
<ghone>
whereas in this case I need to trace network events related to counters
<fche>
there is AFAIK no generic kernel mechanism to intercept those netstat-s counters being modified
<fche>
(it'd sure be nice to have them put in a tracepoint)
<fche>
e.g. __*_INC_STATS() is a family of macros that bump the counters
<fche>
for failures, one can generally trap the skbuf being freed prematurely, as per the dropwatch.stp sample script
ravi has quit [Remote host closed the connection]
<ghone>
thanks fche - it looks more complicated then at the begin
<fche>
yeah
<fche>
but anyway
<fche>
if you want to start with just getting a trace of traffic being sent & received, that's easy
<fche>
to turn that trace into a timeline of rates, that's fairly easy too
<ghone>
well I think ideally I would like to corelate network traffic with kernel network buffers
<ghone>
or to monitor when each (particular ?) network buffer is used
<ghone>
(histogram ?)
<fche>
yeah except buffers are for individual packets, they're very short-lived
lzap has quit [Ping timeout: 276 seconds]
lzap has joined #systemtap
mjw has quit [Quit: Leaving]
przemoc has joined #systemtap
ph7 has quit [Quit: Leaving.]
ph7 has joined #systemtap
mbenitez has quit [Read error: Connection reset by peer]
mbenitez has joined #systemtap
mbenitez has quit [Changing host]
mbenitez has joined #systemtap
srikar is now known as srikar_away
Empyrium has quit [K-Lined]
ravi has joined #systemtap
ph7 has quit [Quit: Leaving.]
gila has quit [Quit: My Mac Pro has gone to sleep. ZZZzzz…]
srikar_away is now known as srikar
naveen has quit [Ping timeout: 260 seconds]
irker282 has joined #systemtap
<irker282>
systemtap: dsmith systemtap.git:refs/heads/master * release-3.0-89-g87f8747 / testsuite/lib/systemtap.exp: Made a small improvement to testsuite/lib/systemtap.exp. http://tinyurl.com/hjbhnsg
naveen has joined #systemtap
mbenitez has quit [Read error: Connection reset by peer]
mbenitez has joined #systemtap
mbenitez has quit [Changing host]
mbenitez has joined #systemtap
srikar is now known as srikar_away
naveen has quit [Quit: Leaving.]
ravi has quit [Ping timeout: 260 seconds]
<irker282>
systemtap: csanting systemtap.git:refs/heads/csanting/PR19624 * release-3.0-90-ged47a72 / parse.cxx: PR19624: Duplicate function paramater names are not detected http://tinyurl.com/gul9w9e
gila has joined #systemtap
ph7 has joined #systemtap
<irker282>
systemtap: csanting systemtap.git:refs/heads/csanting/PR19624 * release-3.0-91-gb426577 / parse.cxx: PR19624: Duplicate function parameter names are not detected http://tinyurl.com/j9fgnls
pmuldoon has quit [Read error: Connection reset by peer]
pmuldoon has joined #systemtap
<irker282>
systemtap: csanting systemtap.git:refs/heads/csanting/PR19624 * release-3.0-90-g5909320 / parse.cxx: PR19624: Duplicate function paramater names are not detected http://tinyurl.com/j4suxuk