20:43 <agentzh> fche: i just tried a simple patch to parallelise the "autoconf" phase for generating stapconf.h. the total Pass-3 time for a simple stap script reduces from 7.7s to 2.1s on my 8c/16t machine when the cache is cold.

20:44 <agentzh> also, tried to make stap-symbol.h a separate CU and knocked out 100ms ~ 300ms from Pass-3 for cases with nontrivial unwind/symbol data as well.

20:45 <agentzh> the latter is not very impressive since stap-symbols.h is already compiling fast anyway.

20:45 <fche> yeah not much in there

20:45 <fche> hm surprised the stapconf autoconf stuff wasn't already parallel

20:45 <fche> interested in your findings!

20:46 <agentzh> also played with gcc's .h.gch thing. got ~200ms knocked off for simple stap scripts from Pass-3.

20:46 <agentzh> fche: it uses serial make commands and >> autoconf.h redirects.

20:47 <fche> ah

20:47 <agentzh> i changed it to separate sub-files and cat everything together.

20:47 <fche> yup, or cat | sort for reproducibility

20:48 <agentzh> and also write the stapexport macros directly to file instead of using many echo xxx >> stapconf.h in makefile.

20:48 <agentzh> i already use $^ make variable to make the order the same as in the buildrun.cxx source file.

20:48 <agentzh> so already reproduciable.

20:49 <fche> nice

20:49 <agentzh> so interested in a patch?

20:49 <fche> definitely

20:49 <agentzh> great. i'll submit it soon.

20:49 <agentzh> to the ml

20:49 <agentzh> i mean

20:50 <agentzh> i'm also thinking about making the map.c, addrs.c separaet CUs.

20:50 <agentzh> as well as splitting up the main xxx_src.c files by the amount of code for bunches of stap global/private functions.

20:50 <agentzh> not easy there.

20:50 <fche> we haven't paid much attention to CU issues, so header file ordering & organizing stuff for the runtime has been unnecessary

20:51 <agentzh> yeah, it takes time to clean those macros up.

20:51 <fche> experimentation ok there, I'm not as optimistic

20:51 <agentzh> some macros do matter for memory layout and branched code.

20:52 <agentzh> i gather there would be gain for large .stp scripts (as with our cases).

20:52 <fche> could be

20:52 <agentzh> for small scripts, it's already fast enough (1 ~ 1.3s on my machine).

20:52 <agentzh> for Pass-3

20:52 <fche> pass-4

20:52 <agentzh> Sorry, Pass-4.

20:52 <agentzh> i referred to Pass-4 everywhere above. sorry.

20:52 <fche> heh ;)

20:53 <agentzh> Pass-3 is for C code generation.

20:53 <agentzh> oh btw, i also implemented a separate --gen-stapconf FILE and a --use-stapconf FILE option for external stapconf file caching and management.

20:54 <agentzh> i found stap's own cache not flexible enough.

20:54 <fche> ok, interested in learning more re. motive etc.

20:54 <agentzh> like encoding the kernel build tree dir name into the cache key and blind cache cleanup without LRU policies, and etc

20:55 <agentzh> furthermore we could pre-compile stapconf.h with the kernel header packages, just like we could pre-compile .ko module files for each kernel package.

20:56 <agentzh> for the latter, we cannot always do that due to userland process changes.

20:56 <agentzh> but stapconf.h only depends on the kernel header and the stap version.

20:56 <agentzh> so the cache hit rate is mugh higher.

20:56 <agentzh> then when building .ko, we do not need to pay for the stapconf generation phase at all.

20:57 <agentzh> it's gone.

20:57 <agentzh> that part is like 600ms for a 8c/16t CPU.

20:57 <agentzh> or 700ms

20:57 <agentzh> even fully parallelized.

20:58 <agentzh> that's a lot.

20:58 <fche> if the caching were to work properly, precompiling would not be necessary

20:58 <fche> just stap -p4 -e 'probe oneshot{}'

20:58 <fche> to 'precompile'

20:59 <agentzh> working with stap's own caching is tricky and also we have a lot of different kernels and a lot of build machines, the first time penalty is still a thing.

21:00 <fche> well, different kernels mean different stapconf* regardless

21:00 <agentzh> fche: yeah, i've already been using that for my own dev box, it works. but for distributed env, no, that's painful...

21:01 <agentzh> yeah, but we can iterate all the kernel packages in a linux distro beforehand.

21:01 <agentzh> and index the resuting stapconf.h in a database.

21:02 <agentzh> so for build boxes generating ko files, they just provide the stapconf.h readily.

21:02 <agentzh> by looking up the db.

21:02 <agentzh> simple and reliable.

21:02 <fche> well, requires someone to manage the database etc.

21:02 <fche> but sure, I wouldn't veto that feature, even if it's only rarely useful

21:02 <agentzh> but i agree it's a special use case and 99.9% of the stap users do not bother.

21:02 <agentzh> we can keep this thing to ourselves.

21:03 <agentzh> just sharing.

21:04 <fche> re. stapconf parallel generation, definite interest & usefulness there.

21:04 <agentzh> sure, will do.

21:04 <agentzh> they are separate patches.

21:05 <agentzh> thanks! and have a good day!