<ztfw>
I need to do something like let [foo; bar] = List.map f ["foo"; "bar"]
<ztfw>
I'm not sure how to do that with tuples
<bobzhang>
you can write a Tuple2.map
<bobzhang>
- : ('a -> 'b) ->
<bobzhang>
'a Batteries.Tuple2.mappable -> 'b Batteries.Tuple2.mappable
<bobzhang>
;;
pilki has joined #ocaml
<bobzhang>
batteries have Tuple2,3,4,5.map T_T
jamii has quit [Quit: Leaving]
letrec has quit [Ping timeout: 268 seconds]
<ztfw>
thanks
smerz has quit [Remote host closed the connection]
pilki has quit [Quit: This computer has gone to sleep]
mcclurmc has quit [Excess Flood]
mcclurmc has joined #ocaml
Submarine has quit [Ping timeout: 240 seconds]
sepp2k has quit [Quit: Leaving.]
oriba has joined #ocaml
oriba has quit [Read error: Connection reset by peer]
pilki has joined #ocaml
pilki has quit [Quit: This computer has gone to sleep]
letrec has joined #ocaml
ankit9 has joined #ocaml
companion_cube has quit [Remote host closed the connection]
mcclurmc has quit [Excess Flood]
mcclurmc has joined #ocaml
bobzhang has quit [Ping timeout: 252 seconds]
tautologico has quit [Quit: tautologico]
NihilistDandy has quit [Remote host closed the connection]
NihilistDandy has joined #ocaml
emmanuelux has joined #ocaml
Submarine has joined #ocaml
pango__ has quit [Remote host closed the connection]
edwin has joined #ocaml
letrec has quit [Ping timeout: 248 seconds]
ftrvxmtrx has joined #ocaml
waterfowl has left #ocaml []
Kakadu has joined #ocaml
bitbckt has quit [Quit: out]
bitbckt has joined #ocaml
ulfdoz_ has joined #ocaml
milosn has quit [Read error: Connection reset by peer]
milosn has joined #ocaml
ulfdoz has quit [Ping timeout: 252 seconds]
ulfdoz_ is now known as ulfdoz
_andre has joined #ocaml
rwmjones has joined #ocaml
Schadenfreude has joined #ocaml
iago has joined #ocaml
ikaros has joined #ocaml
spearalot has joined #ocaml
raichoo has joined #ocaml
iago has quit [Ping timeout: 268 seconds]
iago has joined #ocaml
raichoo has quit [Quit: leaving]
pilki has joined #ocaml
iago has quit [*.net *.split]
iago has joined #ocaml
oriba has joined #ocaml
spearalot has quit [Quit: Computer has gone to sleep]
Kakadu has quit [Quit: Page closed]
pilki has quit [Quit: This computer has gone to sleep]
velov has joined #ocaml
spearalot has joined #ocaml
velov has quit [Read error: Connection reset by peer]
velov has joined #ocaml
raichoo has joined #ocaml
pilki has joined #ocaml
pilki has quit [Quit: This computer has gone to sleep]
pilki has joined #ocaml
pilki has quit [Client Quit]
bzzbzz has joined #ocaml
letrec has joined #ocaml
oriba has quit [Quit: oriba]
elehack has joined #ocaml
elehack has left #ocaml []
bobzhang has joined #ocaml
everyonemines has joined #ocaml
emmanuelux has quit [Remote host closed the connection]
emmanuelux has joined #ocaml
pilki has joined #ocaml
pilki has quit [Client Quit]
rossberg has quit [Ping timeout: 252 seconds]
<ousado>
hi all. any general hints on how to look for memory leaks in ocaml programs?
pango has joined #ocaml
<everyonemines>
Memory leaks?
<everyonemines>
Are you sure you have a memory leak from ocaml?
<thelema>
everyonemines: it's still possible for GC'ed programs to leak
<ousado>
more specifically I'm looking for a memory leak in a compiler that is meant to be run once and then exit, but I am embedding it to run it on demand inside a long running process, and memory usage increases (linearly)
Boscop has joined #ocaml
rossberg has joined #ocaml
<everyonemines>
thelema: Only if you have a bug in the compiler...?
<ousado>
everyonemines: well, at the moment there's no other allocation, except once for the callback into ocaml code
<thelema>
everyonemines: no, if you keep pointers to data you don't mean to keep
<everyonemines>
Oh, well yeah. I don't think of that as being a leak though.
<thelema>
ousado: IIRC, there's some little functions to count the recursive memory usage of a value (and everything pointed to by it)
<everyonemines>
So, how to look for pointers you didn't mean to keep? Do a search for "ref " :-)
<ousado>
I'm also running a Gc.compact after every invocation
<ousado>
yes, refs are used in some places
<thelema>
everyonemines: nope, it's possible to leak by simply prepending to a list
<everyonemines>
eh?
<everyonemines>
You mean the list doesn't get discarded if you prepend to it later?
<thelema>
everyonemines: if his compiler prepended to a list each round, (no ref needed, just passing the new head of this list around), it could have unbounded memory usage
<ousado>
it's well possible
<everyonemines>
Yes, I understood that. A list is basically a container with refs in it.
<everyonemines>
It's sort of a wrapper around tuples with refs.
<everyonemines>
I'm just not sure why you would keep adding to a list you're not using old values of.
<thelema>
everyonemines: umm, ocaml has a special meaning for "ref" - that being a mutable record with a single field.
<ousado>
hm.. I'll try to find them
<thelema>
everyonemines: caches
<thelema>
everyonemines: lists don't have refs, as they're immutable. They have a field containing a tail list. This is implemented with a pointer, which is sometimes called a reference, but this is not an ocaml ref
<everyonemines>
er yes
<everyonemines>
that's true
<ousado>
I should also ask the author where I am most likely to find those
<everyonemines>
Some languages do have mutable lists though. Was I thinking of SML?
<thelema>
ousado: another option is to see if you can reduce the scope of your values so that everything used in compilation goes out of scope when the compiler finishes.
<thelema>
ousado: oh, you didn't write the compiler?
<ousado>
nope
<ousado>
it's the haxe compiler
<thelema>
ousado: eww... That may be quite difficult - compilers are often written with ugly mutable state. Maybe you can use 1st class modules to de-scope the whole compiler after each run.
raichoo has quit [Ping timeout: 268 seconds]
<ousado>
and it's no big deal, actually, it takes about 100,000 cycles to hit 1G mem usage
<thelema>
well, I guess it wouldn't be the whole compiler, just the toplevel. If internal modules are keeping state...
<ousado>
that's for my test only, though
<everyonemines>
How many modules does the haxe compiler use?
<thelema>
ousado: in that case, just watch memory usage and exit (with a script to restart on exit)
raichoo has joined #ocaml
<thelema>
ousado: you've found the Gc module, which can give you memory usage from within ocaml?
<ousado>
yes
<everyonemines>
The crude solution would be to use Sys.command to call the compiler as a separate process.
<ousado>
sure
<ousado>
but I want it in-process for everal reasons
<ousado>
*several
<ousado>
It even has a compilation server
<everyonemines>
If code is in a single module, you can de-scope by replacing ;; with in, and putting everything in ( )
<everyonemines>
But that's probably not the case.
<thelema>
ousado: in your wrapper: if (Gc.quick_stat ()).Gc.heap_words > x then exit 20
<thelema>
and have a wrapper for your progran that restarts it if it exits with return code 20
<ousado>
heh, I want to run it inside an IDE :)
<thelema>
ousado: oh, your code is the ide... hmmm
<ousado>
it should ask the user before doing that :)
<ousado>
I'm considering a shared memory interface, though
<everyonemines>
You're not really supposed to use modules to store state, that's bad form.
<ousado>
then restarting wouldn't be a problem
<thelema>
well, 100K cycles doesn't sound too bad. I'd ask the haxe authors about cleaning up any memory leaks or providing a way to clear any caches it uses
<thelema>
everyonemines: compilers break a lot of rules for ease of implementation
<everyonemines>
thelema: If record namespaces weren't an issue maybe people wouldn't want to do that.
<ousado>
yes, for all practical purposes it's OK for the moment
<everyonemines>
If there's state being stored in modules, I bet the issue is something like a list ref
<everyonemines>
that keeps growing
<everyonemines>
maybe search for "ref []"
<thelema>
everyonemines: I'm thinking a hashtable is more likely
<everyonemines>
oh, good point
Cyanure has joined #ocaml
<everyonemines>
But I don't think a plain list will cause this problem.
<thelema>
everyonemines: depending on the code, it could. I agree it's not likely, but it's certainly possible.
<ousado>
hm.. there are some hash tables for collecting timing info, which are not being reset properly
<everyonemines>
If you have threads then sure, anything's possible, but...
<thelema>
everyonemines: if new data is passed around to keep it a alive (i.e. your whole program is a recursive loop with newly constructed values being passed in as parameters, ...
<everyonemines>
Yes, you need a recursive function.
<everyonemines>
But we're talking about stored state in modules.
<everyonemines>
That's only possible if you stay within a function with the list as a parameter.
<everyonemines>
He's outside the entire compiler system, not adding stuff within a recursive function in it.
<thelema>
granted - the only way to keep state from one function call to the next aside from parameters (which I assume are completely fresh each time) is mutable values.
<everyonemines>
but it sounds like the issue was a hashtable
<everyonemines>
good call
<thelema>
which often means refs, although it can be arrays, hashtables, queues, stacks, and maybe something else in the stdlib.
<everyonemines>
If the compiler used a functor of an empty module
<everyonemines>
and you created a new module for each run
raichoo has quit [Ping timeout: 252 seconds]
<everyonemines>
would that cause a memory leak or would the GC collect the old modules?
<everyonemines>
Sort of a "module as object system" setup.
companion_cube has joined #ocaml
<hcarty>
everyonemines: This is mostly guessing, but I expect that a module would only be GC'd if it is created as a value or with let module Foo = ... in
<everyonemines>
You can do let module foo = ... in ?
<everyonemines>
Huh, interesting.
<hcarty>
Yep, that how the old local open syntax extension worked
<everyonemines>
Why did they change it?
raichoo has joined #ocaml
<hcarty>
From 3.12.0 on there is language-level support without the performance penalty the syntax extension brought
<everyonemines>
What does that have to do with open Module -> Module.(stuff)
<hcarty>
let f x = open Foo in e --> let f x = let module M = struct include Foo let local_x () = e end M.local_x ()
<hcarty>
Or something resembling that
<hcarty>
That transformation is no longer required with 3.12.0+
<everyonemines>
oh, I was just asking why they changed the syntax
<everyonemines>
to Module.(stuff)
<everyonemines>
if it used to be open Module in
<hcarty>
Both are supported, with a slight change -- open Module in is now let open Module in
<everyonemines>
let x = let open Array in 1
<everyonemines>
that looks so weird
<everyonemines>
why
<hcarty>
Avoiding ambiguity? I'm not sure.
<everyonemines>
what ambiguity?
<thelema>
everyonemines: batteries uses it for tests
<everyonemines>
open is a reserved keyword
<thelema>
everyonemines: so is "let"
<hcarty>
Requiring "let open" for local opens may make handling global and local opens easier. I haven't hacked on the compiler though, so I don't know for certain.
ftrvxmtrx has quit [Quit: Leaving]
<hcarty>
everyonemines: You wouldn't generally use the let open M in .. syntax for something that short
<hcarty>
That's why it's nice to have the M.(...) equivalent
<hcarty>
Well, part of why it's nice to have it.
<flux>
it's nice for tuareg mode, it doesn't really deal well with the 'open X in' :)
<everyonemines>
It's not a big deal, I just think it's weird looking because it clashes with the usual meaning of "let"
<flux>
I think it's not that weird. it says that within this scope, there is a bunch of extra symbols
<hcarty>
flux: Very true! open X in broke vim and emacs OCaml indentation IIRC.
<everyonemines>
To me, the "in" says that.
<everyonemines>
Ah, maybe that's why they changed it. But better to change tuareg.
spearalot has quit [Quit: Computer has gone to sleep]
<flux>
if I had to guess, I would imagine 'let open' was simpler to add to the parser..
<flux>
which, if true, in itself has some value, lessening the maintenance burden
<hcarty>
flux: That's my expectation as well
bobzhang has quit [Ping timeout: 256 seconds]
everyonemines has quit [Quit: Leaving.]
<ousado>
oh, thanks everyone, btw
<thelema>
ousado: figure out the leak?
zorun has quit [Read error: Operation timed out]
<ousado>
thelema: not yet
<thelema>
clearing the hashtables didn't help?
<ousado>
no
<ousado>
but there are lots of those ref [] things you mentioned
<thelema>
you said 1GB in 100K compiles = 1MB in 100 compiles = 10KB leak per compile?
<ousado>
about that, yes
<thelema>
32-bit ocaml?
<ousado>
yes
<ousado>
I'll contact the author before I try to find it myself
<thelema>
ok.
zorun has joined #ocaml
<ousado>
I just wanted to evaluate general options first. I don't use ocaml frequently (which is a bit unfortunate)
* thelema
frowns at haxe's compilation instructions
<ousado>
hehe
<ousado>
the docs are a bit lacking
<ousado>
thelema: oh, are you compiling from source?
<thelema>
I'm looking into what it takes so I can automate it with odb
<ousado>
I see
<ousado>
you found the install.ml?
<thelema>
not yet - I'm adding svn support to odb to fetch it