gildor changed the topic of #ocaml to: Discussions about the OCaml programming language | http://caml.inria.fr/ | OCaml 3.12.1 http://bit.ly/nNVIVH
<ztfw> I need to do something like let [foo; bar] = List.map f ["foo"; "bar"]
<ztfw> I'm not sure how to do that with tuples
<bobzhang> you can write a Tuple2.map
<bobzhang> - : ('a -> 'b) ->
<bobzhang> 'a Batteries.Tuple2.mappable -> 'b Batteries.Tuple2.mappable
<bobzhang> ;;
pilki has joined #ocaml
<bobzhang> batteries have Tuple2,3,4,5.map T_T
jamii has quit [Quit: Leaving]
letrec has quit [Ping timeout: 268 seconds]
<ztfw> thanks
smerz has quit [Remote host closed the connection]
pilki has quit [Quit: This computer has gone to sleep]
mcclurmc has quit [Excess Flood]
mcclurmc has joined #ocaml
Submarine has quit [Ping timeout: 240 seconds]
sepp2k has quit [Quit: Leaving.]
oriba has joined #ocaml
oriba has quit [Read error: Connection reset by peer]
pilki has joined #ocaml
pilki has quit [Quit: This computer has gone to sleep]
letrec has joined #ocaml
ankit9 has joined #ocaml
companion_cube has quit [Remote host closed the connection]
mcclurmc has quit [Excess Flood]
mcclurmc has joined #ocaml
bobzhang has quit [Ping timeout: 252 seconds]
tautologico has quit [Quit: tautologico]
NihilistDandy has quit [Remote host closed the connection]
NihilistDandy has joined #ocaml
emmanuelux has joined #ocaml
Submarine has joined #ocaml
pango__ has quit [Remote host closed the connection]
edwin has joined #ocaml
letrec has quit [Ping timeout: 248 seconds]
ftrvxmtrx has joined #ocaml
waterfowl has left #ocaml []
Kakadu has joined #ocaml
bitbckt has quit [Quit: out]
bitbckt has joined #ocaml
ulfdoz_ has joined #ocaml
milosn has quit [Read error: Connection reset by peer]
milosn has joined #ocaml
ulfdoz has quit [Ping timeout: 252 seconds]
ulfdoz_ is now known as ulfdoz
_andre has joined #ocaml
rwmjones has joined #ocaml
Schadenfreude has joined #ocaml
iago has joined #ocaml
ikaros has joined #ocaml
spearalot has joined #ocaml
raichoo has joined #ocaml
iago has quit [Ping timeout: 268 seconds]
iago has joined #ocaml
raichoo has quit [Quit: leaving]
pilki has joined #ocaml
iago has quit [*.net *.split]
iago has joined #ocaml
oriba has joined #ocaml
spearalot has quit [Quit: Computer has gone to sleep]
Kakadu has quit [Quit: Page closed]
pilki has quit [Quit: This computer has gone to sleep]
velov has joined #ocaml
spearalot has joined #ocaml
velov has quit [Read error: Connection reset by peer]
velov has joined #ocaml
raichoo has joined #ocaml
pilki has joined #ocaml
pilki has quit [Quit: This computer has gone to sleep]
pilki has joined #ocaml
pilki has quit [Client Quit]
bzzbzz has joined #ocaml
letrec has joined #ocaml
oriba has quit [Quit: oriba]
elehack has joined #ocaml
elehack has left #ocaml []
bobzhang has joined #ocaml
everyonemines has joined #ocaml
emmanuelux has quit [Remote host closed the connection]
emmanuelux has joined #ocaml
pilki has joined #ocaml
pilki has quit [Client Quit]
rossberg has quit [Ping timeout: 252 seconds]
<ousado> hi all. any general hints on how to look for memory leaks in ocaml programs?
pango has joined #ocaml
<everyonemines> Memory leaks?
<everyonemines> Are you sure you have a memory leak from ocaml?
<thelema> everyonemines: it's still possible for GC'ed programs to leak
<ousado> more specifically I'm looking for a memory leak in a compiler that is meant to be run once and then exit, but I am embedding it to run it on demand inside a long running process, and memory usage increases (linearly)
Boscop has joined #ocaml
rossberg has joined #ocaml
<everyonemines> thelema: Only if you have a bug in the compiler...?
<ousado> everyonemines: well, at the moment there's no other allocation, except once for the callback into ocaml code
<thelema> everyonemines: no, if you keep pointers to data you don't mean to keep
<everyonemines> Oh, well yeah. I don't think of that as being a leak though.
<thelema> ousado: IIRC, there's some little functions to count the recursive memory usage of a value (and everything pointed to by it)
<everyonemines> So, how to look for pointers you didn't mean to keep? Do a search for "ref " :-)
<ousado> I'm also running a Gc.compact after every invocation
<ousado> yes, refs are used in some places
<thelema> everyonemines: nope, it's possible to leak by simply prepending to a list
<everyonemines> eh?
<everyonemines> You mean the list doesn't get discarded if you prepend to it later?
<thelema> everyonemines: if his compiler prepended to a list each round, (no ref needed, just passing the new head of this list around), it could have unbounded memory usage
<ousado> it's well possible
<everyonemines> Yes, I understood that. A list is basically a container with refs in it.
<everyonemines> It's sort of a wrapper around tuples with refs.
<thelema> ousado: use this on various values you keep persistently to identify which one is growing: http://forge.ocamlcore.org/frs/download.php/442/objsize-0.16.tar.gz
<everyonemines> I'm just not sure why you would keep adding to a list you're not using old values of.
<thelema> everyonemines: umm, ocaml has a special meaning for "ref" - that being a mutable record with a single field.
<ousado> hm.. I'll try to find them
<thelema> everyonemines: caches
<thelema> everyonemines: lists don't have refs, as they're immutable. They have a field containing a tail list. This is implemented with a pointer, which is sometimes called a reference, but this is not an ocaml ref
<everyonemines> er yes
<everyonemines> that's true
<ousado> I should also ask the author where I am most likely to find those
<everyonemines> Some languages do have mutable lists though. Was I thinking of SML?
<thelema> ousado: another option is to see if you can reduce the scope of your values so that everything used in compilation goes out of scope when the compiler finishes.
<thelema> ousado: oh, you didn't write the compiler?
<ousado> nope
<ousado> it's the haxe compiler
<thelema> ousado: eww... That may be quite difficult - compilers are often written with ugly mutable state. Maybe you can use 1st class modules to de-scope the whole compiler after each run.
raichoo has quit [Ping timeout: 268 seconds]
<ousado> and it's no big deal, actually, it takes about 100,000 cycles to hit 1G mem usage
<thelema> well, I guess it wouldn't be the whole compiler, just the toplevel. If internal modules are keeping state...
<ousado> that's for my test only, though
<everyonemines> How many modules does the haxe compiler use?
<thelema> ousado: in that case, just watch memory usage and exit (with a script to restart on exit)
raichoo has joined #ocaml
<thelema> ousado: you've found the Gc module, which can give you memory usage from within ocaml?
<ousado> yes
<everyonemines> The crude solution would be to use Sys.command to call the compiler as a separate process.
<ousado> sure
<ousado> but I want it in-process for everal reasons
<ousado> *several
<ousado> It even has a compilation server
<everyonemines> If code is in a single module, you can de-scope by replacing ;; with in, and putting everything in ( )
<everyonemines> But that's probably not the case.
<thelema> ousado: in your wrapper: if (Gc.quick_stat ()).Gc.heap_words > x then exit 20
<thelema> and have a wrapper for your progran that restarts it if it exits with return code 20
<ousado> heh, I want to run it inside an IDE :)
<thelema> ousado: oh, your code is the ide... hmmm
<ousado> it should ask the user before doing that :)
<ousado> I'm considering a shared memory interface, though
<everyonemines> You're not really supposed to use modules to store state, that's bad form.
<ousado> then restarting wouldn't be a problem
<thelema> well, 100K cycles doesn't sound too bad. I'd ask the haxe authors about cleaning up any memory leaks or providing a way to clear any caches it uses
<thelema> everyonemines: compilers break a lot of rules for ease of implementation
<everyonemines> thelema: If record namespaces weren't an issue maybe people wouldn't want to do that.
<ousado> yes, for all practical purposes it's OK for the moment
<everyonemines> If there's state being stored in modules, I bet the issue is something like a list ref
<everyonemines> that keeps growing
<everyonemines> maybe search for "ref []"
<thelema> everyonemines: I'm thinking a hashtable is more likely
<everyonemines> oh, good point
Cyanure has joined #ocaml
<everyonemines> But I don't think a plain list will cause this problem.
<thelema> everyonemines: depending on the code, it could. I agree it's not likely, but it's certainly possible.
<ousado> hm.. there are some hash tables for collecting timing info, which are not being reset properly
<everyonemines> If you have threads then sure, anything's possible, but...
<thelema> everyonemines: if new data is passed around to keep it a alive (i.e. your whole program is a recursive loop with newly constructed values being passed in as parameters, ...
<everyonemines> Yes, you need a recursive function.
<everyonemines> But we're talking about stored state in modules.
<everyonemines> That's only possible if you stay within a function with the list as a parameter.
<everyonemines> He's outside the entire compiler system, not adding stuff within a recursive function in it.
<thelema> granted - the only way to keep state from one function call to the next aside from parameters (which I assume are completely fresh each time) is mutable values.
<everyonemines> but it sounds like the issue was a hashtable
<everyonemines> good call
<thelema> which often means refs, although it can be arrays, hashtables, queues, stacks, and maybe something else in the stdlib.
<everyonemines> If the compiler used a functor of an empty module
<everyonemines> and you created a new module for each run
raichoo has quit [Ping timeout: 252 seconds]
<everyonemines> would that cause a memory leak or would the GC collect the old modules?
<everyonemines> Sort of a "module as object system" setup.
companion_cube has joined #ocaml
<hcarty> everyonemines: This is mostly guessing, but I expect that a module would only be GC'd if it is created as a value or with let module Foo = ... in
<everyonemines> You can do let module foo = ... in ?
<everyonemines> Huh, interesting.
<hcarty> Yep, that how the old local open syntax extension worked
<everyonemines> Why did they change it?
raichoo has joined #ocaml
<hcarty> From 3.12.0 on there is language-level support without the performance penalty the syntax extension brought
<everyonemines> What does that have to do with open Module -> Module.(stuff)
<hcarty> let f x = open Foo in e --> let f x = let module M = struct include Foo let local_x () = e end M.local_x ()
<hcarty> Or something resembling that
<hcarty> That transformation is no longer required with 3.12.0+
<everyonemines> oh, I was just asking why they changed the syntax
<everyonemines> to Module.(stuff)
<everyonemines> if it used to be open Module in
<hcarty> Both are supported, with a slight change -- open Module in is now let open Module in
<everyonemines> let x = let open Array in 1
<everyonemines> that looks so weird
<everyonemines> why
<hcarty> Avoiding ambiguity? I'm not sure.
<everyonemines> what ambiguity?
<thelema> everyonemines: batteries uses it for tests
<everyonemines> open is a reserved keyword
<thelema> everyonemines: so is "let"
<hcarty> Requiring "let open" for local opens may make handling global and local opens easier. I haven't hacked on the compiler though, so I don't know for certain.
ftrvxmtrx has quit [Quit: Leaving]
<hcarty> everyonemines: You wouldn't generally use the let open M in .. syntax for something that short
<hcarty> That's why it's nice to have the M.(...) equivalent
<hcarty> Well, part of why it's nice to have it.
<flux> it's nice for tuareg mode, it doesn't really deal well with the 'open X in' :)
<everyonemines> It's not a big deal, I just think it's weird looking because it clashes with the usual meaning of "let"
<flux> I think it's not that weird. it says that within this scope, there is a bunch of extra symbols
<hcarty> flux: Very true! open X in broke vim and emacs OCaml indentation IIRC.
<everyonemines> To me, the "in" says that.
<everyonemines> Ah, maybe that's why they changed it. But better to change tuareg.
spearalot has quit [Quit: Computer has gone to sleep]
<flux> if I had to guess, I would imagine 'let open' was simpler to add to the parser..
<flux> which, if true, in itself has some value, lessening the maintenance burden
<hcarty> flux: That's my expectation as well
bobzhang has quit [Ping timeout: 256 seconds]
everyonemines has quit [Quit: Leaving.]
<ousado> oh, thanks everyone, btw
<thelema> ousado: figure out the leak?
zorun has quit [Read error: Operation timed out]
<ousado> thelema: not yet
<thelema> clearing the hashtables didn't help?
<ousado> no
<ousado> but there are lots of those ref [] things you mentioned
<thelema> you said 1GB in 100K compiles = 1MB in 100 compiles = 10KB leak per compile?
<ousado> about that, yes
<thelema> 32-bit ocaml?
<ousado> yes
<ousado> I'll contact the author before I try to find it myself
<thelema> ok.
zorun has joined #ocaml
<ousado> I just wanted to evaluate general options first. I don't use ocaml frequently (which is a bit unfortunate)
* thelema frowns at haxe's compilation instructions
<ousado> hehe
<ousado> the docs are a bit lacking
<ousado> thelema: oh, are you compiling from source?
<thelema> I'm looking into what it takes so I can automate it with odb
<ousado> I see
<ousado> you found the install.ml?
<thelema> not yet - I'm adding svn support to odb to fetch it
<thelema> oh, yes. I did see that.
<thelema> looks terrible.
<thelema> also the use of nonstandard environment variables for compilation
<ousado> hm. would be interesting to hear an opinion about the compiler itself
<thelema> oh, not yet.
<thelema> I take that back - install.ml is not too bad.
<thelema> I was thinking you meant
<thelema> the installation script at http://haxe.org/doc/build/haxe_ubuntu_build
<ousado> ah no
<ousado> not that one
zorun has quit [Ping timeout: 240 seconds]
<thelema> it's icky that it still uses svn to download libs
<ousado> yes, svn and cvs
<ousado> there are binary packages, too, of course
<thelema> ousado: My current quest is to automate installation of ocaml programs from source
iago has quit [Quit: Leaving]
letrec has quit [Remote host closed the connection]
<ousado> thelema: does odb support switching between several installed versions?
<hcarty> ousado: If you set a few environment variables properly it will
<thelema> not directly, but if you set your path and other environment variables correctly, it will
<ousado> nice
<thelema> and it now supports svn and cvs sources
velov has quit [Quit: leaving]
<hcarty> thelema: Woo!
<thelema> really? I'm surprised at that.
<flux> "no vc left behind"
<hcarty> thelema: Surprised at...?
<thelema> if anyone wants to give me the command to download a mercurial repo, I'll add that too
<thelema> hcarty: surprised that svn and/or cvs is a big feature
<flux> oh, right, hg would be pretty nice to have :). although I don't use it.
<ousado> hg clone I think
<flux> actually I've used a bit darcs, is darcs support in yet?
<thelema> hg clone <repository>?
<thelema> flux: :P give me the command and I'll support it.
<ousado> hg clone [OPTION]... SOURCE [DEST]
<flux> darcs get <repository>
spearalot has joined #ocaml
<hcarty> thelema: It was somewhat facetious... but SVN is a nice feature to have if versioning systems are supported.
<hcarty> There are still a lot of libraries managed in Subversion
<ousado> git support is already there I assume
<thelema> ousado: yes, it's the one I use, so it was the first added
<flux> thelema, how does one invoke the thing? support for tags and branches would be nice, if there isn't yet.
bobzhang has joined #ocaml
<thelema> for cvs, there's two params, "cvs" is the CVSROOT, "cvspath" is path within the repo to checkout
<flux> ..so it complicates things a bit, for example making git requests two-stage
<flux> also, authentication
<thelema> flux: for others, it just checks out whatever's default at that URL
<thelema> AFAIK, each VCS can handle authentication outside odb
letrec has joined #ocaml
<thelema> flux: I can make any of the VCS handle a branchname, but it's simpler to either require it or forbid it.
<hcarty> thelema: Given odb's new VCS support, it might be cool to have a mostly static, public repository available with some libraries in it.
<thelema> hcarty: I'd rather add patching support to odb and have a repo of patches
<hcarty> thelema: A public version of the local package support, strictly or primarily for HEAD versions of projects.
<thelema> hcarty: I don't see much use for forking people's repos
<hcarty> thelema: Oh no, I don't mean fork the repositories
<hcarty> Just putting references to them
<thelema> a public package file?
<hcarty> So rather than having the Batteries git definition sitting in ~/.odb/ it would be in some public location
<thelema> this could be maintained in odb's repo
<thelema> as a single file. each package is one line
<hcarty> thelema: Yes, that's what I was thinking
<thelema> I guess I could have odb check its binary dir for a package file too
<hcarty> Any given definition shouldn't change that often
<hcarty> And new entries could be added at any time
<thelema> I agree that there *needs* to be some way to distribute this build info.
<thelema> also, it'd be a good set of examples for people wanting to add their own.
bobzhang has quit [Quit: ERC Version 5.3 (IRC client for Emacs)]
tautologico has joined #ocaml
letrec has quit [Ping timeout: 256 seconds]
raichoo has quit [Quit: leaving]
zorun has joined #ocaml
_andre has quit [Quit: leaving]
<thelema> ok, initial version commited - needs more explanation and examples. Any ideas on good examples?]
<thelema> good examples for libraries / programs for odb to build and install, that is
Boscop has quit [Ping timeout: 240 seconds]
raichoo has joined #ocaml
<thelema> proper explanation of the packages file put into comments in the example package file
bobzhang has joined #ocaml
bobzhang has quit [Client Quit]
Boscop has joined #ocaml
Boscop has quit [Changing host]
Boscop has joined #ocaml
<thelema> grr, xml-light doesn't install with findlib
<adrien> thelema: used by haaxe?
<adrien> haxe*
<thelema> yes
<adrien> bah
<adrien> and it hasn't been updated for years?
<thelema> it has a proper makefile, but copies itself to `ocamlc -where` for install
<thelema> yes, maybe since 2006
<adrien> afair, it has been said to be "abandonned" by its author (who is also haxe' lead dev) for more than 5 years
<adrien> makes no sense to me ><
<thelema> one commit in feb 07
<adrien> xD
<thelema> I imagine it's in a "works for me" state
fridim_ has joined #ocaml
<adrien> yeah; a bunch of know bugs however; I wish no new project started using it
Boscop has quit [Ping timeout: 240 seconds]
<thelema> odd - I can't `darcs get` the argot sources - some hash failure - can someone verify?
tautologico has quit [Quit: tautologico]
NihilistDandy has quit [Quit: Linkinus - http://linkinus.com]
testcocoon has quit [Quit: Coyote finally caught me]
oriba has joined #ocaml
BiDOrD_ has joined #ocaml
testcocoon has joined #ocaml
BiDOrD has quit [Ping timeout: 240 seconds]
ftrvxmtrx has joined #ocaml
smerz has joined #ocaml
testcocoon has quit [Quit: Coyote finally caught me]
testcocoon has joined #ocaml
raichoo has quit [Ping timeout: 252 seconds]
raichoo has joined #ocaml
zorun has quit [Quit: leaving]
emmanuelux has quit [Remote host closed the connection]
emmanuelux has joined #ocaml
emmanuelux has quit [Read error: Connection reset by peer]
raichoo has quit [Quit: leaving]
spearalot has quit [Quit: Computer has gone to sleep]
zorun has joined #ocaml
Cyanure has quit [Quit: Quitte]
edwin has quit [Remote host closed the connection]
<thelema> hmm, can't install ocamlnet via odb - netstring wants to install twice, once via ocamlfind and then again by itself, I think
emmanuelux has joined #ocaml
claude_ has joined #ocaml
wagle has quit [Read error: Connection reset by peer]
wagle has joined #ocaml
fridim_ has quit [Ping timeout: 268 seconds]
letrec has joined #ocaml
claude_ has quit [Quit: Leaving]
Morphous_ has quit [Ping timeout: 240 seconds]
<thelema> wow, markus mottl's libraries are a pleasure to package with odb. They just work.
<thelema> we as a community have a long way to go making it normal to build on others' libraries
milosn has quit [Read error: Operation timed out]
Morphous_ has joined #ocaml
emmanuelux has quit [Remote host closed the connection]
emmanuelux has joined #ocaml
emmanuelux has quit [Remote host closed the connection]
tautologico has joined #ocaml
<tautologico> is someone using otags or any other tagging solution for ocaml?
<thelema> tautologico: I just use M-/ for emacs completion, and my F3 is bound to ...
<thelema> highlght-symbol-at-point
<tautologico> what if your function is defined in another file?
<tautologico> I mean, if you want to jump to a definition
emmanuelux has joined #ocaml
emmanuelux has quit [Remote host closed the connection]
srcerer has quit [Ping timeout: 252 seconds]
ikaros has quit [Quit: Ex-Chat]
pilki has joined #ocaml
pilki has quit [Client Quit]
Anarchos has joined #ocaml