adrien changed the topic of #ocaml to: Discussions about the OCaml programming language | http://caml.inria.fr/ | OCaml 4.00.1 http://bit.ly/UHeZyT | http://www.ocaml.org | Public logs at http://tunes.org/~nef/logs/ocaml/
oriba_ has joined #ocaml
oriba has quit [Read error: Operation timed out]
oriba_ has quit [Client Quit]
cfy has joined #ocaml
ollehar has quit [Ping timeout: 276 seconds]
yacks has joined #ocaml
adotbrown has joined #ocaml
ulfdoz_ has joined #ocaml
ulfdoz has quit [Ping timeout: 276 seconds]
ulfdoz_ is now known as ulfdoz
cfy has quit [Read error: Connection reset by peer]
cfy has joined #ocaml
chambart has quit [Ping timeout: 256 seconds]
tac has joined #ocaml
madroach has quit [Ping timeout: 248 seconds]
madroach has joined #ocaml
q66 has quit [Remote host closed the connection]
cfy has quit [Ping timeout: 264 seconds]
gnuvince has joined #ocaml
ahokaomaeha has quit [Ping timeout: 264 seconds]
yezariaely has quit [Quit: changing servers]
Yoric has joined #ocaml
yezariaely has joined #ocaml
Yoric has quit [Ping timeout: 252 seconds]
pkrnj has joined #ocaml
tac has quit [Ping timeout: 245 seconds]
ahokaomaeha has joined #ocaml
tautologico has quit [Quit: tautologico]
ahokaomaeha has quit [Ping timeout: 245 seconds]
pkrnj has quit [Quit: Textual IRC Client: www.textualapp.com]
myx has quit [Ping timeout: 260 seconds]
ahokaomaeha has joined #ocaml
leoncamel has joined #ocaml
yacks has quit [Quit: Leaving]
gnuvince has quit [Remote host closed the connection]
gnuvince has joined #ocaml
dabd_ has joined #ocaml
dabd has quit [Ping timeout: 252 seconds]
dabd_ has quit [Ping timeout: 256 seconds]
ttamttam has joined #ocaml
ttamttam has quit [Remote host closed the connection]
cfy has joined #ocaml
cfy has quit [Remote host closed the connection]
cfy has joined #ocaml
Snark has joined #ocaml
yacks has joined #ocaml
osa1 has joined #ocaml
<osa1> is there a way to generate sexp functions for OCaml's standard modules like Set or Map
mye has quit [Quit: mye]
<adrien> well, Set and Map or Set.Make(...) and Map.Make(...) ?
<adrien> the first one doesn't really make sense
<adrien> the second one should have upstream support
<osa1> second one
<osa1> upstream support ?
nimred has joined #ocaml
mye has joined #ocaml
adotbrown has quit [Ping timeout: 260 seconds]
ontologiae has joined #ocaml
Yoric has joined #ocaml
ttamttam has joined #ocaml
<osa1> adrien: can you give me more pointers please ?
<osa1> I'm still looking for it
Yoric has quit [Ping timeout: 252 seconds]
<adrien> hmm, I thought there was something built-in but apparently not
<adrien> and in my own code I use lists
<osa1> adrien: actually I was happy with lists until I had to write all set functions on lists myself
<adrien> well, I meant you could map -> list -> sexp -> list -> map
<adrien> that's more cpu-time but might not matter much to you
<osa1> adrien: I'll remove all sexp functions after debugging
<adrien> actually it's not more cpu time
<osa1> adrien: my problem is it's still requires lots of boilerplate
<adrien> because the unserialization from sexp to map would have the same cost (insert elements one by one)
ontologiae has quit [Ping timeout: 256 seconds]
ocp has joined #ocaml
cdidd has joined #ocaml
yacks has quit [Ping timeout: 244 seconds]
Yoric has joined #ocaml
yacks has joined #ocaml
cfy has quit [Quit: ERC Version 5.3 (IRC client for Emacs)]
mika1 has joined #ocaml
Oaks has joined #ocaml
hkBst has joined #ocaml
hkBst has quit [Changing host]
hkBst has joined #ocaml
Cyanure has joined #ocaml
Oaks has left #ocaml []
AltGr has joined #ocaml
djcoin has joined #ocaml
pango has quit [Ping timeout: 260 seconds]
pango has joined #ocaml
mye has quit [Quit: mye]
Yoric has quit [Quit: Instantbird 1.4a1pre -- http://www.instantbird.com]
Sergey_ has joined #ocaml
sivoais has quit [Ping timeout: 245 seconds]
sivoais has joined #ocaml
mcclurmc has quit [Ping timeout: 248 seconds]
cago has joined #ocaml
Yoric has joined #ocaml
osa1 has quit [Ping timeout: 245 seconds]
mye has joined #ocaml
cago has quit [Read error: Connection reset by peer]
cago has joined #ocaml
cago has quit [Remote host closed the connection]
cago has joined #ocaml
mye has quit [Ping timeout: 248 seconds]
Kakadu has joined #ocaml
adotbrown has joined #ocaml
thomasga has joined #ocaml
adotbrown has quit [Ping timeout: 255 seconds]
Reventlov has quit [Ping timeout: 272 seconds]
mcclurmc has joined #ocaml
anderse has joined #ocaml
mye has joined #ocaml
leoncamel has quit [Ping timeout: 252 seconds]
Reventlov has joined #ocaml
mye has quit [Ping timeout: 248 seconds]
leoncamel has joined #ocaml
adrien_o1w is now known as adrien_oww
f[x] has quit [Quit: Leaving]
mcclurmc has quit [Ping timeout: 252 seconds]
_andre has joined #ocaml
reynir_ is now known as reynir
reynir has quit [Changing host]
reynir has joined #ocaml
mcclurmc has joined #ocaml
mye has joined #ocaml
dwmw2_gone is now known as dwmw2
adotbrown has joined #ocaml
chambart has joined #ocaml
mye has quit [Ping timeout: 252 seconds]
Yoric has quit [Ping timeout: 264 seconds]
ttamttam has quit [Quit: ttamttam]
mye has joined #ocaml
hkBst has quit [Ping timeout: 248 seconds]
chambart has quit [Ping timeout: 256 seconds]
hkBst has joined #ocaml
ontologiae has joined #ocaml
mye has quit [Ping timeout: 248 seconds]
yacks has quit [Ping timeout: 248 seconds]
Yoric has joined #ocaml
q66 has joined #ocaml
ontologiae has quit [Read error: Connection reset by peer]
ontologiae has joined #ocaml
Cyanure has quit [Remote host closed the connection]
mye has joined #ocaml
<mfp> avsm: there was yet another gotcha related to finalisers: preemption of lwt threads in critical areas (i.e. without explicit context switches)
ahokaomaeha has quit [Ping timeout: 255 seconds]
<avsm> mfp: do you mean due to Lwt.wakeup, or underlying pthreads?
hkBst has quit [Ping timeout: 276 seconds]
<avsm> mfp: i've tended to ban Lwt.wakeup now, as the Lwt-level switching is just too dangerous (and Async guarantees the same behavior too)
<mfp> avsm: if you're not careful, a finaliser can be invoked at any time by a GC run, which can happen in the middle of code where you would not expect a context switch, i.e. without any calls to lwt's machinery
<avsm> hm, I wonder how Async deals with that
<mfp> well, in safe-finalizer I just make sure to Lwt_unix.yield () before the actual finaliser
<avsm> (* Ocaml permits finalisers to be run in any thread and at any time after the object
<avsm> becomes unreachable -- they are essentially concurrent. This changes forces all
<avsm> finaliser code to run sequentially and in a fixed thread. *)
<avsm> so Async has a dedicated finaliser thread too
<mfp> I just force the finalisers to be in the lwt monad and use the main thread: https://github.com/mfp/safe-finaliser/blob/master/src/safe_finaliser.ml
ttamttam has joined #ocaml
cfy has joined #ocaml
<avsm> this really ought to be in Lwt upstream indeed...
mye has quit [Ping timeout: 256 seconds]
adotbrown has quit [Ping timeout: 248 seconds]
<mfp> diml: ^^^^^^^^
f[x] has joined #ocaml
ttamttam has quit [Quit: ttamttam]
f[x] has quit [Read error: Connection reset by peer]
ttamttam has joined #ocaml
hkBst has joined #ocaml
f[x] has joined #ocaml
ttamttam has quit [Quit: ttamttam]
ttamttam has joined #ocaml
yacks has joined #ocaml
hkBst has quit [Ping timeout: 255 seconds]
|jbrown| has quit [Remote host closed the connection]
mye has joined #ocaml
hkBst has joined #ocaml
<diml> mfp: it is already safe in lwt
<diml> the finalizer just save the value into an option and send a notification
<diml> also note that using Lwt_unix.yield in a finalizer may not be safe even if you are in the main thread: it does an allocation and so may run the gc
mye has quit [Ping timeout: 248 seconds]
mcclurmc has quit [Ping timeout: 240 seconds]
Yoric has quit [Ping timeout: 252 seconds]
jbrown has joined #ocaml
mye has joined #ocaml
<mfp> diml: is this a recent change? with lwt 2.4.3, I observe both finalisers being executed outside the main thread and "surprise" context switches
<mfp> you're very right about Lwt_unix.yield (), but can't yet see any way around it. The minor heap was just emptied, but if there are many finalisers executed in the same run or they allocate a lot we can't prevent another GC run, can we?
<mfp> sorry, the above is with the new finaliser reg function, here's the one with Lwt_gc.finalise -> http://paste.ubuntu.com/1657397/
<mfp> my bad, I was running that code in another box with 2.4.0; 2.4.3 seems fine, and I can see you've changed the code
Thorn has quit [Ping timeout: 264 seconds]
mye has quit [Ping timeout: 256 seconds]
q66 has quit [Remote host closed the connection]
q66 has joined #ocaml
q66 has quit [Remote host closed the connection]
q66 has joined #ocaml
q66 has quit [Remote host closed the connection]
q66 has joined #ocaml
Sergey_ has quit [Read error: Connection reset by peer]
mye has joined #ocaml
ahokaomaeha has joined #ocaml
tane has joined #ocaml
fasta_ is now known as fasta
mye has quit [Ping timeout: 248 seconds]
chambart has joined #ocaml
Yoric has joined #ocaml
tautologico has joined #ocaml
mye has joined #ocaml
dabd has joined #ocaml
jbrown has quit [Remote host closed the connection]
Yoric has quit [Ping timeout: 252 seconds]
q66 has quit [Remote host closed the connection]
q66 has joined #ocaml
mye has quit [Ping timeout: 256 seconds]
q66 has quit [Remote host closed the connection]
ontologiae has quit [Ping timeout: 252 seconds]
travisbrady has joined #ocaml
q66 has joined #ocaml
karswell has quit [Read error: Connection reset by peer]
travisbrady has quit [Client Quit]
q66 has quit [Remote host closed the connection]
smondet has joined #ocaml
travisbrady has joined #ocaml
q66 has joined #ocaml
jbrown has joined #ocaml
Cyanure has joined #ocaml
mye has joined #ocaml
mcclurmc has joined #ocaml
mye has quit [Ping timeout: 248 seconds]
darkf has quit [Quit: Leaving]
dabd has quit [Quit: Leaving]
ttamttam has left #ocaml []
S11001001 has joined #ocaml
S11001001 has quit [Changing host]
S11001001 has joined #ocaml
mye has joined #ocaml
cfy has quit [Quit: ERC Version 5.3 (IRC client for Emacs)]
Kakadu has quit [Quit: Page closed]
hkBst has quit [Quit: Konversation terminated!]
ahokaomaeha has quit [Ping timeout: 248 seconds]
mye has quit [Ping timeout: 256 seconds]
q66 has quit [Remote host closed the connection]
q66 has joined #ocaml
mcclurmc has quit [Ping timeout: 255 seconds]
Yoric has joined #ocaml
anderse has quit [Quit: anderse]
mye has joined #ocaml
osa1 has joined #ocaml
djcoin has quit [Quit: WeeChat 0.3.9.2]
travisbrady has quit [Quit: travisbrady]
tac has joined #ocaml
mye has quit [Ping timeout: 248 seconds]
travisbrady has joined #ocaml
cago has left #ocaml []
mika1 has quit [Quit: Leaving.]
myx has joined #ocaml
dwmw2 is now known as dwmw2_gone
mye has joined #ocaml
travisbrady has quit [Quit: travisbrady]
travisbrady has joined #ocaml
Guest34021 has quit [Ping timeout: 255 seconds]
Guest34021 has joined #ocaml
yacks has quit [Quit: Leaving]
ollehar has joined #ocaml
Guest34021 has quit [Ping timeout: 245 seconds]
<osa1> does anyone here know a good introduction to HM(X) algorithm ?
<osa1> other than the part in "advanced topics in types and programming languages" by pottier and odersky's phd thesis ?
Guest34021 has joined #ocaml
thomasga has quit [Quit: Leaving.]
m4b has joined #ocaml
<m4b> i've got a program which uses hashtables of arrays and does a lot of updating; when i profile the program, caml_initialize occupies 40-50% of run-time. From what I understand, this is the garbage collector. This is unacceptable. Is there anyway to change this? Is it because I'm using mutable objects? Would switch the hashtable implementation to Maps, for example, change the matter?
<f[x]> caml_initialize has nothing to do with gc
<m4b> f[x]: ok, that's good, the internet lied to me, and then it means perhaps I can do something about the run time. What does it do?
<mfp> m4b: it is called to initialize arrays
<mfp> that is, initialize the elements
<_andre> mfp: thanks for the lwt tip yesterday, i think i found my memory leak :)
<mfp> m4b: do you often copy arrays or such?
<mfp> _andre: good to hear, was it exactly the Lwt.choose thing?
<_andre> not exactly like your example, but i was using choose when i should be using pick instead
<m4b> mfp: it is a vm that is running a benchmark; here is the gprof flat output http://hpaste.org/82420
<_andre> so i had a bunch of threads that were never dying
<f[x]> gprof is useless, use pmp
<m4b> mfp: the vm is using a hashtable of arrays; in the benchmark, it's making and setting a lot of the elements in the arrays. There is a haskell implementation that is running significantly faster, although our two implementations essentially differ in no significant details. This is somewhat perplexing.
<f[x]> show the code
<mfp> camlArray__copy_1031 467110 caml_array_gather 467110 -> caml_alloc_shr 828735 -> caml_initialize 1204619769
<m4b> f[x]: ok, I will put the hashtable version up; I'm currently switching out the hashtables for a map, don't think it'll make much of a difference though...
<mfp> do you happen to have around 1400 elements per array on average? :)
ocp has quit [Ping timeout: 252 seconds]
<mfp> m4b: it's the Array.copy/sub that's killing you
* mfp brb
<m4b> mfp: (1) I don't know the average amount of elements in the array. for the benchmark inary it runs, I know it copies an array with 11650 instructions. (2) ok, the only Array.cop operation is on the load op; I _need_ to return a fresh copy, otherwise it alias, which is no good. I don't Array.sub anywhere...
<m4b> mfp: also, did you figuer out I am calling Array.sub from the gprof lines you pasted above? f[x]: sorry, not familiar with the profiler pmp; running pacman -Ss pmp returns no results either, so dunno...
<mfp> m4b: a guess, other choices (= functions that call caml_array_gather) are caml_array_append and _concat
<m4b> mfp: cool, good to know :)
<mfp> this looks like a likely culprit let program = Array.make words (instruction_of_hex 0x0) in
<mfp> since you use a boxed type for the instructions, it's going to make sure the values are added to the remembered set
leoncamel has quit [Ping timeout: 252 seconds]
<m4b> mfp: why is it a culprit, and how else would I initialize a new array?
<mfp> if you kept it in the original format, with 4 consecutive ints representing an instruction, the compiler might be able to avoid caml_initialize
<m4b> mfp: are you suggesting I don't represent the instruction as a record type, or as any type at all?
<mfp> caml_initialize is called to add each element of the array being initialized to the remembered set (for GC purposes)
<mfp> indeed, it should be much faster if you avoid using a boxed type
<m4b> mfp: ok I am sort of understanding; for the instruction type, you say that it is a "boxed type"; could you elaborate on this poit more?
<m4b> mfp: also, thank you very much for the detailed and insightful feedback
<mfp> the instruction record is represented as a block w/ 5 words (1 for header, 4 integer fields)
ttamttam has joined #ocaml
<mfp> then your array of instructions becomes an array of pointers to such blocks
Anarchos has joined #ocaml
<m4b> ok
<mfp> which is costly because (1) initialization is more expensive (caml_initialize is called on each element of the array, to be more precise on each position of the array) and (2) you have an indirection each time you read an instruction
<m4b> mfp: so how do you recommend refactoring the "instruction" type? simply passing in the 32-bit number?
<mfp> which can easily mean a cache miss
<m4b> *"instruction" type... and associated functions/code
<mfp> if you're running a 64-bit ocaml, that'd be indeed the easiest solution
<m4b> mfp: I am; if I wasn't, however, or I wanted it to be portable to 32-bit machines, what would you suggest?
<mfp> if you're on a 32-bit platform, it depends: if 31-bit are enough (OCaml takes 1 for tagging), int arrays are fine, otherwise you have to use the bigarray lib
<m4b> mfp: exact 32-bits are necessary; well i'll leave that for now, because I'm sort of cheating with the nand implementation, it masks out the most significant 32-bits to get the unsigned 32-bit number; not sure how to go about emulating unsigned 32-bits on a 32-bit machine in ocaml off the top of my head
<m4b> mfp: but to be clear: you are suggesting that I refactor all of the functions for instructions; in other words, remove hex of instruction, etc., and simply deal with the raw 32-bit instruction directly.
<m4b> on hindsight that makes more sense, heh.
<_andre> m4b, if you find the need of unsigned ints, https://forge.ocamlcore.org/projects/ocaml-uint/
<mfp> m4b: that should help a lot, yes
<m4b> _andre: thanks for the link; I looked around for something simple, but ended up doing the nand hack i mentioned above, since that was the only specific case where the ocaml rep wasn't working (i.e., ¬ (0x0 ∧ 0x0) = -1)
<mfp> the other funny thing is your use of Hashtbl to hold blocks of mem
<m4b> mfp: I'm a funny guy ;) I couldn't think of a better way to emulate the deallocate operation
<mfp> you probably want to use a custom Hashtbl with a better hashing function, or another structure like patricia trees
<m4b> mfp: if you have a better suggestion for a different data structure, I'm all ears. I don't like that my code is slower than the haskell version ;)
f[x] has quit [Quit: Leaving]
<m4b> from url: " La page ou le document demandé n'est plus disponible!.... "
<m4b> domage ;)
<mfp> you can also use DIET sets or something to keep the unused addresses (or just keep allocating n + 1)
<m4b> ah, it was the /mli on the second
<m4b> ok; I will try to implement the patricia trees asap; but forgive me ignorance, I'm not too great with module syntax, etc. For example, I have trouble with the map module, and getting it to use ints as keys, i get syntax errors and shit. but that' another issue.
<mfp> you can instantiate the module with something like module M = Map.Make(struct type t = int let compare = (-) end);; <- note that this compare is NOT correct, though (overflow)
<m4b> mfp: so I have a question; for the instruction of hex, would it be more efficient to just return a 4-tuple of the masked bits? or is it smarter to get the bits individually per request, so to speak
<mfp> the latter
<m4b> mfp: so 4-tuples will have similar issues?
<mfp> creating a tuple is like creating a record, it means allocating a 5-word block
<m4b> mfp: ah, ok, thank you for the clarification
<mfp> gtg now, have fun :)
<m4b> mfp: great thank you for the advice and suggestions, merci!
<Anarchos> m4b what are you programming ?
<m4b> Anarchos: it's a vm for an invented architecture: spec, binaries, benchmark here: http://www.boundvariable.org/task.shtml
<Anarchos> m4b ok
<Anarchos> m4b i almost did one for java in ocaml :)
<m4b> Anarchos: hah damn!
<Anarchos> m4b oh icfp !!! cheater to make us work for you :)
<m4b> Anarchos: from 2006
Reventlov has quit [Quit: leaving]
<Anarchos> m4b well you are late ;)
<m4b> Anarchos: hehe, story of my life :P
cdidd has quit [Ping timeout: 248 seconds]
<m4b> speaking of which, I need to go; maybe if i can implement changes, I'll be back later with some good results on speedup :) thanks again everyone
ahokaomaeha has joined #ocaml
m4b has quit [Ping timeout: 252 seconds]
cdidd has joined #ocaml
tautologico has quit [Quit: tautologico]
mcsquiggedy has joined #ocaml
jbrown has quit [Remote host closed the connection]
chambart has quit [Ping timeout: 248 seconds]
_andre has quit [Quit: leaving]
pango has quit [Disconnected by services]
pango has joined #ocaml
ocp has joined #ocaml
smondet has quit [Remote host closed the connection]
ahokaomaeha has quit [Quit: When I come back, please tell me in what new ways you have decided to be completely wrong.]
tane has quit [Quit: Verlassend]
smondet has joined #ocaml
gnuvince has quit [Ping timeout: 248 seconds]
mcsquiggedy has quit [Ping timeout: 245 seconds]
mcsquiggedy has joined #ocaml
ttamttam has quit [Remote host closed the connection]
Snark has quit [Quit: Quitte]
mcclurmc has joined #ocaml
ontologiae has joined #ocaml
osa1 has quit [Ping timeout: 244 seconds]
<mcsquiggedy> If I were to construct a Lexing.lexbuf from a pipe, the parsing would block until there was content available, yes?
<mcsquiggedy> I'm assuming that the parsing process is built using the input_char function
<jonafan> dart isn't blowing me away
metasyntax has quit [Quit: Leaving]
<orbitz> jonafan: covariant arrays, those fools
Yoric has quit [Ping timeout: 264 seconds]
cdidd has quit [Ping timeout: 252 seconds]
<jonafan> the javascript interop is pretty useless
<orbitz> how come?
<jonafan> dart is meant to run in its own vm, so i guess it's a little bit like doing RPC
<orbitz> ah
<jonafan> adding type checking to existing js libraries would mean wrapping js.Proxy objects in dart objects
<orbitz> Dart also is into actors right?
<jonafan> i haven't looked into much other than js interop
<jonafan> also, it seems to produce about 400k of js for hello world
smondet has quit [Ping timeout: 248 seconds]
<jonafan> ah, i guess a lot of that is overhead from the js interop lib
<jonafan> 72k is bearable
<ousado> jonafan: doesn't dart also compile to JS?
mcsquiggedy has quit [Read error: Connection reset by peer]
emmanuelux has joined #ocaml
<ousado> hm interop..
rwmjones is now known as rwmjones|afk
ocp has quit [Ping timeout: 252 seconds]
Yoric has joined #ocaml
LukeSun has joined #ocaml
LukeSun has left #ocaml []
emmanuelux has quit [Ping timeout: 248 seconds]
cdidd has joined #ocaml
Anarchos has quit [Quit: Vision[0.9.7-H-090423]: i've been blurred!]
Cyanure has quit [Remote host closed the connection]
emmanuelux has joined #ocaml
tautologico has joined #ocaml
Yoric has quit [Ping timeout: 252 seconds]
<orbitz> whoop, very initial protobuf support working!
<nicoo> orbitz: You are working on a protobuf binding for OCaml ?
<orbitz> yes
adotbrown has joined #ocaml
S11001001 has quit [Quit: ERC Version 5.3 (IRC client for Emacs)]
ollehar has quit [Ping timeout: 252 seconds]
<dsheets> suggestions for parameter-space search parallelization libraries?
travisbrady has quit [Quit: travisbrady]
<dsheets> I could retrofit an old process/pipe system but i have a sneaking suspicion someone has constructed a nice library to do just this. Suggestions?
AltGr has quit [Quit: Konversation terminated!]
jbrown has joined #ocaml
* dsheets starts messing with parmap
tac has quit [Ping timeout: 245 seconds]
ontologiae has quit [Ping timeout: 244 seconds]
cdidd has quit [Ping timeout: 252 seconds]
travisbrady has joined #ocaml
cdidd has joined #ocaml
travisbrady has quit [Quit: travisbrady]
travisbrady has joined #ocaml