<edwin>
sgnb: I'd guess that you get EAGAIN, and then the with clause triggers and gives you 0
<flux>
hmm
<flux>
if you read 0 from a descriptor, the peer has closed it
<edwin>
ocamlbuild has a try/with around the read that converts read errors into 0 bytes read
<flux>
but it should rather do: read 0 bytes -> OK, EOF -> wait for the process to finish -> close descriptors, no?
<flux>
oh, so it doesn't really receive EOF then?
<flux>
ah, indeed, it maps all errors to EOF, but EAGAIN would not be an error. but why would the fd be active if it doesn't have either EOF or data?
<edwin>
maybe it read something
<edwin>
then reads again
<flux>
it could also receive EINTR if it receives a signal at an unfortunate point and the automatic system call retry mechanism isn't on (I don't know if it is)
<edwin>
and gets EAGAIN
<edwin>
yeah it would be better to not treat those errors as EOF
<flux>
right, it's in a loop
<flux>
so I assume it has non-blocking socket
<flux>
(or descriptor that is)
<flux>
so it may return EAGAIN and that would be the bug
<thelema>
f[x]: true, they should be packaged together.
<dirkD>
they are not in fedora apparently
<f[x]>
fedora fail
<f[x]>
someone who cares should probably file a bug report
<everyonemines>
I want to implement a bag that elements can be added to, and can return+remove a _random_ element.
<everyonemines>
What's a good way of doing that?
<flux>
ropes perhaps?
<everyonemines>
You mean vects?
<everyonemines>
hmm
thomasga has joined #ocaml
PiepScuim has quit [Remote host closed the connection]
<flux>
some kind of balanced binary tree that knows the number of elements to the right and to the left would also do
<flux>
but that's much more work to implement
<flux>
(wish existing binary trees could just be extended for this kind of functionality)
<flux>
if you do that, please make it two-phase, so that functionality can just be bolted on :-)
<everyonemines>
well, I'm thinking of just using a spare array actually
<everyonemines>
*sparse
<everyonemines>
I like arrays :-)
<flux>
how do you pick a random element?
<everyonemines>
choose a random location, find the first full element from ther
<flux>
meh, O(n) ;)
<everyonemines>
that depends on how full it is
<everyonemines>
it it's half full, it's faster than a btree
<edwin>
wouldn't a hashtable do?
<edwin>
you just pick a random hash value
<edwin>
and see if bucket is empty or not
<everyonemines>
if hashtbl had a remove_min_binding then that might work
<everyonemines>
but it doesn't
<everyonemines>
I guess I could use a btree, add elements with random integer tags, then remove items with the smallest tags
<edwin>
if you want to remove min then a heap is better
<everyonemines>
but that wouldn't be random, actually
<edwin>
as you say, when you insert you insert with a random tag
<edwin>
but I'm not sure if choosing min from a set of random tags yields a random element
<everyonemines>
newer values would have a disproportionate chance of being pulled next because small tag items would be pulled
<edwin>
can't you use a set with a custom compare function
<sgnb>
edwin, flux: on my example, read does return 0 when reading from stdout, and EAGAIN only when reading from stderr
<edwin>
and insert tuples of random_tag, value
<edwin>
then search for another randomly generated tag with the custom compare function
<edwin>
sgnb: I think that is fine, there was no output on stdout hence it reached EOF
<everyonemines>
edwin: Isn't that n^2 though?
<edwin>
everyonemines: Set uses an AVL tree so lookup should be log n
<sgnb>
edwin: but is it right to terminate the job when there is no more output on stdout?
<edwin>
everyonemines: actually you can insert the items with an incrementing counter as first value in the tuple, and just use random value when searching. that way you know for sure you will find something when you search
<edwin>
sgnb: terminate job doesn't actually kill it
<edwin>
sgnb: it just goes to process_terminate, which calls do_read ~loop:true on stdout/stderr
<everyonemines>
edwin: But I want items that are removed when read.
<edwin>
sgnb: waiting until there is no more output from stdout and stderr, and then waiting for process termination with close_process_full
<edwin>
everyonemines: can't Set remove items?
<everyonemines>
you won't be guaranteed to find an item though, you need to retry
<everyonemines>
and your set becomes sparse unless you keep track of which items you've removed
PiepScuim has joined #ocaml
<edwin>
everyonemines: you can remove 2 items: the one you searched for, and the max items. you then readd the max item to fill the gap
<edwin>
and you can keep track of max item with a list
<edwin>
hmm
<edwin>
but taht'd be O(n) to update
<everyonemines>
Hmm, I might do that, but with an array.
<edwin>
everyonemines: or maybe something can be done with skiplists
<edwin>
but I don't have any experience with those
<edwin>
you would keep all items in a skiplist, and then the item's tag would be its position in the list, and the skiplist structure would give you better than O(n) random access
<edwin>
and when you remove an item from a list all other item's position is updated implicitly
<flux>
everyonemines, btw, I don't think your array would give each element the same chance of being chosen
<sgnb>
edwin: you are right, but I still wonder if it is the right thing to do: IIUC, ocamlbuild goes into a mode where it waits for stderr output of a single job, defeating the purpose of this whole mess (i.e. allowing parallel jobs)
<sgnb>
oh
<edwin>
sgnb: true, ocamlbuild doesn't parallelize too well
<sgnb>
this could explain why ocamldep phase is not parallelized
<flux>
ocamlbuild looks like it's lacking one layer of abstraction, that code didn't look very neat..
<edwin>
it should use lwt :)
<sgnb>
since ocamldep output is always redirected, the stdout of the child process is always immediately closed
<flux>
a task should be complete when all the fd's we've given it return EOF and then we can reap them with waitpid from the SIGCHLD handler
<flux>
(actually maybe not even waiting for that is required, if just waitpid from sigchld handler was used)
<edwin>
terminate_job should be made smarter then
<edwin>
it should keep track whether its seen both stdout and stderr closed
<edwin>
err EOFed
<sgnb>
wouldn't it be more correct to wait (non-blockingly) for the child before flushing and closing the parent-side of the pipes?
<edwin>
no, because when the pipes are full
<edwin>
child will block
<edwin>
you have to keep reading from the pipes
<everyonemines>
Oh, I just found out that fork() doesn't work on windows.
<everyonemines>
:-(
<sgnb>
I didn't talk about stopping reading from the pipes
<everyonemines>
create_process instead, huh
<edwin>
sgnb: then you probably want pselect() instead of select
<sgnb>
if there is sth to read, select will return and read will return stuff
<edwin>
is that accessible in ocaml?
<edwin>
but again that relies on child sending SIGCHLD, don't know what happens on Windows (does ocamlbuild work on windows?)
<edwin>
sgnb: any reason not to use threads?
<edwin>
one thread to deal with the pipes, and one thread to spawn/wait for the children
<sgnb>
I was not clear... what I meant is: do select(), read from ready fds, on EOF, try to (non-blockingly) wait for the child and if it succeeds, then terminate the job, and in the end, then go back to select
<edwin>
ah with WNOHANG?
<sgnb>
yes
<sgnb>
I don't think threads are needed for that
<edwin>
right
<edwin>
won't have time to write a patch until the weekend though, so if you want to take a shot at it
<edwin>
actually I don't know how much the Inria team likes patches, maybe they'd like to write that themselves
<edwin>
as it involves a bit of code refactoring
<edwin>
it'd probably improve ocamlbuild's ability to parallelize a lot
<edwin>
if it can run ocamldep steps earlier it can start rest of compilation earlier
<sgnb>
edwin: and about windows, this code path seems to be enabled only on unix
<sgnb>
(I mean, the whole ocamlbuild_executor module)
<edwin>
then we could use SIGCHLD as flux suggested
fraggle_ has joined #ocaml
ztfw has joined #ocaml
<sgnb>
actually, this was my first suggestion in the bugreport :-)
<sgnb>
and now, I think that my non-blocking wait() stuff might not be enough
<sgnb>
what a crappy interface
<sgnb>
I will summarize this in a comment to the bugreport
everyonemines has quit [Quit: Leaving.]
PiepScuim has quit [Ping timeout: 252 seconds]
seafood has joined #ocaml
seafood_ has joined #ocaml
seafood has quit [Read error: Connection reset by peer]
seafood_ is now known as seafood
Associat0r has joined #ocaml
Associat0r has quit [Changing host]
Associat0r has joined #ocaml
sebz has quit [Quit: Computer has gone to sleep.]
Cyanure has quit [Remote host closed the connection]
emmanuelux has joined #ocaml
edwin has quit [Quit: Leaving.]
edwin has joined #ocaml
lopex has quit []
seafood has quit [Quit: seafood]
fracek has joined #ocaml
edwin has quit [Remote host closed the connection]
edwin has joined #ocaml
sebz has joined #ocaml
surikator has quit [Quit: Scientific discovery is just maximal compression of strings. Nothing more, nothing less.]
ulfdoz has joined #ocaml
avsm has joined #ocaml
lopex has joined #ocaml
Kakadu has joined #ocaml
sebz has quit [Quit: Computer has gone to sleep.]
surikator has joined #ocaml
dancannon has joined #ocaml
<dancannon>
Hi, is there anyway to do something like this in ocaml? min(min(min(a, b), c), d)
surikator has quit [Quit: Scientific discovery is just maximal compression of strings. Nothing more, nothing less.]
<testcocoon>
dancannon: min (min (min a b) c) d) for example
<dancannon>
testcocoon: Hmm syntax error?
<_habnabit>
just one too many )s
<Kakadu>
List.fold_left min a [b;c;d]
<Kakadu>
a little bit crazy solution
<_habnabit>
or, even better, List.min [a; b; c; d]
<dancannon>
I wish I could use list or array
<dancannon>
Its an exercise for uni. I am only allowed to use Pervasives and for this exercise im not allowed to use any if statements
<_habnabit>
then ask your TA
<_habnabit>
(not us)
<dancannon>
Oh right sorry
<dancannon>
I was just asking what the correct way to do multiple function calls
<_habnabit>
same as one function call
<_habnabit>
just... multiple times
<dancannon>
Well its working now thanks
<Kakadu>
let (>>=) = min;;
<Kakadu>
1 >>= 2 >>= 3 >>= 4;;
<dancannon>
Hmm that tidies it up thanks
<Kakadu>
dancannon: 1 more example
dancannon has quit [Quit: ChatZilla 0.9.87 [Firefox 6.0.2/20110902133214]]
avsm has quit [Quit: Leaving.]
hyperboreean has quit [Ping timeout: 240 seconds]
hyperboreean has joined #ocaml
ttamttam has quit [Remote host closed the connection]
everyonemines has joined #ocaml
<everyonemines>
are there any tricks for fast bigarray operations?
<flux>
do them in C?-)
<thelema>
everyonemines: not really - what are you wanting to do?
<everyonemines>
write and read arrays <-> bigarray sections
<flux>
for-loops perhaps
<thelema>
everyonemines: only way to do that is iter, I think
<everyonemines>
and iteration over bigarray sections of simple functions
yezariaely has joined #ocaml
<everyonemines>
basically bigarray access and writes are slower than it seems like they should be
<thelema>
there's an extra pointer dereference that ocamlopt doesn't optimize out, could that cause your observation?
ttamttam has joined #ocaml
<everyonemines>
well it seems like bigarray iteration should be faster than array iteration
<everyonemines>
but it isn't?
<thelema>
correct, array iteration is actually faster because of this indirection.
<everyonemines>
what do you mean, because of
<everyonemines>
surely there's something else in bigarray functions that's causing the slowdown
<everyonemines>
not the lack of indirection
<thelema>
oh, yes. you need to specify the type of the bigarray
<thelema>
if your function is generic, ocaml'll produce much worse code for it
<thelema>
that was the other thing that specifying types helps for - bigarrays
<thelema>
and arrays too - avoiding the runtime check for float arrays
<thelema>
comparison and array indexing
<thelema>
(are the two things helped by type annotations)
<everyonemines>
so I need to define bigarray_function_float and _int ?
<thelema>
if you use it on differently typed bigarrays, yes
<everyonemines>
is there a way to do that without copy-paste?
<thelema>
preprocessing
<everyonemines>
not that it's a big deal, I mean, just on principle
<everyonemines>
camlp4 huh
<everyonemines>
maybe there's an existing p4 module for that
<thelema>
but ocamlopt will need to see two functions to generate two hunks of code
<thelema>
probably no existing code for this
<thelema>
I was thinking something more like sed, not camlp4
<everyonemines>
haha
<everyonemines>
so if I do that, bigarrays are at least as fast as arrays?
<thelema>
they're properly competitive except for that last dereference
<everyonemines>
logically they should be faster :-(
<thelema>
they can be slower, related to unpacking and re-packing bytes into words for computation
thomasga has quit [Quit: Leaving.]
ttamttam has quit [Quit: ttamttam]
larhat has quit [Quit: Leaving.]
<everyonemines>
Well, I know who I'm going to blame.
<everyonemines>
VON NEUMANN!! :x
tlockney_ has joined #ocaml
everyonemines has quit [Quit: Leaving.]
lpereira has joined #ocaml
serpentologist has joined #ocaml
Associat0r has quit [Quit: Associat0r]
bobry has joined #ocaml
metasyntax|work has quit [Quit: WeeChat [quit]]
joewilliams_away is now known as joewilliams
dirkD has quit [Ping timeout: 258 seconds]
yezariaely has quit [Quit: Leaving.]
Kakadu has quit [Quit: Konversation terminated!]
ztfw has quit [Remote host closed the connection]
_andre has quit [Quit: leaving]
ulfdoz has quit [Read error: Operation timed out]
fracek has quit [Quit: You are not prepared!]
sbi_ has joined #ocaml
bobry has quit [Ping timeout: 255 seconds]
sbi_ is now known as rferranti
avsm has joined #ocaml
bobry has joined #ocaml
edwin has quit [Remote host closed the connection]
joewilliams is now known as joewilliams_away
lpereira has quit [Quit: Leaving.]
sebz has joined #ocaml
seafood has joined #ocaml
ikaros has quit [Quit: Ex-Chat]
avsm has quit [Quit: Leaving.]
dnolen has joined #ocaml
schme has quit [Ping timeout: 258 seconds]
seafood has quit [Quit: seafood]
Amorphous has quit [Read error: Operation timed out]
Amorphous has joined #ocaml
lamawithonel has quit [Remote host closed the connection]
lamawithonel has joined #ocaml
dgbaley27 has joined #ocaml
<dgbaley27>
It might be the runtime wrapper that was given, but it seems to me like Printf.printf and Printf.eprintf don't come out interleaved. All of stderr is coming after stdout. Does this sound right, and is there a way around this?