<det>
I've also been playing with eclipse ocaml plugin
<det>
I've been using emacs forever
<det>
the plugin + _tags method works with it
<flux>
it doesn't link xml-light in
<flux>
so: Error while linking autotv.cmo: Reference to undefined global `Xml'
<flux>
apparently it does use it while compiling it, otherwise it would be compile time error..
<flux>
ocamlfind ocamlc -o autotv -linkpkg -package xml-light,netclient autotv.ml produces the binary without problems
alexyk has joined #ocaml
<flux>
while compiling it uses: ocamlfind ocamlc -c -package netclient -package xml-light -o autotv.cmo autotv.ml and while linking it uses ocamlfind ocamlc -linkpkg autotv.cmo -o autotv.byte
<flux>
the first one is right, the second isn't
<det>
<det> I think your program will fail to link without the *.byte and *.native
alexyk has quit []
<det>
try: "<*.ml> or <*.byte> or <*.native>: pkg_xml-light, pkg_netclient"
<det>
flux, ping
<det>
*.ml is just used when comping the cmx/cmo (ie. ocamlc -c file.ml)
<det>
.native and .byte are used during linking
<flux>
ah
<flux>
yes, that works
<flux>
the example in the page (<*.ml> or "prog.byte": pkg_nums, pkg_str
<flux>
) didn't make sense to me
<flux>
as I didn't understand I needed the <> there too
<flux>
but, does the myocamlbuild.ml-mechanism strike as inelegant to anyone?
<flux>
wouldn't it be better if there was a directory with a much of ml-files which would be taken as the whole myocamlbuild.ml?
<flux>
bunch, not much :)
<det>
that would be nicer, yes
<det>
I'd prefer something like Python distutils
<flux>
also there should be a possibility for system/user-wide override of such files
<det>
a setup.ml that drives everything
<flux>
is myocamlbuild.ml used with a preprocessor that supports including other files?
<flux>
that is, how easy would it be to emulate that behavior..
alexyk has joined #ocaml
<det>
I have no idea how ocamlbuild works, to be honest.
<det>
But that is a good thing, it means I have not been forced to learn much about it. Just dropping in that file and editing _tags has worked for me so far.
<det>
But yeah, It does strike me as inelegant, the ocamlfind/ocamlbuild integration, yes.
<flux>
perhaps I should take a look how batteries uses it
jeremiah has quit [Read error: 104 (Connection reset by peer)]
filp has joined #ocaml
Yoric[DT] has joined #ocaml
mishok13 has joined #ocaml
alexyk has quit []
velco has quit ["Ex-Chat"]
_zack has joined #ocaml
ysneakers has quit [Read error: 110 (Connection timed out)]
velco has joined #ocaml
ulfdoz has joined #ocaml
Kerris0 has joined #ocaml
_zack has quit ["Leaving."]
seafood has joined #ocaml
<Yoric[DT]>
hi
<flux>
hello
Kerris0 has quit [Read error: 113 (No route to host)]
_zack has joined #ocaml
Camarade_Tux has quit [Read error: 110 (Connection timed out)]
Camarade_Tux has joined #ocaml
<Yoric[DT]>
_zack: I've just committed a small change to make your patch work with GODI.
struktured_ has joined #ocaml
<Yoric[DT]>
Mmmhh....
<Yoric[DT]>
Can anyone think of a situation in which once an input channel has reached its end we might wish to not close it?
<gildor>
Yoric[DT]: doing a seek back to the beginning ?
<Yoric[DT]>
Mmmhhh....
<Yoric[DT]>
Yes, possibly.
* Yoric[DT]
wonders if it wouldn't be interesting to add an option to auto-close inputs once the end has been reached.
<Yoric[DT]>
That would simplify some programs.
<mfp>
Yoric[DT]: also, when you hold a lock on the fd?
struktured has quit [Read error: 110 (Connection timed out)]
<Yoric[DT]>
mfp: what do you mean?
<mfp>
the lock is released when you close the fd
<Yoric[DT]>
Ah, ok.
<mfp>
admittedly contrived, but it could lead to bad surprises
<Yoric[DT]>
Gasp, I'm currently being bitten by a difference of package names between Debian and Godi.
<mfp>
Yoric[DT]: what about close on GC?
<Yoric[DT]>
You mean holding a weak reference and all that?
<Yoric[DT]>
Strikes me as a bad idea.
<mfp>
just using Gc.finalise
<Yoric[DT]>
(still)
<Yoric[DT]>
Assume channel [a] reading from channel [b].
<Yoric[DT]>
If you close [a], this will presumably close [b].
<Yoric[DT]>
However, [a] may become inaccessible while you still have need of [b].
<Yoric[DT]>
(for the reasons mentioned above)
<mfp>
why would a close b?
<mfp>
if you just let each channel close itself when it's reclaimed, this example works
<Yoric[DT]>
Mmmmhhhhh....
<mfp>
hmmm
<mfp>
the problem is caused by [b] being used in 2 contexts: within [a], and somewhere else
<mfp>
but this is also problematic if we are closing manually
<mfp>
if closing [a] implies closing [b
<mfp>
[b]
<mfp>
but we're still using [b] somewhere else
<Yoric[DT]>
Well, it's the same issue as above. Contrived but possible.
<Yoric[DT]>
I actually believe that closing [a] should also close [b].
<Yoric[DT]>
The contrary would make the user's work more difficult.
<mfp>
then the problem is with the sharing of [b], not how [a] is closed
<mfp>
same problem when closing manually, on EOF or with Gc.finalise
<Yoric[DT]>
(i.e. if you open a network channel, on top of this uncompression, on top of this Unicode transcoding, you don't want to pass around all three channels just to be able to close them all)
<mfp>
-> that's what Gc.finalise would take care of
<Yoric[DT]>
Well, it's a problem of manual resource management.
<mfp>
(add manual close to avoid leaks when the refs aren't discarded)
<Yoric[DT]>
... and you're back to the start
rageway has joined #ocaml
<mfp>
yes, but Gc.finalise has made it easier for most cases
<mfp>
and didn't cause any additional problem
<Yoric[DT]>
That is, provided you designed your library to not close stuff manually.
<Yoric[DT]>
s/manually/as a consequence of something else closing/
<flux>
if you do close it manually, you must also unregister the finalisation
<mfp>
whether you close manually or not doesn't change anything regarding the "closed channel in use somewhere else" problem, does it?
<flux>
also if you rely on fd's being autoclosed, you can suddenly find yourself in a situation where you've run out of fd's
<rageway>
I would like to create an empty array corresponding to an empty list .. what should I left for the init var ? I mean the x right here let op = Array.make 0 x ?
<mfp>
other langs use close-on-GC (Ruby, Perl, Python, I think) plus HOFs that open a file and close it
<flux>
mfp, but they (atleast python) has deterministic GC
<mfp>
Ruby doesn't
<flux>
mfp, so once you lose the last reference, the file getes closed immediately
<mfp>
Ruby's is conservative
<flux>
well, how does ruby handle this? for (100000 files) string.concat(load_file(filename)) ?
<flux>
(hm, s/concat/append/)
<Yoric[DT]>
rageway: well, anything with the correct type.
<mfp>
also, OCaml's GC is precise, isn't it?
<flux>
precise perhaps, but you don't know when it gets run
<mfp>
flux: load_file is actually File.read(filename), which opens AND closes
<rageway>
Yoric[DT], there is no type. My function gets an empty list and creates the right array
<flux>
mfp, so it does closing explicitly instead of relying on GC?
<mfp>
both
<Yoric[DT]>
rageway: well, there is no such thing as an empty array of elements with no type.
<Yoric[DT]>
rageway: you always need a type.
<mfp>
<mfp> other langs use close-on-GC (Ruby, Perl, Python, I think) plus HOFs that open a file and close it
<mfp>
in Ruby, File.open(file){|f| ... } is a string -> (channel -> 'a) -> 'a HOF that closes the file
<Yoric[DT]>
Side-note: 10 lines (including one blank) for a gunzip de-compressor written with Batteries Included.
<mfp>
but there's also close-on-GC, which ppl are told not to rely on too much because the GC is conservative and closures capture way too much environment
<rageway>
Yoric[DT], my objective is to create an array concording to a list. if my function gets an empty list, then I don't know what to do
<flux>
actually I think the same problem is in ocaml gc too, given the documentation regarding Gc.finalise
<mfp>
flux: close-on-GC can serve as a safety net; clearly using HOFs is the way to go to ensure timely close()
<Yoric[DT]>
rageway: [||] is an empty array with "no-type-yet"
<Yoric[DT]>
Well, technically, regions are the way to go to ensure timely close().
<Yoric[DT]>
But we don't have regions (yet) for OCaml.
<Yoric[DT]>
(although I have the beginning of a prototype of a limited region system lying around somewhere)
<flux>
mfp, one way to look at it could be to assert false; when you find that a descriptor has leaked.. ;)
<flux>
yoric[dt], linear typing also could help with that?
<Yoric[DT]>
Also.
<mfp>
Yoric[DT]: is that reasonably doable without changing the compiler?
<Yoric[DT]>
Well, depends on your definition of "reasonably", I guess.
TypedLambda has joined #ocaml
<Yoric[DT]>
But probably not.
<_zack>
Yoric[DT]: np for the patch, but I don't see in the archive, did you push it?
<Yoric[DT]>
Gasp, I have a buffering problem.
<Yoric[DT]>
_zack: no, not yet.
<Yoric[DT]>
At the moment, I'm trying to get gzip-compression to work.
<_zack>
urgh
<Yoric[DT]>
The core of my example is [copy inp (GZip.compress out)]
<Yoric[DT]>
(where [copy] is [IO.copy])
<Yoric[DT]>
The problem is that this never flushes out.
<Yoric[DT]>
The problem is that this never flushes [GZip.compress out], that is.
<Yoric[DT]>
I'm wondering where the responsibility of this flush should lie.
<Yoric[DT]>
(I believe that this is an issue we're also going to meet with [tab_out] et al.)
<_zack>
I see
<Yoric[DT]>
Now, this is where a reference counter would be nice.
<Yoric[DT]>
That problem can be worked around, but it's annoying.
<Yoric[DT]>
That is, a call to [flush_all ()] solves everything.
<Yoric[DT]>
But that's not satisfactory.
<flux>
btw, it would be doubly annoying if the flush was called by cloe which would be called by gc, and you appended something to out after that line ;)
<Yoric[DT]>
:)
<flux>
copy ~flush:true?
<flux>
but I suppose it's not copy's responsibility, although it happens to be the sole owner of the output stream
<flux>
type 'a refcounted ;)
<flux>
(but that's PIA)
<Yoric[DT]>
PIA?
<flux>
pain in the..
<flux>
neck.
<Yoric[DT]>
Ah, ok.
<Yoric[DT]>
In addition, mixing ref-counting and OCaml's GC is probably going to lead nowhere.
<flux>
yeah, it would need to be somehow enforced
<Yoric[DT]>
Well, I suppose [copy] should perform the flush.
<Yoric[DT]>
But that's not the real solution.
<Yoric[DT]>
s/the/a/
<flux>
well, it's a solution..
<_zack>
Yoric[DT]: I'm getting kinda lost in the chat backlog (i.e., no time to re-read everything right now), can you at the end post a summary to batteries-devel?
<Yoric[DT]>
I will.
<Yoric[DT]>
I will try.
<flux>
with_flush (copy in) (GZip.compress out)
<_zack>
Yoric[DT]: tnx, np for the "try" part, we are all best effort servers here :)
<Yoric[DT]>
_zack: what do you mean?
<Yoric[DT]>
Ah, ok :)
<_zack>
:)
TypedLambda has left #ocaml []
<Yoric[DT]>
Unfortunately, by that account, every single function should have an option for flushing.
<Yoric[DT]>
Mmmhh...
<kig>
is copy lazy?
<Yoric[DT]>
kig: no, why?
<kig>
then it should flush after the last write
<Yoric[DT]>
Sure.
<Yoric[DT]>
But there are plenty of other functions which take as argument an [output] and write stuff on it.
<Yoric[DT]>
Surely not all of them should flush...
<flux>
[>needs_flushing] output
<flux>
:)
<flux>
I think education and documentation could be the first solution
<Yoric[DT]>
Well, a local solution would be to have
<flux>
if one was using monads, the argument would need to be named anyway (well..), and flushing would then be easy
<Yoric[DT]>
[with_zip_out]
<Yoric[DT]>
Sure.
<Yoric[DT]>
But we're trying to avoid monads here :)
<flux>
I was trying to say that it's not much worse than the alternative ;)
seafood_ has joined #ocaml
<flux>
perhaps there could be a separate type for "wrapped output"
<Yoric[DT]>
Sounds like overkill.
<flux>
yes, there is that
<kig>
is the problem that you're reading the file before the write is flushed?
TypedLambda|away has joined #ocaml
seafood has quit [Read error: 110 (Connection timed out)]
<Yoric[DT]>
kig: no, the problem is that we have two outputs: [out] and [GZip.compress out].
<Yoric[DT]>
The second one is anonymous and closed when it's garbage-collected.
TypedLambda|away has quit [Client Quit]
<Yoric[DT]>
(mmmhh... seems I have already implemented that, silly me)
<Yoric[DT]>
The first one is named and closed when we leave the scope, due to a higher-order function.
<Yoric[DT]>
Unfortunately, the first one should only be closed after the second one is flushed.
<Yoric[DT]>
Adding flushing to [copy] seems to do the trick.
Jedai has joined #ocaml
hkBst has joined #ocaml
|Jedai| has quit [Read error: 110 (Connection timed out)]
<kig>
possible replacements: with_file_out fn (copy (gzip i));; with_gzip_out fn (copy i);; with_file_out fn (fun o -> with_gzip_out o (copy i));;
<Yoric[DT]>
Well, at the moment, it's
<Yoric[DT]>
something like this, yes.
Spiwack has joined #ocaml
<kig>
from my understanding the gist of the problem is that the fd returned by gzip_out isn't closed, which causes trouble when you have lazy writes and a strict close of the fd used in the gzip. So it could be fixed by making the writes strict or enforcing that the gzip always closes
<kig>
enforcing that gzip closes = with_gzip, strict writes = let write o .. = write' o; flush o
filp has quit ["Bye"]
<kig>
in a sense, the default write is like let write o s = write' o s; lazy o and flush is let flush o = let o' = Lazy.force o in flush' o'; o'
<kig>
and the way that requires that write results are explicitly handled reminds me of clean :S
<Yoric[DT]>
At the moment, my solution is to make some writes strict.
<Yoric[DT]>
That's hardily satisfying, though.
<flux>
another case where it breaks down is simply copy (open_in "foo") (open_out "bar") ?
<flux>
the closing part that is
<flux>
~close_in:true ~close_out:true.. :(
<Yoric[DT]>
Yeah...
sporkmonger has joined #ocaml
<flux>
maybe we should just see that this is doomed to fail and switch to c++!
<Yoric[DT]>
:)
<Spiwack>
maybe some linear voodoo could be, like, cool with file reading...
<Yoric[DT]>
Actually, a partial solution to [open_in "foo"] is just what I said above: auto-close files once reading is complete.
<Yoric[DT]>
Yeah, but I don't see much linear voodoo being implemented for OCaml.
<flux>
what about when there's an exception and the file never gets read in full?
<Spiwack>
that's because it's not possible !
<flux>
spiwack, sure it's possible, there's the compiler source :)
<Yoric[DT]>
flux: I can do that.
<Spiwack>
but anyways, for the files themselves, what about closing them at garbage collection?
<Yoric[DT]>
Spiwack: dependency issues.
<flux>
yoric[dt], do what?
<Yoric[DT]>
You've missed the discussion :)
<Yoric[DT]>
flux: close the file when there's an exception.
<Yoric[DT]>
Mmmhh...
<flux>
yoric[dt], I mean if the exception is thrown by something else than the stream that is being read
<Spiwack>
(well, it's impossible in any reasonnable way to reuse some part of ocaml type system)
<Yoric[DT]>
kig: actually, we also have pipes in Batteries.
alexyk has joined #ocaml
<Yoric[DT]>
_zack: actually, I believe I've found a bug.
<_zack>
Yoric[DT]: tell me
<_zack>
regarding the flushing issue, can't [f o] keep a reference to [o], such that [o] is not garbage collected? (that won't solved the general problem of someone closing [o] by hand, but that can be documented)
<Yoric[DT]>
Well, that would be in contradiction with the usage of, say [with_file_out].
<Yoric[DT]>
Which does close things manually.
<Yoric[DT]>
For the possible bug: [GZip.close_out] seems to raise an exception.
<Kerris4>
thanks Yoric, I might be using JoCaml in a few months, or next summer at the latest
<Yoric[DT]>
np
<Yoric[DT]>
What are you going to do with it?
<Kerris4>
parallelising some existing OCaml stuff
<Kerris4>
compilers, verifiers
<Yoric[DT]>
Nice.
<Yoric[DT]>
Are you aware of Acute, BSML and Opis?
<Kerris4>
Well, attempting to, that is :)
<Yoric[DT]>
:)
<Yoric[DT]>
Oh, and coThreads and OCamlErl?
<Kerris4>
Nope, not heard of them at all. I was a Haskell/Erlang/Lisp tourist until very recently
<Kerris4>
thanks for the headsup, I'll definitely look into them before the big summer project
<Yoric[DT]>
Acute is another attempt at doing concurrent/distributed stuff with an extension of OCaml. I don't know its current status.
_JusSx_ has joined #ocaml
sporkmonger has joined #ocaml
<Yoric[DT]>
BSML and Opis are libraries for OCaml for doing distributed computing.
<Yoric[DT]>
coThreads is for concurrency
<Yoric[DT]>
OCamErl I'm not quite sure :)
<Yoric[DT]>
np
<Kerris4>
strange, coThreads is giving me a blank sourceforge page
<Yoric[DT]>
Sourceforge migrated recently and most webmaster never took the time to migrate their website.
thelema has quit [Read error: 110 (Connection timed out)]
<Yoric[DT]>
(that gives me an excuse for not having migrated mine)
<Kerris4>
hahaha#
alexyk has joined #ocaml
alexyk has quit [Client Quit]
vixey has joined #ocaml
Spiwack has quit [Remote closed the connection]
velco has quit [Remote closed the connection]
tomh has joined #ocaml
jlouis has joined #ocaml
<_zack>
Yoric[DT]: does it make sense to have gzip-ed channel 'a channels, where 'a is the same of the underlying channel?
<_zack>
wouldn't it be more appropriate to have gzip-ed channel unit-channels?
<Yoric[DT]>
I don't know.
<Yoric[DT]>
I never quite understood the 'a.
<_zack>
same here :)
<_zack>
but the problem now is
<_zack>
currently Gzip'd chans close the underlying chan upon close
<_zack>
I think that's worng
velco has joined #ocaml
<_zack>
but if they don't close it, there is no link between the 'a of gzip'd chans and the 'a of the underlying chan
<_zack>
hence the only solution is having them return unit, the only available 'a will be the underlying one
<_zack>
'a seems to be meaningful only for the "outer" channel, as there is no way to combine the result of "inner" channels with the (future) result of the "outer"
<Yoric[DT]>
Yeah, I think it's wrong.
<Yoric[DT]>
Mmmhh...
<_zack>
have a look at IO.output_string
<Yoric[DT]>
Let me think this over.
<_zack>
ok
* Yoric[DT]
will be back in a few minutes.
<flux>
I think I understood the 'a in output some time ago, but I don't remember it anymore ;)
<_zack>
flux: my intuition is that it is something computer over what passes through the channel
<flux>
maybe it was for extending streams, like provide a stream that counts the number of octets it passes through
<_zack>
precisely
<flux>
but perhaps it could also be used to provide an additional interface to the stream?
<flux>
dunno why it would need to be attached to the stream that way, though
<flux>
you could implement the counter too without it
<flux>
val counting_strema : output -> (output * int ref)
<flux>
but it wouldn't be functional..
<_zack>
but anyhow, the point is that if you want to support that for depending channels
<_zack>
you need to change the api of create_in
<_zack>
so that you can combine together values returned by inner chans
<_zack>
with values of outer chans
<_zack>
I don't think it's worth
<_zack>
but while it is not possible, depending channels can only be unit channels
<_zack>
... or I'm missing something :)
<flux>
well, iirc you can cast any channel into a unit channel
<_zack>
s/create_in/create_out of course
<_zack>
yes, so?
<flux>
but yes, it does seem a bit pointless
<flux>
perhaps someone can give a compelling example where 'a output rules :)
<flux>
I wonder if the extlib authors should be approached of the matter.. I mean, if batteries has essentially forked extlib, then perhaps this would be a chance to get rid of it, if it serves no better use
<_zack>
Yoric[DT]: actually even tab_out closes it's underlying channel when it gets closed, I don't think it should
tvn1981_0 has joined #ocaml
<Yoric[DT]>
_zack: fair enough.
<_zack>
I think that we have all closed the underlying chan because it made a nice 'a -> 'a type :-D
<Yoric[DT]>
:)
<_zack>
it remains IMO only to reason a bit about IO.lmargin
<_zack>
it smells of a functional way of wrapping channels
<_zack>
I'll leave that to volunteers, I'm leaving RSN
<Yoric[DT]>
Have you pushed your fixes?
<_zack>
in a bit
<Yoric[DT]>
ok
<_zack>
well, yes, the exception fix is there already
<_zack>
the unit channel will be in a bit
<Yoric[DT]>
ok
threeve has joined #ocaml
<_zack>
Yoric[DT]: done, tab_out still needs to be fixed though
<_zack>
yup, I'm not sure it is that important though
<mbacarella>
;)
<Camarade_Tux>
mbacarella, it should be doable as one of the cab compression algorithm is LZW which is probably already available and anyway easy to implement
* Camarade_Tux
is not sure about the other one but it could be deflate
<Yoric[DT]>
_zack: there's now a new function, [IO.wrap_out], which should handle the dependency thing.
* Yoric[DT]
is now testing.
tp76 has joined #ocaml
<Yoric[DT]>
mmmhhhhh...
<Yoric[DT]>
Should work better if I don't invert the direction of dependencies.
<_zack>
:)
<Yoric[DT]>
And work better it does.
<Camarade_Tux>
mmmhhhhh
<Camarade_Tux>
Should work better if MS did not produce five 15MB one-liner xml files...
<Yoric[DT]>
Gasp.
<Yoric[DT]>
The "nice" thing with XML, is that this probably requires 500+ Mb RAM to read each of these files.
<Yoric[DT]>
Mmmhhh....
<Yoric[DT]>
I have a slight problem with gzip/gunzip.
ofaurax has joined #ocaml
<Camarade_Tux>
actually vim "only" takes 51MB right now ;)
* _zack
hides
<Yoric[DT]>
Camarade_Tux: yeah, but that's not real XML reading :)
<Yoric[DT]>
_zack: no, it's actually my fault.
<Yoric[DT]>
The resulting gunzip-of-gzip-of-test-file is one byte larger than the original.
<Yoric[DT]>
There's a newline at the end.
* Yoric[DT]
assumes it's an issue with [copy].
jeddhaberstro has quit []
alexyk_ has joined #ocaml
<Yoric[DT]>
_zack: changes pushed.
<Camarade_Tux>
Yoric[DT], it will come soon, right now I need to understand the format ;)
alexyk has quit [Read error: 104 (Connection reset by peer)]
alexyk has joined #ocaml
smimou has quit ["bli"]
<Camarade_Tux>
vim found 350k lines to indent, it can process about 500 lines per second, I think I have time to do some housechores :p
<_zack>
Yoric[DT]: you didn't set your name in git, did you?