boogie has quit [Remote host closed the connection]
eikke has quit [Ping timeout: 272 seconds]
eikke has joined #ocaml
jwatzman|work has quit [Quit: jwatzman|work]
es917 has joined #ocaml
es917 has quit [Client Quit]
Thooms has quit [Quit: WeeChat 0.4.3]
zpe has joined #ocaml
zpe has quit [Ping timeout: 245 seconds]
hhugo has joined #ocaml
hhugo has quit [Client Quit]
hhugo has joined #ocaml
WraithM has quit [Ping timeout: 272 seconds]
oriba has quit [Quit: oriba]
WraithM has joined #ocaml
rgrinberg has quit [Quit: Leaving.]
rgrinberg has joined #ocaml
eikke has quit [Ping timeout: 246 seconds]
ontologiae has quit [Ping timeout: 250 seconds]
shinnya has quit [Ping timeout: 246 seconds]
cesar has joined #ocaml
cesar is now known as Guest31329
q66 has quit [Quit: Leaving]
Guest31329 has quit [Ping timeout: 245 seconds]
oscar_toro has quit [Ping timeout: 272 seconds]
hhugo has quit [Quit: Leaving.]
bytbox has quit [Remote host closed the connection]
rgrinberg has quit [Ping timeout: 272 seconds]
rgrinberg has joined #ocaml
rgrinberg has quit [Ping timeout: 245 seconds]
rgrinberg has joined #ocaml
waneck has joined #ocaml
simulacrum has quit [Quit: No Ping reply in 180 seconds.]
simulacrum has joined #ocaml
rgrinberg has quit [Ping timeout: 250 seconds]
rgrinberg has joined #ocaml
jave_ has joined #ocaml
rs0 has joined #ocaml
ddosia1 has joined #ocaml
Snark_ has joined #ocaml
Antoine60 has joined #ocaml
Ptival_ has joined #ocaml
dwoos_ has joined #ocaml
stux|away has joined #ocaml
kanzaros_ has joined #ocaml
manizzle has quit [*.net *.split]
cespinoza has quit [*.net *.split]
taion809 has quit [*.net *.split]
ddosia has quit [*.net *.split]
tobiasBora has quit [*.net *.split]
jave has quit [*.net *.split]
Antoine59 has quit [*.net *.split]
kanzaros has quit [*.net *.split]
Snark has quit [*.net *.split]
stux has quit [*.net *.split]
rs0_ has quit [*.net *.split]
Ptival has quit [*.net *.split]
dwoos has quit [*.net *.split]
kanzaros_ is now known as kanzaros
dwoos_ is now known as dwoos
cespinoza has joined #ocaml
manizzle has joined #ocaml
taion809 has joined #ocaml
tobiasBora has joined #ocaml
cesar has joined #ocaml
cesar is now known as Guest57020
bytbox has joined #ocaml
Guest57020 has quit [Ping timeout: 258 seconds]
taion809 has quit [Remote host closed the connection]
<whitequark>
Drup: so for js_of_ocaml, you could do this
philtom has joined #ocaml
<whitequark>
you could fork the frontend and add your ## and pass it via pp= in META
ygrek has joined #ocaml
<Pepe_>
with the ocaml interpreter I can see a module type with module Foo = The_module, utop just displays "module T = Core.Std.Time" :(
<Pepe_>
or is it Core?
<Pepe_>
nah
philtom has quit [Ping timeout: 245 seconds]
<whitequark>
it's module aliases from 4.02
<whitequark>
try #show_module
<Pepe_>
thanks
axiles has joined #ocaml
troydm has quit [Ping timeout: 250 seconds]
troydm has joined #ocaml
rgrinberg has quit [Quit: Leaving.]
troydm has quit [Ping timeout: 250 seconds]
troydm has joined #ocaml
troydm has quit [Ping timeout: 250 seconds]
ygrek has quit [Ping timeout: 240 seconds]
rgrinberg has joined #ocaml
troydm has joined #ocaml
troydm has quit [Ping timeout: 250 seconds]
morphles has joined #ocaml
rs0 has quit [Remote host closed the connection]
MercurialAlchemi has joined #ocaml
tac_ has quit [Quit: Leaving]
slash^ has joined #ocaml
Kakadu has joined #ocaml
morphles has quit [Ping timeout: 240 seconds]
tane has joined #ocaml
eugene_klm has joined #ocaml
martintrojer_ has quit [Ping timeout: 272 seconds]
martintrojer has joined #ocaml
oscar_toro has joined #ocaml
Kakadu has quit [Quit: Konversation terminated!]
Submarine has joined #ocaml
Simn has joined #ocaml
shinnya has joined #ocaml
tani has joined #ocaml
tane has quit [Ping timeout: 258 seconds]
cesar has joined #ocaml
cesar is now known as Guest1925
Guest1925 has quit [Ping timeout: 272 seconds]
ontologiae has joined #ocaml
Kakadu has joined #ocaml
hhugo has joined #ocaml
hhugo has quit [Client Quit]
bezirg has joined #ocaml
rgrinberg has quit [Quit: Leaving.]
morphles has joined #ocaml
eikke has joined #ocaml
WraithM has quit [Ping timeout: 250 seconds]
Anarchos has joined #ocaml
eikke has quit [Ping timeout: 272 seconds]
morphles has quit [Ping timeout: 272 seconds]
bezirg has quit [Quit: Leaving.]
bezirg has joined #ocaml
fraggle_ has quit [Remote host closed the connection]
fraggle_ has joined #ocaml
pminten has joined #ocaml
nicoo has quit [Ping timeout: 264 seconds]
eikke has joined #ocaml
nicoo has joined #ocaml
Khady has quit [Quit: No Ping reply in 180 seconds.]
Khady has joined #ocaml
morphles has joined #ocaml
troydm has joined #ocaml
lordkryss has joined #ocaml
cesar has joined #ocaml
cesar is now known as Guest80901
eikke has quit [Ping timeout: 260 seconds]
tani has quit [Quit: Verlassend]
Guest80901 has quit [Ping timeout: 260 seconds]
ontologiae has quit [Ping timeout: 250 seconds]
araujo has quit [Ping timeout: 240 seconds]
araujo has joined #ocaml
ygrek has joined #ocaml
ggole has joined #ocaml
q66 has joined #ocaml
<Drup>
whitequark: If I want to fork the compiler, I would like to do it for a good reason, like "adding eliom's typing stuff"
<companion_cube>
or "add implicits"
ygrek has quit [Ping timeout: 260 seconds]
<whitequark>
well, not the compiler, just the frontend
<whitequark>
merlin has a fork of frontend
<whitequark>
camlp4 has one too
larhat has quit [Quit: Leaving.]
<Drup>
whitequark: yeah, and considering the mess merlin is on the side, to support various version of the compiler .... >_>
Anarchos has quit [Quit: Vision[0.9.7-H-20140108]: i've been blurred!]
iorivur_ has joined #ocaml
iorivur has quit [Ping timeout: 260 seconds]
Snark_ is now known as Snark
jbj has joined #ocaml
araujo has quit [Quit: Leaving]
<jbj>
GADTs can reduce the number of "match" cases I need to write. But is there syntax for writing a match with 0 cases?
<Drup>
what would that even mean ?
<jbj>
It would mean that we're in an impossible case
<jbj>
For instance, if I obtain a value of type "bool t", and I can see in the GADT declaration of t that it can only produce "int t" and "string t"
<Drup>
I see
<Drup>
the answer is no
<jbj>
That's a shame. I'm trying to encode type non-equality, defining "type ('a, 'b) not_equal = ('a, 'b) eq -> an_empty_type". But I end up needing empty pattern matches.
tane has joined #ocaml
c74d has quit [Remote host closed the connection]
morphles has quit [Ping timeout: 272 seconds]
c74d has joined #ocaml
parcs has quit [Remote host closed the connection]
iorivur_ has quit [Ping timeout: 272 seconds]
<asmanur_>
jbj: you might want to define differently your empty type
<asmanur_>
eg. type bot = { bot : 'a . 'a } let bot_elim { bot } = bot
<jbj>
Thanks. That takes care of one empty pattern match.
<jbj>
Providing a witness for my not_equal type still requires destructing an impossible "eq" constructor, which I'd also encode as an empty pattern match
<jbj>
Example: let x : (int, bool) not_equal = (function)
<jbj>
There are no cases because the Refl constructor in the eq type has no (int, bool) case -- it has only ('a, 'a)
<gasche>
jbj: the OCaml exhaustivity checker does not attempt to be complete and arguably makes a bad job of it
<gasche>
it will work ok for "natural" GADTs you get while programmer, but degrades relatively badly if you attempt to do explicit proofs
<jbj>
gasche: thanks -- good to see it's being tracked
<gasche>
on the particular matter of (int, bool) eq
<gasche>
it's actually only a syntactic problem: the "official" syntax doesn't support empty pattern-matching
<gasche>
but the type-checker works as expect in that situation
<gasche>
(I'll create a bugreport about the lack of such syntax, I thought we discussed it already but cannot find it again)
<jbj>
Yes, I tried adding an additional Dummy constructor to the eq type, and the type checker is able to see that Refl is not necessary for exhaustiveness.
<whitequark>
isn't empty pattern matching basically equivalent to `assert false`?
<jbj>
If you use "assert false" in the wrong place, your program might explode at run-time. The idea with empty pattern matches is to get a static guarantee that this case is impossible.
<whitequark>
ah, right, exhaustiveness checks.
cesar has joined #ocaml
cesar is now known as Guest44467
<gasche>
the bad thing with using the revised syntax for this purpose is that it makes you nostalgic of currified type constructors
Guest44467 has quit [Ping timeout: 272 seconds]
<jbj>
yeah, but not of all those semicolons and brackets
<gasche>
as it turns out, symmetric delimiters on pattern-matching is a rather good idea
<gasche>
of course "match end" and "function end" as in Coq have the best of both worlds
<Drup>
it will also solve the cross-package link issue
larhat has joined #ocaml
<tane>
quick test yielded a compilation failure
<tane>
well, i'll have another look later
<Leonidas>
Yeah, I was thinking of the same thing, a central documentation would be super-handy.
<mfp>
ugh I have code that behaves differently depending on the compilation unit I place it in (AFAICS with no relevant differences in the environment) :-|
<Leonidas>
yes, couldn't get opam doc to compile either
<Leonidas>
tane: well, it only works with 4.01, so I'd have to downgrade first.
<mfp>
try_lwt M.foo () with exn -> do stuff (where M.foo also does try_lwt .... with exn -> log exn; raise_lwt exn) -> do stuff not executed if that code is put outside M's compilation unit
<Drup>
mfp: if you are inside M, how can you address ?
<mfp>
any Lwt expert around? looks like the waiter for the try_lwt wrapper is being GCed
<Drup>
address M*
<mfp>
Drup: inside M, the code is just try_lwt foo () with exn -> do stuff
<Drup>
right.
<mfp>
Drup: i.e., I actually wrap foo () with try_lwt .... with exn -> log & raise_lwt exn
<mfp>
and I see that wrapper is executed
<Drup>
I'm not really an lwt expert, so I can't help, sorry :(
<Leonidas>
maybe the OPAM package is outdated, but I remember it failed even when using 4.01
<Leonidas>
not sure why anymore
<mfp>
but the try_lwt wrapper in another module isn't
darkf has quit [Quit: Leaving]
<mfp>
Drup: actually, after reading lwt.ml, I'm starting to point my gaze at the compiler...
<Drup>
Leonidas: instead of apierror, I think you should call it "Result"
<Leonidas>
Drup: indeed.
<Drup>
I mean, there is a tag `Success in this variant ...
<Drup>
Leonidas: also, you may want to make it compatible with the usual `a result
<mfp>
I've already checked the waiter is not GCed using a finalizer for it; I've tried attaching Lwt.on_any to the thread to no avail (only works if I do it inside the same compilation unit)
Simn has quit [Read error: Connection reset by peer]
<mfp>
also defined foo with let rec foo () = .... to prevent inlining within the same compilation unit
<Leonidas>
Drup: `a result?
<mfp>
(and the wrapper still works inside the same unit, but not in another)
<Drup>
Leonidas: something like `Result of 'a | `Error of error
<Drup>
not necessarily a good idea, I don't know.
<Leonidas>
Drup: and then error is the apierror type?
<Drup>
Leonidas: but in your case, it's not clear from the type when you should except a result and when it's side-effect only
<Leonidas>
Yes, I kinda dislike that success is kinda crammed into the apierror/result type
<Leonidas>
there's always a result, but sometimes not a very interesting one :-)
<Drup>
Leonidas: also, you don't de-serialize into a friendly format :(
<Leonidas>
`Success of empty JSON or something.
<Leonidas>
Drup: thats planned.
<Leonidas>
but first I'd like to get rid of the strings everywhere
<Drup>
yeah, I think the first step would be to give meaningful return types
<Drup>
currently, you sort of need to know the types of the rest api ... which is a bit sad
<Leonidas>
I just wanted to put out a "usable" binding as 1.0, then iterate on that.
<Drup>
calling it 1.0 and knowing you're going to change everything is a bit ... :p
<Leonidas>
hey, I'm using semver :p
<Leonidas>
that's an experiment
<Leonidas>
well, I won't change it, I'll just put more layers on top
<Drup>
well, no, you should *change* the return type of everything
<Leonidas>
you mean, parse the contents of the JSON?
<Drup>
yes
<Leonidas>
yep, that can be done on top of it too
<Drup>
sure, but that means adding lot's of useless stuff in your api, while you could keep it nice and simple
<Leonidas>
oh, already got my first feedback from the mailinglist
<Drup>
ahah, exactly what I'm saying :D
<Leonidas>
Drup: not sure I follow, since I need to get that JSON anyway and need to transform it into some struct or so, so there is nothing additional, useless.
<Leonidas>
yep, I'll look into it
<Drup>
Leonidas: I mean that you should change the api of the current functions, not add new ones :)
<Leonidas>
He also mentions a thing I wasn't too happy about, namely returning that HUGE apierror type
<Leonidas>
Drup: well, dunno. I don't think it matters that much.
<Drup>
My opinion is that, the smaller the api, the better for everyone
<Drup>
(at equal features)
<Leonidas>
I thought it might be nice for people to be able to choose how high or low-level they want to go
<whitequark>
yeah I don't have a problem with the huge apierror type
<Drup>
Leonidas: does the uninterpreted json type have a point ?
<whitequark>
with how people would typically handle the result (match on 1 or maybe 2 different variants and have a catchall), it's not an issue
<Drup>
is there multiple way to interpret it ?
<ebzzry>
for projects that use Core, how should a typical Makefile look like?
<Drup>
ebzzry: generated by oasis.
<Leonidas>
Drup: no, but they might add stuff that I am not yet handling, so people can just get the raw json and get the key they want by themselves
<whitequark>
Leonidas: I have an issues with other things though
<whitequark>
what the fuck are those "string"s for.
<Drup>
Leonidas: if your library is simple enough (and it is) a patch is quicker ...
<ebzzry>
Drup: ok. let me read about oasis.
<whitequark>
and yeah, the uninterpreted json result is pretty bad.
<Leonidas>
whitequark: yes, I'll be adding more types.
<Leonidas>
but first I'd like to get the error handling right
<Leonidas>
as such I am quite happy about the discussion.
<Drup>
well, my issue is not with the error handling, it's with the success handling :D
<Leonidas>
What I am annoyed about is the fact that every function returns this huge result type of which most cases simply can't ever happen in that method.
<whitequark>
it looks dirty, but as I described above, I don't think it really matters much
<Drup>
whitequark: It's annoying from a documentation point of view
<whitequark>
and you have more pressing issues :p
<Drup>
the type is not self-documenting about the errors
<Leonidas>
no worries, adding a token and timestamp type will be a breeze
<Leonidas>
there is no explicit timestamp type in ocaml, is there? Just float?
<def`>
ebzzry: ok :), with this .merlin, it is expecting to find bar.cmi /home/rmm/home/shop/src/test/_build
<def`>
bar.cmi in …*
<ebzzry>
ok
ontologiae has quit [Ping timeout: 260 seconds]
<def`>
the workflow is to build with corebuild/ocamlbuild, and have merlin reuse the incremental files to provide help while developing.
<ebzzry>
def`: let me do that.
shinnya has quit [Ping timeout: 250 seconds]
Kakadu has quit [Quit: Page closed]
slash^1 has quit [Read error: Connection reset by peer]
<ebzzry>
def`: that, PLUS, B _build/src
<ebzzry>
def`: thanks
octachron has quit [Quit: Leaving]
hhugo has quit [Quit: Leaving.]
hhugo has joined #ocaml
ygrek has quit [Ping timeout: 272 seconds]
<def`>
ebzzry: not yet documented (!!), but _build/** should do it too :)
<Drup>
\O/
teiresias has joined #ocaml
ygrek has joined #ocaml
rgrinberg has joined #ocaml
hhugo has quit [Quit: Leaving.]
Kakadu has joined #ocaml
<ebzzry>
def`: really? that's cool!
<ebzzry>
Should 'B _build/**' by itself suffice, or should I have both 'B _build B _build/**'?
hhugo has joined #ocaml
rgrinberg has quit [Quit: Leaving.]
<def`>
B _build/** should suffice (** is interpreted as "." or any subdirectory)
<ggole>
def`: speaking of merlin documentation, somebody was wondering what PRJ did
ygrek has quit [Ping timeout: 246 seconds]
<def`>
ggole: :D, it was one of the experiment around managing multiple projects.
<ggole>
Ah, I see
<ggole>
"was"?
rgrinberg has joined #ocaml
<def`>
the idea is that PRJ is followed by a string which is the project name. all files with the same PRJ "key" are managed in the same 'namespace' by merlin
<def`>
though, using the path of the .merlin as a key seems to be enough
<def`>
"was": the semantic is a bit unclear to me, with merlin 2.0 we managed to provide better isolation of the compiler state, so two files loaded at the same time with different configs shouldn't interact
<ggole>
I see.
<def`>
so if I find a clear semantic, use-case and implementation, I'll document PRJ :)
<ggole>
I peeked at the code and didn't see anything except -project-find
<ggole>
I was wondering if it was a bit of an atavism.
<def`>
the main use is in project_by_key in src/frontend/command.ml
<def`>
is by*, not in
<def`>
but nowadays, only the .merlin path is used
badkins has joined #ocaml
ontologiae has joined #ocaml
iorivur has quit [Quit: No Ping reply in 180 seconds.]
iorivur has joined #ocaml
hhugo has quit [Quit: Leaving.]
jc1 has joined #ocaml
Thooms has quit [Quit: WeeChat 0.4.3]
jc1 has left #ocaml [#ocaml]
hhugo has joined #ocaml
<ebzzry>
def`: ok. thanks, a lot.
<def`>
ebzzry: you're welcome
simulacrum has quit [Remote host closed the connection]
jc1 has joined #ocaml
hhugo has quit [Quit: Leaving.]
jc1 has left #ocaml [#ocaml]
hhugo has joined #ocaml
zpe has joined #ocaml
<Drup>
I'm looking at ocaml.org's static html generator, I didn't know it was *that* hacky :|
<adrien>
what? you still haven't rewritten it with eliom?
<Drup>
eliom is not suited for static html generation
<Drup>
(we might change that at some point, but it's not for now :D)
<adrien>
static is boring, you'd have added live collaboative toplevel suppot
<adrien>
collaborative*
<adrien>
I need to stop eating bread on top of my keyboard, stucks keys
hhugo has quit [Quit: Leaving.]
<Drup>
That would actually not be that difficult. The issue is security
<Drup>
"oh wait, I can freeze everyone's browser by entering something silly !"
<Drup>
so, hem, yeah, the colaborative is sort of figured out already.
<adrien>
Drup: yeah, had that in mind
<adrien>
you shouldn't have to "redo" webrtc
<adrien>
that's the point of webrtc
philtom has joined #ocaml
manizzle has quit [Ping timeout: 245 seconds]
<Drup>
adrien : I mean, to redo a video conference thing
<adrien>
you need something to do the initial connection but that's it
<adrien>
but that way people can help others live with video and screams
<Drup>
:)
<adrien>
caps to shout are so 1990
<adrien>
now you can actually shout
<Drup>
adrien, do you noticed it's not friday anymore ? :p
<Drup>
did*
manizzle has joined #ocaml
Anarchos has joined #ocaml
<elfring>
Can a customised OCaml module be directly tested in a command-line interface like "utop"?
<Drup>
sure, just #use it
<nicoo>
elfring: What do you mean by a « customized module » ? But yes, you can load your own modules in the toplevel.
<nicoo>
Drup: #use_mod
<elfring>
I updated one of my OCaml source files. Now I would like to see if they still work as expected without installing the affected software into a special test directory.
philtom has quit [Ping timeout: 260 seconds]
<Drup>
elfring: if you modified some source, you need to recompile everything that depends on it
<elfring>
I updated a "small" source file which should also be reusable by itself.
shinnya has quit [Ping timeout: 272 seconds]
<nicoo>
Then you can #use it
<nicoo>
However, everything will be dumped in your environment
hhugo has joined #ocaml
waneck has quit [Read error: Connection reset by peer]
SethTIsue has quit [Quit: SethTisue]
morphles has joined #ocaml
waneck has joined #ocaml
<elfring>
The interface "utop 1.15" gave the feedback "Unknown directive `use_mod'.". Where should I read more about this directive?
hhugo has quit [Quit: Leaving.]
<Drup>
use a more recent version of ocaml
jbj has joined #ocaml
<elfring>
To which OCaml release was the support for the directive "use_mod" added? What did it try to improve?
<Drup>
4.02, iirc
Hannibal_Smith has quit [Quit: Sto andando via]
jbj has quit [Ping timeout: 245 seconds]
<gasche>
elfring: #mod_use is 4.01
<gasche>
it improves on #use if you want to use it as you would use #load, without compiling first
<gasche>
Drup: stog is a bit over-engineered for my needs, but I like it
<gasche>
(it's not that it is over-engineered, but that I use it in a way that makes some of the engineering redundant)
<Drup>
gasche: why didn't you choose something more mainstream, like jekyll ?
<Drup>
you don't seem to use any special features
<gasche>
stog is implemented in OCaml, so I thought it would be nice for Gallium's blog
<Drup>
(also, I don't mind the sources of your blog, if you can provide)
<Drup>
okay, it's only the NIH factor :p
<gasche>
I think NIH is a rather subtle concept
<gasche>
if you overdo NIH-fear at the level of a community, you end up developing other people's ecosystem at the expense of yours
oriba has joined #ocaml
<Drup>
don't be overly uptight, gasche, it was only little trolling =)
<gasche>
an interesting example is lambda-term, which made utop possible, and was originally written because Jérémie basically wanted to re-implement Emacs in OCaml
<adrien>
...
<adrien>
(must not be easy to fail indentation as hard as emacs though)
<gasche>
(also there is a difference between tools that are "mainstream" and solid, well-designed, such as Pandoc, and tools that are "mainstream" and crap, like PHP -- and I don't know Jekyll but "ruby" + "popular" doesn't always mean the first category)
bytbox has quit [Remote host closed the connection]
<Drup>
jekyll is okay
<Drup>
it just means customisation is in Ruby, which is why I want something else, since I need some customisation
<oriba>
Is there another csv-library, other than the one that can be found on OCaml forge?
<Drup>
what's wrong with the one in the forge ?
<oriba>
it eats up too much memory with large csv files
<oriba>
I have 150 MB and when the program used > 1.5 GB memory I stopped it
<Drup>
hum, indeed
<gasche>
are you sure that's only the library and not your processing code?
demonimin_ has joined #ocaml
demonimin has quit [Ping timeout: 260 seconds]
<oriba>
the processing code is not just more than reading the data, printing it and do a Gc.compact() (because it was planned to read more than one file)
<oriba>
there are about 3.7e6 lines, and csv-lib uses list-of-list...
<gasche>
do you use the record-by-record interface, or load the whole file in memory at once?
<oriba>
I use Csv.load
<adrien>
oriba: iirc you have several APIs
<gasche>
if you could use Csv.next instead, you would consume much less memory
<gasche>
list-of-list don't imply a 10x space overhead
axiles has quit [Remote host closed the connection]
<oriba>
hmhh there is a Csv.next maybe that makes sense... not sure
<gasche>
or Csv.fold_left
<oriba>
no experiments ;-)
<adrien>
it's possible that some APIs are not tail-recursive or do lots of operations with lists, meaning there could be a lot of duplicated information
<oriba>
ah, I also have a Csv.to_array .... I can try commenting it out...
<adrien>
sounds like you might need a data structure that isn't a list nor an array :)
<mrvn>
a tree or a hashtable?
<mrvn>
or a graüh?
<mrvn>
or a graph?
<oriba>
I just tried Csv.load and removed all other stuff that came later. The program fin9ishes earlier.... but the mem consumption is not much better
morphles has quit [Ping timeout: 260 seconds]
<oriba>
I think array of list or array of array would be ok
<oriba>
but list of list not
<oriba>
I will try with Csv.next
<nicoo>
gasche: *That* would be an awesome feat
everyonemines has joined #ocaml
<nicoo>
Emacs in a sane language, with decent concurrency
<everyonemines>
i've managed to avoid OO for so long
<everyonemines>
and this past week I was forced to use C#
<adrien>
OO isn't bad
<Drup>
well, most implementation of OO are terrible.
<adrien>
its abuse is
<adrien>
for instance, the configure script for Qt
<Drup>
adrien: I discern a pattern in your current rants
<Drup>
would you be currently hacking into Qt ? :3
<adrien>
well, they need 10 lines to split a string in two (delimiter is '='), re-assemble it (with a '=' in the middle) and write it to a file
<Drup>
everyonemines: and so, how was it ?
<adrien>
10 lines of shell script
<adrien>
and their implementation has the appearance of being a full API which covers many cases
milanst has joined #ocaml
<everyonemines>
OO in large projects gets absurd
<adrien>
yet it actually only covers one cases (merging two strings with an '=' in the middle)
<everyonemines>
people inherit from a class without knowing what's actually in it
<everyonemines>
they just trust/assume that it's more or less what they want
<adrien>
Drup: I believe the Qt project now owes me 10k€ for all the time I've lost (plus the SAN points)
<everyonemines>
i have a better understanding now
<everyonemines>
of why so much software is so buggy
<everyonemines>
i want to actually understand what I'm doing with some code
<everyonemines>
so it drives me crazy
<everyonemines>
sure, when you call a function you don't read all the sub-functions
<everyonemines>
but generally the function name and comments are way more descriptive and precise than with objects
<gasche>
one hard technical problem with implementation inheritance (open recursion) is that you cannot reason about the correctness of your code just looking at the state/invariants at the enter and exit point of a method
<gasche>
any self calls may get re-intepreted by a future inheritance, and thus has to be specified with well-defined pre/post conditions
rand000 has joined #ocaml
<elfring>
everyonemines: Do you occasionally prefer to think in terms of things (or objects) instead of a single callable function?
Kakadu has quit [Quit: Konversation terminated!]
<NoNNaN>
everyonemines: if you combine it with some code generator tool, then maintaining the code becomes "interesting"
<everyonemines>
i think of a "thing" as a bundle of data
<everyonemines>
that functions can be applied to
<everyonemines>
NoNNaN: :-(
<elfring>
everyonemines: How do you call the interface which groups related functions together for your data bundle?
<everyonemines>
in ocaml? i use modules
<everyonemines>
and if modules aren't needed I just use a name prefix for functions
<oriba>
Time profan, but how can memory profiling be done?
<elfring>
everyonemines: How do you think about to raise the abstraction level a bit more in the object-oriented software design direction?
<oriba>
ooops
<oriba>
swapping system
<everyonemines>
elfring: I'm not sure I understand what you mean, but...
<everyonemines>
OCaml with ad-hoc polymorphism could do anything OO can do, without objects
<mrvn>
OO is a state of mind
<everyonemines>
I guess functional programming forces large groups of programmers to be more disciplined
<everyonemines>
it can make code easier to understand, but it also requires people to understand the code better to get something to work
<mrvn>
don't think so.
<mrvn>
strict static type checking does though
<everyonemines>
java has static typing too
manizzle has quit [Ping timeout: 245 seconds]
<Drup>
ahah
<Drup>
no it doesnt.
<Drup>
it's not static, and it's barely typing
<everyonemines>
yeah yeah yeah
manizzle has joined #ocaml
<Drup>
and more important : types are not as self documenting in Java as they can be in a real statically typed language
elfring has quit [Quit: Konversation terminated!]
<oriba>
with array of lists (using Csv.next) its about 1.4 GB; with just reading lines and store them in list, it's about 253 MB for the < 150 MB file... so, it looks like csv-lib is just eating up the mem. Possibly even the reading function of that lib does it...
<oriba>
Csv.next did not help much ...
<mrvn>
array of list of chars?
<NoNNaN>
what is the recommended way to map some mutually dependent OO class (A->B, B->C, C->A, C->B) to module & functions? how to map an object method that pass `this` to an another object method and the other object have some method that calls back and also pass `this` ?
<mrvn>
you can use recursive modules or you can lift the recursion
<Drup>
there are two things, dependency and open recursion
<oriba>
mrvn, you mean me?
<mrvn>
oriba: yes
<Drup>
open recursion is "easy" : use a record of functions and add an extra argument "this" to each functions
<Drup>
it's, for example, how the mapper for ocaml's parsetree is done
<mrvn>
Drup: or a first class module
<oriba>
mrvn, it would need a rewrite of the csv parser I think
<mrvn>
oriba: that wasn't a suiggestion, that was a question
<Drup>
mrvn: well, sure, it's just much more verbose for the same thing.
<Drup>
(packing, unpacking, packing ...)
<oriba>
mrvn, question without "?"
<mrvn>
23:26 < mrvn> array of list of chars?
<oriba>
aha
<oriba>
array of list of strings
<oriba>
and Csv offers list of list of strings
<gasche>
do you really need to store the whole document in memory at once?
<mrvn>
array should save half the space as compared to a list
<Drup>
NoNNaN: for the dependency issue, if it's inheritence, just include, if it's dependency + modularity, functors
<oriba>
but the reading with Csv.next uses a certain function for reading the data..., it does not read directly from OCaml in-channel
<oriba>
it has it's own wrapper around it
<NoNNaN>
Drup: where can I found the mapper for ocaml's parsetree, which file in ocaml?
<Drup>
ast_mapper
<Drup>
in the parsing directory
<gasche>
if you could call Csv.next on a record, process it, then discard it, you would use less memory -- streaming
<oriba>
gasche, looking up the file again and again needs accessing the disk .... slow... it's just about 150 MB
<gasche>
it's hard to give good advice without any idea of the kind of processing you're doing, and the kind of data you're processing
<oriba>
gasche, I need the data in mem.
<gasche>
(for example, simply sharing content of fields may work well if there is a lot of repetition)
<oriba>
it's railway data and bus data.... I want to read the data and then ask the program for how to drive
<oriba>
of course I could chang e the approach and read in all the data each time I look for how to drive
<oriba>
But if I want to be flexible and change drivin ways, it would be better to hold anything in mem
everyonemines has quit [Quit: everyonemines]
<oriba>
All the datasets together have about 159 MB
<oriba>
the big one has 147 MB
<gasche>
(note that the Csv library you're using lets you parse records stored in Memory as a string)
<oriba>
the problem is the many lines I huess
<oriba>
and what does that mean / what doy ou want to tell me?
<oriba>
gasche and what does that mean / what do you want to tell me?
jc2 has joined #ocaml
<gasche>
well
<gasche>
you could read the file into a string, and then use Csv.string followed by Csv.fold_left each time you want to iterate
<gasche>
so you have (1) in-memory storage (2) no need to keep the CSV structure
<oriba>
hmhhh...
<gasche>
but I don't believe that is the right solution, because I would guess it is possible to store all your data in a way that is both compact and convenient to access
<gasche>
(re-parsing the CSV each time doesn't count as convenient to access)
<oriba>
the csv data structure is list-of-lists
<gasche>
again, that is relatively compact
<gasche>
there is not much wrong with list of lists
<oriba>
"type t = string list list"
<gasche>
so what?
<oriba>
why does it eat 1.5 GB mem for 146 MB data?
<mrvn>
because something makes 10 copies
<oriba>
"something" is not me ;-)
<NoNNaN>
you can mmap it to memory, and build an index structure with offsets
<gasche>
do you have your code and some test data available somewhere?
<NoNNaN>
but that's is lot more complicated
<oriba>
NoNNaN, who creates the index? Csv-lib?
<NoNNaN>
oriba: you have to create your own index
<gasche>
again, I don't really believe in a 10x overhead due to the representation
<oriba>
NoNNaN, but then I need to rewrite the csv-parsing also
<NoNNaN>
than divide and conquer
<oriba>
maybe switching to C :P
<NoNNaN>
determine the file size, divide it by 10, read a line from that offset (than you have a position from 0 - offset)
<Drup>
if I may : csv.ml is 800, maybe we can take a pick into it, before burning everything to the ground ?
<Drup>
800 lines*
<adrien>
:D
<oriba>
gasche, the csv-lib-code looks quite strange... so crazy macthes there...
<nicoo>
oriba: I'm actually tempted to write you something to parse CSV files based on ocaml-re + some simple abstraction to iterate over lines (i.e. not dumping the whole file in memory)
<oriba>
nicoo, so you want to write anoth ercsv-lib?
<oriba>
I just tried to use a lib. Writing one would be, what is necessary if the available libraries don#t work.... which I'm just exploring.
<oriba>
So if you want to write it, nice :-)
<Drup>
(23:42:35) Drup: if I may : csv.ml is 800, maybe we can take a pick into it, before burning everything to the ground ?
<oriba>
Drup, you want to locate the mem-"bottleneck"?
<Drup>
no, because I'm doing something else that requires my brain cells at the moment
<nicoo>
oriba: I could, but not this evening (combining alcool and tiredness doesn't make for great code)
<Drup>
oriba: there is a switch for memory profiling in opam
<adrien>
oriba: do you use load_rows?
<oriba>
Drup, but what did you mean with "if I may : csv.ml is 800, maybe we can take a pick into it" ?
<oriba>
adrien, I used Csv.next
<Drup>
oriba: I mean that maybe you can patch the library before spinning a new one blindly.
<adrien>
don't have that in my version of csv (ssh'ed to an old computer)
<oriba>
adrien, docs say, load_rows is deprecated
<adrien>
as long as it's equivalent
<oriba>
Drup, aha
<gasche>
oriba: I just did an experiment
<NoNNaN>
is there any ocaml modular staging in ocaml itself? eg.: to have some kind of controlled abstraction
<gasche>
got a largeish CSV file
<adrien>
and is your code readable anywhere?
<Drup>
NoNNaN: what does that mean ?
<nicoo>
NoNNaN: Not sure what you mean by “modular staging”
<gasche>
(73Mio)
<gasche>
and iterated on it using Csv.iter
<gasche>
the peak memory usage of this was 4Mio
<gasche>
(not doing anything with each line's content)
<oriba>
iter means: data is not in mem I guess
<Drup>
oriba: you'll have to show your code, I think :D
<adrien>
NoNNaN: views?
* adrien
-> bed
<gasche>
this means that the additional overhead of parsing, comparing to storing the actual data you want, is less than 4M
<oriba>
Drup, just kicking out any stuf fthat is not used (a lot of unused code), then I can put it online somewhere
<gasche>
(which would be, for example, a reasonable size for a buffer)
<oriba>
gasche, try Csv.load
<oriba>
should be renamed to Csv.bloat ;-)
<adrien>
it's going to be faster than the other one though
<adrien>
well, difficult to predict
<adrien>
but it might well give you a better I/O access pattern
<adrien>
the alternative being that interleaving CPU and I/O might allow prefetching of the data
<adrien>
in other words: can't tell in advance
<adrien>
had this program with O(n*log(n)) behaviour beat a rewrite with O(n) probably because of I/O even though the constant factors were close and n was several millions
milanst has quit [Quit: Page closed]
<oriba>
gasche, how many line has that csv-file? I huess the problem is num of lines, not size of the file
<adrien>
I'd bet on length of lines actually
jc2 has quit [Quit: Leaving.]
<oriba>
I have 3.7e6 lines, quite short
<oriba>
example line: "1,06:11:00,06:11:00,9439925,1,,0,1,"
MercurialAlchemi has quit [Ping timeout: 245 seconds]
<NoNNaN>
Drup: an dsl language (written as ocaml like dsl constructs or even ocaml itself) that has special representation Rep(T) type, and that dsl language is interpreted so I can have transformations at compile time from Rep(T) -> ocaml types (or even other languages), so i can implement inlining, type specialization in the dsl itself
<gasche>
you should compress the fields
<gasche>
eg.
<gasche>
type data = Int of int | String of string
<gasche>
let compress str = try Int (int_of_string str) with _ -> String str
jc2 has joined #ocaml
<Drup>
NoNNaN: you know about MetaOCaml, right ?
tane has quit [Quit: Verlassend]
jc2 has quit [Client Quit]
<nicoo>
adrien: log(1000000) < 20. Cache effects and I/O interleaving has way stringer impact
<nicoo>
stronger*
<NoNNaN>
Drup: yes, I know, but I also want to generate another non-ocaml language code from it
<gasche>
oriba:
<Drup>
well, being able to have something language-agnostic seems impossible to me
<Drup>
NoNNaN: you
<Drup>
NoNNaN: you *will* need an IR that is specific to your language
<adrien>
nicoo: yes, I believe the issue was that for the O(n) code, I/O was spread on two files while with the O(n*log(n)) one, the first file was read and only then was the second one read
<Drup>
or just something like LLVM's IR, but then you don't output something inside the language anymore, it's lower level
<gasche>
on my test file, just sharing strings by caching them in a hashtable divided memory usage by 2 when using Csv.load
<gasche>
hm
<adrien>
from experience I also really believe the issue is not ocaml-csv but how it's used; I also had quite a lot of data to parse and had no issue (on the contrary, performance was excellent)
<adrien>
now, really going to bed
<nicoo>
adrien: Ok, no rewriting a CSV parsing lib then
<oriba>
adrien, if you wait some minutes I put the data online
<oriba>
otherwise, I would need a bit longer and the code also
<NoNNaN>
if it possible use non-extractive parsing
<oriba>
but just calling Csv.load is nothing that must be put online, I think
<oriba>
hmhh scp needs 10 minutes
<NoNNaN>
never assume you have enough memory, somebody may give you an 1PiB sparse file
<oriba>
adrien, gasche NoNNaN nicoo link to data, one line above
SethTIsue has joined #ocaml
<NoNNaN>
Drup: why the language-agnostic way seems impossible? If I can define very language construct (in fact the whole ocaml language) mapping to the target language: if/then/forloop/patternmatching/etc
<Drup>
which IR to which IR ?
<Drup>
there are multiple IR in OCaml, so which one ?
<Drup>
also, it's probably a terrible way 1) to optimize anything 2) to produce code in the target language
<NoNNaN>
Drup: ocaml language construct, so I can use ocaml as the dsl language to for example C codegen
<Drup>
then why not manipulate a representation of the C language ?
<Drup>
OCaml is a terrible representation of C.
<NoNNaN>
Drup: I don't want to represent C, I want to eliminate the abstractions
<Drup>
I think I just don't understand.
<Drup>
you wan't to manipulate something to compile it to C, but you don't want any abstraction, yet you want a DSL like language to do hyper-compilation
<NoNNaN>
you write in high level language + you write to representation -> target language mapping
<Drup>
if you had a reference, why didn't you started by linking it, instead of trying to explain in it me badly ? :D
troydm has quit [Quit: What is hope? That all of your wishes and all of your dreams come true? (C) Rau Le Creuset]
troydm has joined #ocaml
troydm has quit [Remote host closed the connection]
<NoNNaN>
Drup: it is possible to "virtualize" ocaml similar to this? so ocaml itself could be used as a dsl language and each language "construct" eg.: ifthenelse/for/patternmatch could be redefined if required
troydm has joined #ocaml
<Drup>
well, you can perfectly do that with metaocaml, in fact
Anarchos has quit [Quit: Vision[0.9.7-H-20140108]: i've been blurred!]
<Drup>
NoNNaN: I mean, let's look at Trx modules in meta ocaml
<Drup>
a quoted code is of type "code_repr", you can close it, to obtain a Parsetree.expression
<Drup>
then you just need to walk the parsetree to transform it into something else.
SethTIsue has quit [Quit: SethTisue]
<Drup>
in fact, I think it's exactly what's done in FFTW
troydm has quit [Client Quit]
troydm has joined #ocaml
ggole has quit []
<NoNNaN>
Drup: thx for the tip, i'll check it out
<nicoo>
oriba: The example tool provided with the lib performs reasonably well on this file, it seems
<NoNNaN>
Drup: it looks like ber n100 already prepared for different target languages: "Processing code values such as printing and executing is done in the ‘user-level’ libraries."
<Drup>
NoNNaN: sure
oscar_toro has quit [Ping timeout: 260 seconds]
<oriba>
nicoo, what does "reasonably well" mean, and do you look at mem consumption too? perfomance is depending on the hardware....
<Drup>
NoNNaN: be careful, you should look at BER n101
<nicoo>
oriba: The code given with the lib uses Csv.iter and Csv.of_channel
zpe has joined #ocaml
<oriba>
code given with the lib?
<oriba>
Is tehre example code?
<oriba>
there
<oriba>
Did you used my code with Csv.load?
<oriba>
ah, there is example code
<oriba>
how much mem is now used, when using the code I provided?
<oriba>
nicoo, .....
<Drup>
Leonidas: indent, it will be easier to read :D
zpe has quit [Ping timeout: 260 seconds]
<Drup>
(I'm not even kidding)
<nicoo>
oriba: Here, your program fails with an out-of-memory
<nicoo>
Probably because of the giganormous array allocation
<Leonidas>
Drup: usually I do EOL 79, but I just hacked up 500 lines of terrible repetition and was lazy. Will do.
<nicoo>
And then trying to fill it all
<oriba>
nicoo, haha out of memory? lol
<nicoo>
Yes
<nicoo>
I have about 2GB free ><'
<Leonidas>
maybe I can fold these signatures in a more compact way
<oriba>
nicoo, use the other read-function, the one with array was testing
<oriba>
nicoo, use read_csv_file (which just calls Csv.load and prints lines and columns then)
<Drup>
oriba: isn't your csv file well structured ?
<Drup>
(no, I'm not going to open a 150Mo file to look at it in emacs :p)
<Leonidas>
but thats something for tomorrow :-)
<oriba>
Drup, don't know. I did not checked it so far.
<Drup>
oriba: well, you should have started by that, to find a nice memory representation ...
waneck has quit [Read error: No route to host]
<oriba>
Drup, use Csv.load and read it in, then use Csv-lib to make it correct ;-)
<oriba>
Drup, why should *I* find a nice memory representation?
<Drup>
because you know what data are in it ?
<oriba>
I wanted to use Csv-lib and then work with the data
<oriba>
Drup, thats not mY data
<Leonidas>
Funny, I planned to take a look on BER MetaOCaml too :-)
<oriba>
It's data provided by the public transport agency
waneck has joined #ocaml
<Drup>
oriba: doesn't change anything
<oriba>
Drup, 1: read data with library; 2: if data is not well formed, use the library to make it well formed, 3: work with the data
<oriba>
In 1) it stopped
<oriba>
because of big mem usage
<nicoo>
oriba: I did some testing, and Csv.load is the issue. You should use of_channel and some iterator (iter or fold)
<oriba>
nicoo, so you say, Csv. load maxes the data 10 times bigger as the original?
<oriba>
s/maxes/makes/
<nicoo>
The lib has a choice of data representation which isn't very compact, dans Csv.load tries to load it all at once.
<Drup>
well, it returns a string list list, of course it's not compact u_u'
<nicoo>
Drup: I was tempted to be very rude about that
<nicoo>
Drup: Actually, I began a small patch to make it return a string list sequence
<nicoo>
It is companion_cube all the way down !
<Drup>
:D
bezirg has quit [Ping timeout: 245 seconds]
<oriba>
nicoo, so you change the lib? Because it is so good?
<nicoo>
oriba: You can already do what you want with of_channel and the provided iterators. The lib patch was for fun
<oriba>
nicoo, I will look at the other functions then. maybe it can be used just for the parsing.
<Drup>
nicoo: well, "fun () -> next ic"
<nicoo>
Drup: Egg-zactly
<Drup>
oh no, that's a gen, my bad.
<Drup>
not a sequence.
<nicoo>
Yes, but then you know how to turn it in a sequence
<nicoo>
The second part of the patch is make it use less-stupid file access
<Drup>
(fun f -> Csv.iter ~f ic) is the correct sequence
<nicoo>
(Simple enough, they have a channel abstraction)
<Drup>
err
<nicoo>
?
<Drup>
you just need to a bit carefull with lazyness and file closing, that's all
<nicoo>
Yes, I know
<nicoo>
Actually, I'm not sure the lib does the right thing, as-is
thetabyte has joined #ocaml
<thetabyte>
is there a way to pattern match where one of the clauses is if the item being matched is a member of a list? essentially, I'm loading a list of data from json, and I one of my matching clauses to be "if item is a member of this list"
<Drup>
thetabyte: "x when List.mem x l -> ..."
<thetabyte>
Drup: thank you so much! interesting that nowhere in the two classes I've been in that have used ocaml has the 'when' keyword been covered.
<Drup>
it's called a guard
lordkryss has quit [Quit: Connection closed for inactivity]
bezirg has joined #ocaml
bytbox has joined #ocaml
ontologiae has quit [Ping timeout: 258 seconds]
q66 has quit [Quit: Leaving]
madroach has quit [Ping timeout: 250 seconds]
madroach has joined #ocaml
NoNNaN has quit [Remote host closed the connection]
<oriba>
ok, I now have about 780 MB mem usage, when useing Csv.iter and directly convert to a data record, without Csv.load stuff. Not perfect, but at least thats better...
<oriba>
So: Scv.load is not the way for big files. Only convenience for small datasets.