<toolslive>
anyway, there have been rather interesting experiments with haskell DSLs for .... (damn you're first)
<ousado>
I can't find much. A little surprising, Janestreet certainly has lots of applications for that kind of stuff
pib1942 has quit [Remote host closed the connection]
<ousado>
but maybe they just use CUDA
ollehar1 has joined #ocaml
<wmeyer`>
ousado: I believe hongbo is working on some C qutations mechanism
<toolslive>
there's an interesting paper called "deconstraining DSLs".
<ousado>
yes, indeed :)
<ousado>
wmeyer`: unfortunately he's not working on his blog, though :)
<toolslive>
the problem being that if you have a dsl with different backends you don't want to end up with the gcd of the backends as front end
ggole has quit []
ollehar1 has quit [Quit: ollehar1]
asmanur has joined #ocaml
<ousado>
haxe had to deal with similar issues, I think they took a sensible approach, having the gcd as true x-platform API, and offer platform specific APIs on top of that
<toolslive>
yes, but then you cannot take full advantage of the backend's features...
<ousado>
you can
<ousado>
it's just not x-platform anymore
<toolslive>
ah yes, like that.
<toolslive>
it's a balancing exercise.
<ousado>
it works out very well
<ousado>
but I'm not sure that wouldapply to the cuda/opencl stuff equally
<ousado>
however, opencl aims to be a standard already
<toolslive>
in the end, we did not go for the gpgpu approach as the cost of moving the data back and forth (to the card, off the card) was too high
<ousado>
toolslive: what was that for?
<toolslive>
rateless codes for storage.
ttamttam has joined #ocaml
<toolslive>
the codec we used can be seen as a very large sparse linear system
<toolslive>
so encoding & decoding would be done on the gpus to speed it up
<ousado>
ah - basically you want a function, that when executed gives you some file?
<toolslive>
no I have a system, that when solved gives me the file.
<toolslive>
so encoding is creating the system from the file, decoding is solving the system
<thelema>
the decoding still needs an appropriate number of data blocks
<toolslive>
'enough'
<thelema>
I'm surprised that there's much use for GPU in this; decoding can be done incrementally in a very straightforward fashion, no?
<toolslive>
the cool thing is that you can add the description of an equation to the equation, so you don't have to micromanage your encoded blocks
Snark has quit [Read error: No route to host]
<toolslive>
well, in the end it's an 8400x8400 system
<toolslive>
well, in the end it's an (8400 + z) x 8400 system
ollehar has quit [Ping timeout: 276 seconds]
<toolslive>
where z is some number of extra equations.
zpe has joined #ocaml
<thelema>
and each time you get a data block, you simplify that system; just like finding the value of one variable.
<toolslive>
the GPU approach was about as fast as the CPU approach, but offloading makes your CPU free for other stuff.
Snark has joined #ocaml
<toolslive>
well, the incremental approach works but is too slow,
<toolslive>
in the beginning you're reducing the system a lot without recovering any block.
<toolslive>
and after a while, you start getting more and more data blocks recovered for every encoded block you receive.
<toolslive>
better wait until you know you have enough encoded blocks and then solve the system in one go.
<toolslive>
it's faster, and simpler.
<thelema>
because less wasted simplification... got it.
<ousado>
did you have any patant issues?
<ousado>
*patent
<toolslive>
sorry, can't talk about the IP part ....
<toolslive>
but the I can shamelessly plug the company website: www.amplidata.com
<ousado>
very interesting
<ousado>
what's TCO in that context?
<thelema>
ousado: total cost of ownership
<toolslive>
in the end, storage === telecom, so everything that works in telecom, works in storage.
malo_ has quit [Ping timeout: 260 seconds]
<ousado>
ah of course
zpe has quit [Ping timeout: 248 seconds]
<ousado>
Aha: "Enhancements to Amplidata’s patent-pending BitSpread Erasure Coding software, the industry’s first rateless erasure coding, now allows AmpliStor to drive 10 Gigabit Ethernet network interfaces to full capacity per storage controller."
<toolslive>
well, that's the commercial bla bla. I just solved the math & CS problems.
<ousado>
haha
<thelema>
toolslive: it's awesome that you're using OCaml for this work
contempt has joined #ocaml
<ousado>
yeah.. the real work is always done by the PR guys
<toolslive>
well, it's not only ocaml. ocaml for coordination, C & asm for the crunching.
<toolslive>
not only that. the whole solving of the systems was plain C.
malo_ has joined #ocaml
<toolslive>
the xorring in asm
<thelema>
toolslive: yes, the xoring has to be done in asm; why solve systems in C other than faster solving?
<toolslive>
you wanted to solve the system in ocaml?
<thelema>
alternately, why do any of it in OCaml?
<ousado>
sounds like a code generation job to me
<ousado>
but htat doesn't mean anything, since everything sounds like a code generation job to me
<toolslive>
well, one of the things where OCaml sucks (by design) is different views of the same thing.
<toolslive>
you want to look at some bytes as a number, or as a byte array, or as 2 numbers or as ....
<toolslive>
just lifting a char out of a number is very slow in OCaml
<thelema>
fwiw, there's some primitives added in svn trunk to improve this.
<thelema>
I'm not sure it includes lifting char from int32
oriba has quit [Quit: oriba]
<toolslive>
hm, can you give me a link?
<thelema>
lookoing
<thelema>
*looking
contempt has quit [Ping timeout: 264 seconds]
contempt has joined #ocaml
<toolslive>
the other thing where ocaml is bad is explicit memory management (like scratchpads etc)
<toolslive>
so I used OCaml for the part where OCaml is perfect, C for the part where C is ok (I could have used Ada or whatever) and ASM for the fancy intel instructions....
<toolslive>
actually, we were contemplating Haskell too, but in 2009 the haskell platform (toolchain, ...) was not mature enough.
<toolslive>
seems OCaml has been overtaken...
<thelema>
toolslive: unfortunately yes, although we're striking back with better tools
<toolslive>
yes, but I can't even get typerex to work ok in my emacs.
<toolslive>
opam, is a massive improvement btw.
<thelema>
toolslive: even the v2 branch
<thelema>
?
zpe has joined #ocaml
<thelema>
(or typerex)
<thelema>
*of
<toolslive>
the opam installed one. 1.99+
<companion_cube>
you may also try merlin as an editor tool
<toolslive>
I tried that too today, I couldn't get it installed via opam.
<toolslive>
so there's still work there.
<thelema>
merlin is working for me...
<companion_cube>
for me too
<thelema>
I'm just starting to figure out how to use it; but it does do something in my emacs