<mbj>
snusnu: #call(input) exception is a Morpher::TransformError, if any other exception comes out its a bug.
<snusnu>
mbj: how's printing being handled?
<mbj>
snusnu: external.
<mbj>
snusnu: There is a pretty printer class that gets configured at clas level.
<mbj>
snusnu: It looks cleaner, but still can be very verbose.
<snusnu>
mbj: so how do i print in case of transformation errors?
<mbj>
snusnu: Evaluation#description
<snusnu>
mbj: hm
<snusnu>
mbj: what's #output in such a case?
<mbj>
snusnu: Morpher::Undefined
<snusnu>
hm
<snusnu>
why not reuse it?
<mbj>
Dont use #output on #success? => false
<snusnu>
#success? => false should be enough to know that #output isn't what you expect
<snusnu>
so why not be able to access both "states" via the same method?
<mbj>
?
<snusnu>
if #success? => false, why not have #output contain #description
<snusnu>
just a quick thought anyway
<solnic>
hey guys
<snusnu>
yo solnic!
<mbj>
snusnu: I think Morpher::Undefined is the better choice.
<mbj>
snusnu: Maybe you want to have a #description also for a successful evaluation.
<snusnu>
mbj: i guess i can see where you're coming from .. i was mostly wondering about "special casing" the method calls based on #success? .. seems not needed
<snusnu>
makes for an if in my client code
<solnic>
dkubb: progress on sql is crazy, you're in-between client projects or something?
<snusnu>
:)
<dkubb>
solnic!
<solnic>
dkubb: this stuff looks super pro :)
<dkubb>
solnic: nope, just trying to get an hour or two each night
<mbj>
solnic: I think we team up a lot on SQL.
<solnic>
hah! that's great, I once had my "2 hours of OSS in the morning" but then my KIDS ruined it ;)
<dkubb>
it's turning out better. I know parsing is technically not needed, but I really like the idea of round-tripping. I think it results in a better ast, and it also results in a better understanding (for me) of what I need
<mbj>
solnic: Its not special case. I nee to fill in a value.
<solnic>
dkubb: so, it's gonna be able to parse sql AND generate sql from ast?
<mbj>
s/solnic/snusnu/
<dkubb>
yeah, everything will be round-trippable
<dkubb>
I found a few bugs already
<solnic>
dkubb: I think this might turn out to be useful for people
<dkubb>
it uses ragel for the scanning, and racc for the parsing
<dkubb>
yeah, I always wanted an SQL parser too
<snusnu>
dkubb: in theory, could an axiom relation be constructed from a sql query string?
<solnic>
I never did anything with ragel/racc
<mbj>
snusnu: What to place into #output if I #sucess? => false because of a rescued anima error?
<dkubb>
in theory, it may be possible for someone to write what they want in SQL, and then have something spit out what it is in axiom
<mbj>
snusnu: nil is not a good choice, since it could be a valid output. (If some person ignores #success?)
<dkubb>
yup
<snusnu>
nice
<mbj>
snusnu: So Morpher::Undefined is better.
<dkubb>
I always wanted to do something in ragel
<dkubb>
it's surprisingly easy once I got the hang of it
<solnic>
dkubb: how is it performance-wise?
<snusnu>
mbj: hmm, not entirely sure i follow your arguments
<dkubb>
I haven't done any benchmarks. it seems to be pretty fast though
<solnic>
nice
<mbj>
solnic: This is the same stack used into whitequark/parser and that thing is pretty fast.
<snusnu>
mbj: then again, i haven't thought much about it, i just know that for both vanguard and ducktrap, i had to call different methods in case "evaluation" failed
<mbj>
solnic: for a pure ruby implementation.
<solnic>
yeah I'm just curious
<dkubb>
I mean, maybe it could process 100-1000 sql queries per second.. I'm totally guessing
<dkubb>
more than I would need, that's for sure
<solnic>
snusnu, mbj: I can't wait to refactor virtus to use morpher :)
<snusnu>
mbj: what i mean is this, if i call either vanguard or ducktrap or morpher, and i see #success? => false, i know that i need to enter a different codepath
<mbj>
solnic: Virtus would be some kind of a high level DSL for morpher. And I'd love to see such.
<solnic>
dkubb: ugh that's pretty fast I suppose :)
<dkubb>
ragel is what mongrel, puma and unicorn use in their http parser, and I've never heard of it as the bottleneck
<solnic>
dkubb: yeah I saw that
<dkubb>
I dunno, maybe it could do more
<mbj>
snusnu: Just as Morpher.
<snusnu>
mbj: so part of that client code could be "dried up" by calling #output in both scenarios
<mbj>
snusnu: You use #success? for steering control flow.
<dkubb>
I think benchmarking it now wouldn't be much point, because it's not done yet. it's easy for something early on to scream, but then as you add to it, it slows down
<solnic>
mbj: yeah I think we're gonna see rom/virtus being a bunch of convenient DSLs built on top of powerful tools, see axiom, morpher and probably more will come
<mbj>
snusnu: I dont think this DRY up should be done.
<snusnu>
mbj: exactly, and i need to know what method to call in case of #success? false, why is that
<dkubb>
that's why I always hold new framework benchmarks as suspect. it's easy to beat rails when you only do 1/10th of it
<dkubb>
or new ORMs
<solnic>
dkubb: yeah for sure
<snusnu>
mbj: i realize that my usecase probably is a bit special, but i'd love to have identical toplevel api for both vanguard and morpher
<mbj>
dkubb: But if you actually use 1/10th of an framework it should NOT be as slow as the 100%
<mbj>
snusnu: vanguard and morpher have a differnd domain.
<snusnu>
mbj: vanguard has Result#{output, violations}, morpher has Evaluation#{output, description}
<mbj>
snusnu: So I dont expect the toplevel domain should be equivalent.
<snusnu>
mbj: "really"?
<snusnu>
mbj: it isn't about the domain tho
<snusnu>
mbj: it's about the interface
<snusnu>
you call a method object
<snusnu>
and get an evaluation back
<mbj>
snusnu: We need to talk about my plans on how to use morpher inside of vanguard.
<snusnu>
if the evaluation fails, your output will not be what you expect
<mbj>
snusnu: Instead of violations, you should use the Evaluation tree.
<solnic>
morpher inside vanguard?
<snusnu>
my point really is about assuming that if #success? is false, #output will contain whatever you need to further process the error state
<snusnu>
why would you need an Undefined slot
<mbj>
solnic: Morpher has a very strong predicate algebra (conceptually, did not implemented all nodes) and it allows traced evaluation. (That you need for reporting!).
<snusnu>
it makes no sense to ever access that
<solnic>
mbj: that seems like a candidate for extraction?
<mbj>
snusnu: I have a totally differend idea for reporting than current #violations
<mbj>
solnic: Yeah. But currently lots of code is shared between predicates and transformers in morpher.
<snusnu>
mbj: it's not about the specific idea of *what* to do with it, it's about where to find it
<snusnu>
if the call to the method object wasn't successful, it will give back information
<mbj>
snusnu: In Evaluation object
<snusnu>
why not access that at the same slot than a valid result
<mbj>
snusnu: The Evaluation object is actually a tree, consisting on other evaluation objects.
<snusnu>
Undefined is NEVER of use
<snusnu>
for further processing
<mbj>
snusnu: Exactly.
<mbj>
snusnu: Exactly that is the idea. Dont use this slot.
<snusnu>
but why
<mbj>
snusnu: Use Evaluation#evaluations for nary root nodes.
<snusnu>
why give it a different name
<snusnu>
hmm i don't get it
<solnic>
when Undefined will be returned?
<snusnu>
why not point #output to #evaluations for an nary root
<mbj>
snusnu: because the #output slot cannot hold everything you need. And you need to be able to get an evaluation tree with all detals also for #success? true.
<solnic>
I can def see a reason to use Undefined instead of nil in some cases
<mbj>
solnic: NO
<snusnu>
so what mbj, if #success? is true, there can be a different object
<mbj>
solnic: Undefined will NOT be returned.
<mbj>
guys, forget everything.
<snusnu>
anyways, the only assumption about #output can be that it is of type Object
<solnic>
sorry I wasn't following earlier discussion
<mbj>
I'll post some code.
<solnic>
sure
<solnic>
in fact, I gotta go :)
<solnic>
ttyl
<snusnu>
have a nice eve
<mbj>
solnic: have fun!
<solnic>
I'm coming back to work BUT THANKS
<solnic>
;)
<snusnu>
heh :/
<mbj>
snusnu: Think about the following constraints:
<mbj>
* Have a predicate like #success?
<mbj>
* Allow full tracing for successful evaluations
<mbj>
* Allow full tracing for unsuccessful evaluations
<mbj>
* Record input and output for all evaluation steps
<mbj>
* Have Nullary, Unary and Nary evaluation objects.
<mbj>
wit this constraints you cannot fill #output for unsuccessful Evaluations with details that need to be present on successful evaluations also.
<snusnu>
why
<snusnu>
it's not like you have to remove #evaluations or #description or anything
<mbj>
why full the object with redundant information.
<mbj>
Why should I decied "what" details you'll be interested in?
<snusnu>
hehe
<snusnu>
well, Undefined will never interest either one of us
<mbj>
If you want to have an #output that mirrors #description, use a generic unary node in top.
<snusnu>
why would i do that
<mbj>
snusnu: To have an #output that you like.
<mbj>
That "generic" unary node, lets call it the snusnu-node would delegate to another node
<mbj>
and set #output based on #success?
<snusnu>
mbj: anyways, it's nothing super important for me, i was mostly wondering exactly why i need to know a different method name for what is obviously the evaluation result, be it some "real" object, or some evaluation result (tree)
<mbj>
snusnu: The tree has all the details.
<snusnu>
mbj: it really only becomes an "issue" once you use more than one of those apis and write "wrappers" for them
<snusnu>
like i do for vanguard and ducktrap
mbj_ has joined #rom-rb
mbj has quit [Read error: Connection reset by peer]
<snusnu>
mbj: my point i guess is this: when i call any of those libs, i know that i will be signalled success via #success? .. and there is *one* object i can access if #success? returns false .. it will definitely NOT live on on the "success path" of my app .. why not make both evaluation results available at the same slot
<mbj_>
Place this Foo infront of everything you wanna have such an #output for.
mbj_ is now known as mbj
<snusnu>
mbj: i guess i'm mostly interested in your reason for having to do evaluation.success? ? evaluation.output : evaluation.something_else
<snusnu>
when you could decide which path to take from here on, and just send #output along
<mbj>
Because what to use for #output for an unary node vs an nary node vs an nullary node?
<mbj>
there is no universal decision here.
<snusnu>
man, i really need to finally remember what exactly those words mean for you!
<mbj>
Yeah
<mbj>
they have a special meaning in my ducktrap/vanguard/morpher code
<snusnu>
nullary, unary, nary
<mbj>
nullary = does not delegate to another node
<mbj>
unary = does delegat to one other node
<mbj>
nary = does delegate to an arbirarily number of nodes
<snusnu>
and what exactly does delegate mean in this context?
<mbj>
"does compultation on to generate #evaluation(input)"
<mbj>
or "#call(input)"
<mbj>
If a node uses another node to do its job, its at least unary or nary.
<snusnu>
sorry if i may sound stupid, but could you give an example for each one?
<mbj>
nary = block
<snusnu>
i'm starting to get it i guess, but i need concrete examples to finally remember :)
<mbj>
unary = guard
<snusnu>
guard is something like a primitve check?
<mbj>
snusnu: So best, start and read the morpher code
<mbj>
guard is a node that allows you to place a predicate on the curent input into your tranformation tree.
<mbj>
for example testing a primitive
<snusnu>
right, ok, and nullary?
<mbj>
a primtive is (in morpher) a nullary node.
<mbj>
Because a primitve check does NOT use another node.
<snusnu>
i see
<snusnu>
thx for that
<mbj>
It turned out this is hte best strategy to write such code. (Tried dozens of times).
<mbj>
So primtive is a nullary paramterize node.
jordanye_ has quit [Quit: MacBook went to sleep.]
<mbj>
Nullary in the "transformation algrebra domain"
<snusnu>
still tho, and i'm deeply sorry, i don't exactly see how those different types of nodes relate to what ends up in #output
<mbj>
But it has a param from outside this domain. (The type to check against).
<mbj>
You are arguing #output on #success? false should be seeded with something that is related to the evalation tree.
<mbj>
But what to place in #output on success false for a nullary node? There is no tree.
<mbj>
Or for an nary node, place all evaluations in #output that succeeded, the one that not succeeded and stoped th block?
<snusnu>
i guess i'm mainly arguing that there are only 2 values both vanguard and morpher can "return" .. the valid/transformed/whateverd result, or something that represents the error
<mbj>
As it does.
<snusnu>
if i program offensively, i expect to *know* how to handle the error i get, on the error path
<snusnu>
be it whateverary
<mbj>
snusnu: We should talk in german.
<snusnu>
heh
<mbj>
snusnu: And you should read the code.
<snusnu>
but then the explanation is lost, and i can never again look it up
<snusnu>
(and i will have to)
<snusnu>
,)
<mbj>
snusnu: morpher is 30% of the size of ducktrap.
jordanyee has joined #rom-rb
<mbj>
And written very straightforward (apart from hash transform, that I'll kill).
<snusnu>
really, i guess my question cannot be solved by reading morpher code
<mbj>
Or I'm currently not stroing enough to explain.
<mbj>
Just answer what to place unde #output in case of failures.
<mbj>
Give me a description what would make sense here.
<mbj>
nil wouldnt.
<snusnu>
my point is, that i'm totally ok with the #output object being "arbitrary" (it is anyway)
<snusnu>
in case of failure, #output would contain the evaluation error object
<mbj>
No
<snusnu>
in case of succes, it'd contain whatever
<mbj>
Thought error.
<mbj>
If #success? is false, the receiver is the evaluation error object.
<mbj>
This is an evalaution tree.
<mbj>
If success is true, the receiver is the evaluation object.
<snusnu>
yeah i was imprecise, i mean the object you use to inspect what went wrong
<mbj>
You use the receiver
<mbj>
evaluation = morpher_node.evaluation(input)
<mbj>
evaluation # this one is your history
<mbj>
and evaluation#success? is the root of your history
<mbj>
it tells you if the history lead to a successful output
<mbj>
or not
<mbj>
there are only 3 types of history nodes.
<mbj>
unary, nullary, and nary.
<mbj>
Read the code luke.
<mbj>
If you dont care about the history: Use #call
<mbj>
call returns output if successful or raises error if not.
<mbj>
This is the "performant" interface that does not create a new object for each step (apart from nodes that MUST do so, think about map).
<mbj>
Its around 20 - 100 times faster (if JIT kicks in) than normal ducktrap.
<mbj>
There'll be a hybrid strategy
<snusnu>
huh? #call was "changed" to raise on #success? => false ? i don't like that
<mbj>
using #call first, if no exception happy, if not use #evaluation to gain a history.
<mbj>
snusnu: Read the history of this chat.
<snusnu>
?
<mbj>
snusnu: I told in the beginning.
<mbj>
#call(input) # output or exception
<snusnu>
but why?
<mbj>
#evaluation(input) # evaluation tree
<mbj>
Much easier to implement.
<mbj>
snusnu: sorry, I'd love to chat more, have to sleep.
<snusnu>
i mostly care about being able to do #call and get back something that knows #success? which is false if something went wrong
<mbj>
snusnu: use hybrid.
<mbj>
snusnu: As I was saing for ages now :D
<mbj>
snusnu: That printout does not have #success? flags currently. and its actually for a successful evalation.
<mbj>
But currently errors are presented this way.
<snusnu>
nice
<mbj>
I dont expect the duplication we had with ducktrap.
<mbj>
Where you saw subtrees multiple times.
<snusnu>
mbj: btw, remember that my main issue was not with hybrid, but with #output :p
<mbj>
snusnu: heh
<mbj>
#output will be the output of the evaluation if it was successful or undefined.
<mbj>
If you need details: Use the tree API.
<mbj>
Because the tree API has node type specific details.
<snusnu>
no need to reiterate here, but i still don't see a reason why #output can't point to the tree API if something went wrong, but i'll stop here ;)
<mbj>
Because the object #output is defined of already holds all the tree api.
<mbj>
there is no concept of an error object
<snusnu>
aha
<mbj>
The tree itself is the result object.
<mbj>
If the result is an error or not, is dictate by #success?
<mbj>
*defined on
<snusnu>
i think i'm starting to see
<mbj>
There is nothing more detailed than the evaluation tree by itself.
<mbj>
Why even try to create an error object.
<mbj>
?
<mbj>
The tree consists of:
<mbj>
* input
<mbj>
* output
<mbj>
* success?
<snusnu>
and the evaluation tree is what you get back from #evaluation
<mbj>
yeah
<mbj>
Its like an "AST without abstract :D"
<snusnu>
and #call does not return such a tree, but only something that knows #success? ?
<mbj>
no
<snusnu>
lol
<snusnu>
damnit
<mbj>
#call == #evaluation.output
<snusnu>
?
<mbj>
BUT without creating that evaluation object. and raising of #output could not be computed.
<mbj>
you could implement call via:
<mbj>
def call(input)
<mbj>
evaluation = evaluation(input)
<mbj>
unless evaluation.success?
<mbj>
raise
<mbj>
end
<mbj>
evaluation.output
<mbj>
end
<mbj>
BUT this would create a full evaluation tree (costly).
<mbj>
For that reason #call is a special code path that raises on the place of the error (instead to create that tree).
<snusnu>
hm, i thought whatever i get back from #call, responds to #success?
<mbj>
Basically #evaluation and #call implement the same algorithm
<mbj>
#call is the fastpath
<mbj>
#evaluation is the slowpath
<mbj>
For that reason the hybrid node takes the fastpath #call first.
<snusnu>
so #call => output or raise ?
<mbj>
yeah
<mbj>
snusnu: grep through this discussion, I typed it 3x already :D
<snusnu>
maybe i didn't get it because it's rather different from ducktrap
<mbj>
Yeah. Its ducktrap reshaped and made small.
<mbj>
If you use the hybrid stuff in front its like ducktrap again.
<mbj>
hybrid_morpher.call(....) => Evaluation#success? true #outpu => Person ....
<snusnu>
aha
<snusnu>
now i get it
<mbj>
This this is only possible because #call and #evaluation are different.
<snusnu>
so yeah, i want the hybrid one
<snusnu>
:p
<mbj>
Jep.
<mbj>
some of my db mapping scripts from to domain/json work 20times faster with this one ;)
<snusnu>
so, how would you actually use the "slow" morpher in an app?
<mbj>
for error reporting....
<snusnu>
ok, by just accepting that it will be run twice
<snusnu>
fair enough
<mbj>
You dont wanna stare on a java like stacktrace without inputs and outputs.
<mbj>
Exactly.
<mbj>
If #call is 1/20th the cost of #evaluation you have 21/20th in case of error and 1/20th in case of success.
<snusnu>
yeah, so what you do is the golden path does hybrid_morpher.call and if #success? => false you do slow_morpher.evaluation
<mbj>
With ducktrap you always have 20/20th.
<mbj>
no
<mbj>
read the souce.
<mbj>
read the gist!
<snusnu>
ok, sorry ;)
<snusnu>
but yeah, same idea
<snusnu>
just that it's wrapped inside the hybrid_morpher already
<mbj>
exactly.
<snusnu>
can you share a few of your ideas how to implement substation on top of morpher?
<mbj>
snusnu: substation would just need to inject some additional context into the evaluator nodes.
<mbj>
snusnu: For example the rom env.
<mbj>
snusnu: and substation would call morphers like "sub machines".
<mbj>
After routing etc.
<mbj>
Substation would be the "higher level assembly and router" to call morpher evaluations.
<mbj>
The ability to record the whole evaluation history will greatly benefit in a substation implementation.
<mbj>
IMHO it should be done by default. Because evaluating a node has side effects in substation (You cannot just create a user twice ;) ).
<mbj>
routing does not have to be http here.
<mbj>
Also domain specific routing.
<snusnu>
so you'd expect that what's currently a processor in substation terms, would be a morpher node, and what's currently a handler in substation, would be more or less a "custom" transformation, that oftentimes won't be inversible?
<mbj>
yeah
<mbj>
handler == evalaution tree nodes
<mbj>
processors == morpher nodes
breakingthings has quit []
<snusnu>
just a wild thought, but if one wanted, one could implement undo with that
<mbj>
snusnu: You cannot undo the sending out of an email.
<mbj>
snusnu: dont even try.
<snusnu>
obviously within domain limitations
<mbj>
snusnu: :D
<snusnu>
hehe
lfox has joined #rom-rb
<mbj>
snusnu: with substation I don think the inversiblity is the intersting property of morpher. Its more the tracablilty.
<snusnu>
so, it'd be cool to have OR, AND, PARALLEL_AND, etc nodes in morpher
<snusnu>
mbj: i fully agree
<mbj>
Yeah
<mbj>
I'd love to add these.
<snusnu>
it'd be very awesome
<mbj>
BTW if you talk about morpher and nodes, there are transforming nodes and predicate nodes.
<snusnu>
i'd love to do cookie deserialization/sanitization/mapping in parallel with e.g. parameter extraction/sanitization
<mbj>
I think the synchronization / looking costs will be gigger than the gain.
<mbj>
*bigger
<snusnu>
OR nodes could be used for doing json/html requests, etc
<mbj>
And they'd be assemble from predicates ;)
<mbj>
BTW its better to implement them as XOR ;)
<snusnu>
hehe
<mbj>
But what if you need to send two emails after a signup.
<mbj>
That stuff is expencive because of latency.
<mbj>
Here a parallel setup would make sense.
<mbj>
Or any kind of RCP parallelization that is not data dependant.
<mbj>
s/RCP/RPC/
<mbj>
Like sending out many queries to DB at once.
<snusnu>
yeah
<mbj>
If I ever design a languge, morpher type evaluaton trees etc would be first level citizens.
<mbj>
You should be able to evaluate any pice of code with an evaluation tree, if you whanted to.
<snusnu>
yeah, would allow for awesome dataflow visualizations for debugging
<snusnu>
and not only that :)
<mbj>
yeah
<mbj>
snusnu: sidenote, I think morpher is easily portable to any other language around.
<mbj>
snusnu: making a java port would be relatively easy.
<snusnu>
mbj: might even be cool for jruby performancewise
<snusnu>
i've not written java in ages
<snusnu>
must be more than 8 years now
<mbj>
heh
<snusnu>
mbj: btw, how will vanguard use morpher?
<snusnu>
mbj: isn't it mostly about adding some predicate nodes?
<mbj>
snusnu: Yeah
<mbj>
snusnu: vanguard will probably execlusively use predicate nodes.
<mbj>
snusnu: And add a vanguard specific unary node that stores the information about how to generate error messages.
<mbj>
snusnu: Probably it'll only store the associated uid to reference a message template.
<mbj>
snusnu: And I'll provide some flat accessors to the evluation tree for stuff like range mismatches etc.
<mbj>
snusnu: But mostly I'll be morpher. I exect a big reduction of code because we "only" have to turn an evaluation tree into a message.
<mbj>
And the tree contains all the details.
<mbj>
morpher predicates should be really easy to turn into js.
solnic has joined #rom-rb
bffff has joined #rom-rb
<snusnu>
mbj: sorry, got sidetracked, having them generate js would be awesome!
<snusnu>
mbj: re the way vanguard uses morpher, sounds cool, morpher really seems to scratch an important itch