<mongrelion>
can I start using rom in my project already or should I wait a little bit more? If I can start to use it already, where is a good starting point to check out a how to put it all together? thanks.
djsell has quit [Ping timeout: 246 seconds]
dkubb has joined #rom-rb
dkubb has quit [Client Quit]
dkubb has joined #rom-rb
<postmodern>
mongrelion, i would wait
<postmodern>
mongrelion, pieces of it are done, but not the main components
<splattael>
mbj: np, I'm still reading mutant's source
<mbj>
splattael: hah
<mbj>
splattael: You found Mutant::Killer::Rspec ?
<splattael>
yes
<mbj>
splattael: BTW mutant is the last "dirty" projects of mine
<mbj>
splattael: Mostly because it is not mutation covered
<mbj>
I plan to fix this with the 0.3 release
<mbj>
But probably this will go into 0.4
knowtheory has quit [Quit: Computer has gone to sleep]
<mbj>
splattael: So writing a killer is basically: Identify tests to run, and run them.
<mbj>
There are many identification strategies, what is the most common style with minitest?
<splattael>
you mean file pattern?
<splattael>
glob
<splattael>
test/**/*_test.rb
<mbj>
Where files for Foo::Bar would be?
<mbj>
test/unit/foo/bar_test.rb ?
<splattael>
ah, ok. it depends ;)
<splattael>
yes
<splattael>
I think same "rules" apply as for rspec
<mbj>
so somehow a 1:1 mapping between subject (class) and killing test file
<mbj>
The rom project has a file per public method
<splattael>
ah ok
<splattael>
file per class is mostly fine
<splattael>
for minitest
<splattael>
but it depends.
saturnflyer has joined #rom-rb
<splattael>
mbj: but I get the picture
<saturnflyer>
hey mbj! sorry. I got distracted
<mbj>
does the minitest api have a nice way to run a specific test file?
<mbj>
saturnflyer: np, I was late also ;)
<splattael>
saturnflyer: np we were late too ;)
<mbj>
splattael, saturnflyer: Let me do a short intro about mutants design and the terminology
<mbj>
Mutant has the concept of a "subject" that gets mutated.
<mbj>
currently subjects are, scopes (::Foo, ::Bar), namespaces (::Foo::Bar**), instance_methods (::Foo#bar) and singleton_methods (::Foo.ba)
<mbj>
The distinction between scopes and namespaces is new in 0.3, and not present in 0.2
<mbj>
0.2 is the current "stable" release"
<saturnflyer>
ok
<splattael>
ack
<mbj>
Currently only the method subjects can emit mutations
<mbj>
The other subjects delegate to method subjects per reflection
<mbj>
So a scope subject (::Foo) will reflect about instance and singleton methods defined here
<mbj>
instantiate method subjects and emits mutations
<mbj>
These mutations can be injected into the ruby vm via the subjects api again
<mbj>
This is done inside the runner
<mbj>
The runner was greatly improved in 0.3
<mbj>
The problem: There is no "per subject" configuration about how to kill a specific mutation
<mbj>
Currently this information is global
<mbj>
So you have to decide about a "subject => how to kill mutation" strategy, and use this over the whole project.
<mbj>
So we can add a minitest killer, and add the options to set it globally
<mbj>
Also note, each mutation is killed inside a fork() so side effects of the mutated code running do not leak into next tests and causimg them to fail
<mbj>
Ideally we add both, the flexibility with subject <=> killer mappings and a minitest killer
<mbj>
Often there is code in subclasses that is private and the mutations should be killed via an inherited parent class interface
<mbj>
We should have some kind of a "kill expresion" that can go into a .mutant.yml file
<mbj>
splattael, saturnflyer: Does someone of you know the internas of the minitest api?
<mbj>
We cannot spawn a new interpreter and should bypass the cli if possible.
<saturnflyer>
not well
<splattael>
in minitest 5 you just do Minitest.run
<splattael>
mbj: I would call the strategy MiniTest not TestUnit
<mbj>
splattael: sure
<splattael>
Minitest (for 5)
<mbj>
splattael: I mix test/unit and minitest all the time.
<splattael>
I think supporting MT 5 for now is ok.
<mbj>
I'd love to deduplicate between the rspec strategy and the test unit strategy as far as possible
<mbj>
Basically the rspec strategy only appends _spec.rb where the minitest/testunit one appends _test.rb for the test selection.
<splattael>
true, so wie have spec-y things (rspec and minispec) and unit-y things (minitest and well "old" test/unit)
<splattael>
wie -> we
<mbj>
Ahh okay
<mbj>
I forogt minitest also ships rspec style syntax
<splattael>
aka minispec (or minitest spec) yes
<splattael>
they use the same runner
<mbj>
nice
<saturnflyer>
we can configure our test suites to load any file. is my understanding correct that mutant currently just looks at the end of a file name for which way to handle it?
<mbj>
no
<mbj>
saturnflyer: Given a specific subject ::Foo::Bar#baz
<saturnflyer>
ok
<mbj>
mutant will decide based on current strategy how to kill mutations
<mbj>
in the best supported "--rspec-dm2" strategy it will run tests in spec/unit/foo/bar/baz_spec.rb
<saturnflyer>
i don't understand what you meant above then.
<mbj>
It reflects on the classes namespaces and method name
<mbj>
saturnflyer: Mutant identifies "subjects" these subjects are currently methods.
<mbj>
So if you have a method ::Foo::Bar#baz
<mbj>
mutant will execute a strategy defined action
<mbj>
for rspec that action would be the execution of the specs spec/unit/foo/bar/baz_spec.rb to kill the mutations on ::Foo::Bar#baz
<saturnflyer>
ok.
<saturnflyer>
thanks
<splattael>
mbj: `bundle exec rake ci` should pass on v0.2.20, right?
<splattael>
with 1.9.3
<mbj>
splattael: jo
<mbj>
splattael: But the setup of devtools might be outdated
<mbj>
splattael: I can create a branch that passes if you like!
<splattael>
np I'm installing flog and flay :)
<mbj>
saturnflyer: So for supporting another test framework we just need to add a new strategy
<saturnflyer>
ok. got it
<mbj>
saturnflyer: And deduplicate it with existing ones.
<splattael>
mbj: it keeps saying "Flog is not available. In order to run flog, you must: gem install flog"
<saturnflyer>
that's odd. why would you need to dup the version?
<saturnflyer>
running bundle update successfully so far. thanks!
<snusnu>
saturnflyer: tbh i have no idea why rubygems tries to mutate the version string, but it obviously does
<saturnflyer>
yeah. that's a bug
<snusnu>
saturnflyer: we like to freeze all our consts, because well, they're supposed to be consts .. all our other libs do the #dup in the gemspec already, dunno why yardstick didn't do it till now
<saturnflyer>
yeah. freezing makes sense. but it's strange that rubygems would try to manipulate that
<snusnu>
saturnflyer: totally, i was *really* surprised when i saw that for the first time
<snusnu>
i mean, what on earth do you need to mutate the version string for?
<snusnu>
...
<saturnflyer>
yeah. currently digging into rubygems source to find out
<snusnu>
cool
lorenzo_ has quit [Quit: leaving]
<snusnu>
mbj: around?
djsell has joined #rom-rb
<mbj>
saturnflyer: for some reasons rubygems parses the gem version with gsub!
<snusnu>
lol
<snusnu>
mbj: got some time to talk in memory adapter ?
<mbj>
snusnu: tbh, no!
<snusnu>
damnit, thought so :)
<saturnflyer>
looks like the frozen version problem may be in bundler, FYI
<mbj>
saturnflyer: The last time I investigated it was in rubygems ;)
<saturnflyer>
the only place I found gsub! related to the gemspec was from a yaml load. yaml doesn't mark frozen strings
<mbj>
That gateway delegates all operations to an array of tuples.
<mbj>
If you do #delete(tuple) you delete the tuple from the array
<mbj>
if you do #each on the gateway you filter the tuples and yield the matching ones
<mbj>
so pick an existing gateway and modify it.
<snusnu>
so here's the thing, write ops need to propagate to the respective base relations tho, as long as the process is alive, you should be able to reach, say, all :people relation tuples
<snusnu>
the sql adapter translates a relation to a sql string, Insertion operations might be *somewhere* in the tree of ops
<snusnu>
my feeling is, that we need to "mimic" that, only, i have no real idea how to achieve that
<snusnu>
it somehow feels like, ok, let everything do it's thing normally, but whenever a read/write is actually propagated down to a base relation, intercept there, and use the array of tuples stored in that "slot" of the hash that is the in memory database
<mbj>
snusnu: At this change you should IMHO only write to base relations wrapped in a gateway
<mbj>
Inmemory gateway for this instance
<snusnu>
mbj: wdym by "at this change"?
<mbj>
snusnu: s/change/stage/
<mbj>
We dont plann write to joins currently.
<snusnu>
oh yes we do
<mbj>
snusnu: Start simple!
<snusnu>
no, i'm afraid i have to say that's wrong
<mbj>
We have to catch dkubb than
<mbj>
Dunno how he plans to implement writes to joins
<mbj>
I have some ideas
<snusnu>
if we cannot write to joins, we can't support relationships, and worse, we cannot even "fake" them by simply having a mapper for a joined relations (say people and profiles), in relationship speak: people.has_many :profiles
<snusnu>
mbj: it's already implemented
<snusnu>
!
<snusnu>
but there's no code that actually tries to "translate" it to either a QL or to access in memory structures
<mbj>
snusnu: yeah it is
<mbj>
snusnu: So it will work OOTB
<snusnu>
yes
<snusnu>
if we intercept at the right place
<mbj>
snusnu: left and right will be gateways
<mbj>
so *works*
<snusnu>
yeah, at the leafs
<snusnu>
so, we need to hook into the right place, and read from a big in memory hash
<mbj>
snusnu: If you do gatewayed_relation.restrict(:foo => :bar) you get a gateewayed relation back!
<mbj>
so you dont have to hook anything.
<mbj>
It *works*!
<snusnu>
no
<snusnu>
at some point you need to specify the FROM part .. in which base relation the actual tuples live
<snusnu>
and you need to access the hash at that base relation's name slot
<mbj>
?
<mbj>
We are talking about in-memory
<mbj>
now sql?
<snusnu>
i just used it as reference
<snusnu>
the thing is, data actually lives in base relations
<mbj>
jo
<snusnu>
and they are stored in a hash, with keys being the relation names
<snusnu>
at every key, there an array of tuples, or rather, a base relation instance
<mbj>
maybe
<snusnu>
wdym?
<mbj>
I'd do it in a differend manner
<snusnu>
tell me
<mbj>
There is a registry with a hash
<mbj>
but you check out a gateway from that registry
<mbj>
and that gateway will only know about a specific base relations tuples
<snusnu>
ack so far, it's how i imagine it too
<mbj>
so where is the problem? The gatewayed relations will always live in the correct leaves
<mbj>
They'll bubble "up" while pusing RA ops down.
<mbj>
so both gateways will consume #insert and add tuple to the array of in memory tuples!
<snusnu>
where is the place to "intercept" that makes sure that eventually people attributes end up in registry[:people] and profile attributes in registry[:profile]
<snusnu>
dude i don't get it .. somewhere there is a registry object that is backed by a hash keyed by base relation names … if an insertion happened, basically the respective base relation needs to be replaced in the correct "slot" because we're immutable
<snusnu>
where does this code go?
<snusnu>
there's no such thing as free lunch
<snusnu>
unfortunately
<mbj>
snusnu: if we have an in-memory adpater the list of tuples must be mutable.
<snusnu>
mbj: but the axiom ops are not
<mbj>
snusnu: no problem
<mbj>
snusnu: my axiom-adapters send commands to mutable datastores also
<snusnu>
of course, but imo that's not even the point, if the in memory db is immutable or mutable
<snusnu>
at least not the core of what i'm not yet understanding
<mbj>
adapter.relation(:people) # an instance of in memory gateway
<mbj>
InMemoryAdapter itself is frozen
<mbj>
but does not freeze the list of tuples
<mbj>
adapter.relation(:people) # an instance of in memory gateway having a ref to tuple list
<mbj>
mutable tuple list
<mbj>
the gateway itself is also frozen
<mbj>
but does not freeze that specific ivar
<snusnu>
right, that's all how i think it should work too, but my question is this, how is the gateway implemented? i.e. how does it make sure that in some "complex" joined/whatever'd relation, when write ops are already pushed down to the relevant base relations, it actually reads and writes from those?
<mbj>
snusnu: The join node will be in memory and partition the inserts for you, so both gateways will be hit with the "part" of the tuples.
<mbj>
snusnu: If you join two nodes from differend adapters you should get the join node back unwrapped.
<snusnu>
that's why i said before it's already implemented
<snusnu>
its already pushed down to the proper base relations
<mbj>
And just accept left and right are gateways and append the tuple to the set of tuples tracked within the inmemory adapter.
<snusnu>
when i see a hash of tuple arrays tracked inside the in memory adapter, i wonder how the content of that hash will be modified
<mbj>
snusnu: the hash will not be modified
<mbj>
only the arrays/sets the hash contains
<mbj>
*as values
<snusnu>
of course
<snusnu>
i wasn't precise enough but meant that
<mbj>
snusnu: for insert we'll do tuples << the_inserted_tuple
<mbj>
@tuples << the_inserted_tuple
<snusnu>
*where*
<mbj>
for delete we'll do
<snusnu>
????
<mbj>
Gateway!
<snusnu>
*where* in there?
<mbj>
def insert(tuple)
knowtheory has joined #rom-rb
<mbj>
@tuples << tuple
<mbj>
self
<mbj>
end
<mbj>
def delete(tuple)
<mbj>
@tuples.remove(tuple) # assuming tuples is a set
<mbj>
end
<mbj>
# and Set#remove is ruby stdlib api ;)
<snusnu>
i was just about to say those ops accept sets
<snusnu>
but still, how does it work then?
<mbj>
dude
<snusnu>
lol
<mbj>
[1, 2].to_set << 3
<mbj>
works!
<snusnu>
hehe
<snusnu>
but
<mbj>
1 2 3 are tuples
<mbj>
1 2 are alrady in the adapter
<mbj>
3 was inserted
<mbj>
Just DO it.
<mbj>
:D
<snusnu>
dude, i tried it already and failed
<snusnu>
remember when i said i got inmemory working?
<snusnu>
there's fuckin' code in dm-mapper that does NOT work
<mbj>
snusnu: Because you did #insert on RAW relations
<mbj>
WITHOUT a gateway
<snusnu>
yeah
<mbj>
so do it WITH a gateway
<snusnu>
so, i don't understand the effect of the gateway properly it seems … if i completely rewrite #insert to access the in memory db, how on earth do i restrict that to only happen on base relations?
<snusnu>
btw, i'm laughing already, because i *know* that i just don't get it, i'm not trying to convince you it's hard .. i just seriously don't get it
<snusnu>
:D
<mbj>
no problem
<mbj>
snusnu: you dont rewrite #insert
<mbj>
snusnu: You just intercept #insert
<snusnu>
but i only need to intercept it if it's for a base relation, right?
<snusnu>
if you say yes, and you then tell me how i can do that, i'm good
<snusnu>
hehe
<snusnu>
is it as simple as asking the wrapped relation if it is a base relation? and only then intercept
<snusnu>
?
<mbj>
snusnu: it is
<snusnu>
and if it were that simple, would it be a good way of doing it?
<mbj>
snusnu: IMHO yes.
<snusnu>
man
<snusnu>
that was hard (for me) wasn't it?
<mbj>
hah
<mbj>
dunno
<mbj>
busy ;)
<snusnu>
lol
<snusnu>
mbj: thx for bearing with me :)
<mbj>
np
<snusnu>
mbj: and as a sanity check, i would need to intercept #each for base relations and read from the in memory db there
<mbj>
snusnu: If your gateway consumed a restriction you need to run that restriction against the tuples inside the base relation
<snusnu>
mbj: and what/where is the base relation?
<mbj>
snusnu: s/inside the base relation/the gateway/
<mbj>
snusnu: It is not sooo super simple
<snusnu>
right ...
<mbj>
snusnu: But just start with operations on base relations!
<mbj>
snusnu: Than try a cross gateway join ;)
<mbj>
snusnu: bottom up
zekefast has quit [Quit: Leaving.]
<snusnu>
mbj: believe me, what i want to have working, is the in memory adapter, and that alone, i don't care about fancy cross adapter joins (for now)
<mbj>
snusnu: So start with a simple base relation wrapping gateway
<snusnu>
mbj: i'm actually gonna give it a go i guess
saturnflyer has quit [Quit: saturnflyer]
<snusnu>
mbj: the whole gateway thing still confuses me tho, should i start by including https://github.com/dkubb/axiom/blob/master/lib/axiom/relation/proxy.rb into the gateway relation, have it wrap an adapter and a relation, or shall i actually return a materialized base relation subclass from in_memory_adapter.gateway(:people) and overwrite #insert #delete #replace and so on?
<snusnu>
ugh, i can't stand my brain resisting so much against this ...
<snusnu>
i mean, resisting against finally understanding how it's supposed to work ;)
knowtheory has quit [Quit: Computer has gone to sleep]
<snusnu>
mbj: i guess the subclassing idea can't work
cored has quit [Ping timeout: 240 seconds]
<mbj>
snusnu: The gateways are the *ugliest* part of the axiom adpaters.
<mbj>
snusnu: Check the arango adapter gateway
<mbj>
It is my latest and probably cleanest
<snusnu>
ok will do
<snusnu>
mbj: well, there's almost NO difference to the sql one
<snusnu>
but that's because you also generate a QL
<snusnu>
in memory is differebnt
<snusnu>
what i mean is this, i don
<snusnu>
don't get any new insights from looking at the arango gateway vs. the sql gateway
<snusnu>
with a QL adapter, you need to visit the relation ast and produce statements
<snusnu>
with the in memory one, you need to intercept at the right place
<mbj>
snusnu: Just dont create a ql and excute the primitive operation against the tuples...
<snusnu>
mbj: and basically copy the gateway, and add #insert #delete #replace methods that do "def insert(tuples) if relation.instance_of?(Axiom::Relation::Base); adapter[relation.name] << tuples; end"
<snusnu>
mbj: ?
<mbj>
snusnu: for example
<mbj>
snusnu: but you can also call #insert on a rename
<snusnu>
mbj: fuck it i'm just gonna do that now and see what it does
<snusnu>
mbj: but eventually, tuples must be inserted into base relations
<mbj>
snusnu: yeah play around!
<mbj>
snusnu: And write specs ;)
<snusnu>
mbj: axiom will propagate that down, won't it?
<snusnu>
mbj: i will *start* with a spec
<snusnu>
mbj: the way i "understand" it is that an Insertion basically pushes the relevant attributes to either self, left or right "until it's at a leaf", and that leaf is a base relation … ?
<mbj>
snusnu: yeah
<snusnu>
mbj: ok, then nothing special should be needed for things like #rename
<snusnu>
mbj: only base relations need to be intercepted
<snusnu>
mbj: if the gateway wraps a base relation, access tuples from the in memory db
<mbj>
snusnu: But you have to keep the gateways wrapping nodes that have functional #insert
<snusnu>
mbj: of course
<snusnu>
mbj: so i need to intercept, and then, hmm .. damn
<mbj>
snusnu: IMHO there should not be functional implementations for #insert and #update and #delete
<mbj>
We could still have #union and #difference as functionals. The #insert and #update would be pushed down always.
<mbj>
This would ease our job ;)
cored has joined #rom-rb
<snusnu>
mbj: i guess i don't know what you mean
<snusnu>
mbj: insertion is implemented by returning a proxy to a union'ed relation
<snusnu>
mbj: do you mean we should just intercept union and difference if they act on base relations?
<mbj>
snusnu: no
<mbj>
snusnu: I'm going to sleep now
<mbj>
snusnu: Can you release a substation with Substation::Chain support?
knowtheory has joined #rom-rb
<snusnu>
mbj: hehe, yeah, can do so tomorrow
<mbj>
snusnu: I could need this know, on a project where we cannot use git sources :(
<mbj>
s/know/now/
<mbj>
So feel pressure pls ;)
<snusnu>
ok, i actually only haven't done it yet because i didn't get around to updating the README
<mbj>
Going to sleep, you can also add me to rubygems I'll do the release.
<cored>
solnic not around still
<cored>
:-(
<snusnu>
mbj: no worries, i'll do it so that when you wake up it'll be on rubygems
<snusnu>
cored: he's asleep i'd expect, its 1am where he lives
<snusnu>
mbj: or do you want me to do it *now*?
mbj has quit [Ping timeout: 240 seconds]
<cored>
snusnu: oh, got it
<cored>
snusnu: I'll try to make it early in the morning tomorrow then, so I can catch him