<snusnu>
dkubb: i'm asking, because while i obviously think that sql write support is really important, i wonder if it would be even more beneficial for rom development to have #group #ungroup support in axiom
<snusnu>
dkubb: also, that would imo postpone the need for relationships
<snusnu>
dkubb: as it would make it simpler for rom-mapper to map ev/ec i guess
<snusnu>
dkubb: if rom-mapper were to receive grouped tuples from axiom, sth like: { name: 'John', tasks: [ { id: 1 }, { id: 2 } } it would be very simple to implement … maybe it wouldn't even be necessary to begin with … ducktrap can already map such a structure from tuple land to object land and back
<snusnu>
!
<snusnu>
dkubb: imo it's a worthy goal to postpone implementing relationships (or rather, a dsl for them) .. if we have the fundamentals in a shape where it's easy to live without them, that must be a sign that we're doing it right
<snusnu>
dkubb: i guess if axiom allowed me to group tuples in such a way, i probably wouldn't even miss relationships all that much .. i'd just write the joins + grouped relations
<snusnu>
dkubb: i know that for most persistence to work properly, we need something (relationships) to infer the dependencies for the uow, otherwise session can't make sure to write the tuples in the proper order
<snusnu>
dkubb: but from rom-{relation, mapper}'s pov, and in order to interact with the in memory reference implementation, they're not really needed
therabidbanana has quit [Quit: leaving]
<dkubb>
I would think most of the time relationships would be implemented using group and ungroup
<dkubb>
you'd do the join you need, with the filtering and then one of the last operations would be to group it
<dkubb>
snusnu: sql write generation would be nice to have, but I am leaning towards the grouping stuff
<dkubb>
I can't remember if it's group/ungroup or nest/unnest in RA
<dkubb>
but the implementation is roughly the same.. you restrict/join/union/whatever and then before you pull the data back you nest it
<dkubb>
well, technically you can nest it anytime, and the optimizer will probably reorganize everything so it's done as late as possible
<dkubb>
snusnu: if you had a memory implementation then there'd be nothing holding you back on the ROM side of things
<snusnu>
dkubb: yeah, that's what i was thinking, now that we have the in-memory adapter, if axiom would support group/ungroup, we would have everything in place to do the mapping
<snusnu>
dkubb: session would work too of course, but we would need some sort of FK relationship object in order to infer write order dependencies for object graphs
<snusnu>
dkubb: but as long as persistence doesn't matter, that's not needed
<snusnu>
dkubb: so strictly speaking, we wouldn't even need relationships for the dependency resolving uow .. but we would need the concept of FK's between axiom relations
<snusnu>
dkubb: relationships (as we know them) can be built on top of that information, using join and group under the hood
<dkubb>
snusnu: write order is only important when you have to write to the datastore to have one of the default values set. if we use some kind of external (thread-safe) counter, or UUIDs or something for the keys then we'll be fine
cored has joined #rom-rb
cored has quit [Changing host]
cored has joined #rom-rb
cored has quit [Ping timeout: 276 seconds]
snusnu has quit [Quit: Leaving.]
mbj has joined #rom-rb
mbj has quit [Ping timeout: 276 seconds]
mbj has joined #rom-rb
solnic has joined #rom-rb
solnic has quit [Read error: Connection reset by peer]
<solnic>
you do use the returned relation to check data that's inside
<solnic>
it doesn't work like that with other adapters
<dkubb>
yeah, then the other adapters are probably wrong.. they need to be fixed
<solnic>
I mean, blah, it works like that too BUT you can just check the original adapter instance
<solnic>
and you will get data back
<solnic>
you know what I'm talking about right?
<dkubb>
it means those other adapters are working in a way that isn't compatible with how Axiom::Relation itself works
<dkubb>
oh yeah I know what you're talking about
<dkubb>
100% clear. I thought about this alot
<solnic>
damn
<solnic>
I must say
<solnic>
this is a huge difference
<dkubb>
I knew you'd complain about it :P
<solnic>
I'm not complaining
<solnic>
I just wasn't aware that it was supposed to work like that
<solnic>
we built stuff with the assumption it works like DO adapter works
<dkubb>
that was probably a side effect of the implementation, not an intentional interface
<solnic>
I need to think about this, maybe it's not such a big deal for current rom's implementation
<solnic>
my gut feel tells me it is :)
<solnic>
right
<solnic>
if I understand this right
<solnic>
this would mean we have to update registry with new relations every time a mutable action was called
<solnic>
like insert or delete
<solnic>
those produce new relations
<solnic>
so our registry gets out of sync
<solnic>
as I said, it works with DO adapter because it simply fires a db query and gets data back
<solnic>
blah
<solnic>
hmm
<dkubb>
to change it the other way we would need to make all relations mutable, including Axiom::Relation, for it to be consistent
<solnic>
well
<solnic>
rom's relations aren't mutable
<solnic>
the trick is that we build relations and store them in a registry
<dkubb>
because in Axiom::Relation the #insert and #delete method returns a new relation that represents the old relation plus the new operation applied to it
<solnic>
you access relations through that registry
<dkubb>
what if the registry was more like a proxy to the adapter?
<solnic>
how would that help?
<dkubb>
for each named relation you specify a name and adapter, then when you do registry['users'] it knows to ask memory_adapter['users']
<dkubb>
then you're always getting the relation from the adapter, which is going to return the current relation
<solnic>
yes but there's rom' relation layer with mappers on top of that :)
<dkubb>
hmm
<solnic>
current interface forces you to use returned relations when you want to fetch data that you just inserted
<dkubb>
the mapper is on top of this though, the relation talks to the registry underneath it?
<solnic>
yes rom relation talks to the adapter's relation
<solnic>
so we inside rom relation we do def insert(object); relation.insert(mapper.dump(object); end
<solnic>
when the other relation is adapter's gateway
<solnic>
so, we do return new relation representing state change and if I get data from that relation I will get the inserted objects
<dkubb>
actually, afaik there are no axiom adapters with #insert or #delete
<solnic>
oh hmm
<dkubb>
the only thing with #insert and #delete are Axiom::Relation objects (and their descendants)
<solnic>
omg of course you're right
<dkubb>
I wish I had a reference implementation earlier of course
<solnic>
ok, so we basically need to figure out how to tackle that inside rom-relation
<solnic>
dkubb: is it 4:30am? :)
<dkubb>
it's late tonight so I probably can't be much help, but tomorrow I can discuss it with you and snusnu
<dkubb>
2:30 am
<solnic>
ah ok
<solnic>
dkubb: yes, sorry for keeping you longer
<dkubb>
no, it's fine
<solnic>
dkubb: I will think about this and we can discuss further "tomorrow" (as in later today :)
<dkubb>
you know what's funny, if I had explained the interface better to you, you could've been using Axiom::Relation all along with it's #insert and #delete stuff assuming you had been storing it in your own registry
<dkubb>
this assumes we had figured out how to handle all this earlier
<solnic>
I do use it, see tests in rom-relation :)
<solnic>
or rom-session
<solnic>
I'm worried this interface will be hard to grasp for new comers, but that's ok
<dkubb>
I'll admit it's different
<dkubb>
it *would* be simple to make a wrapper that is mutable that makes it work this way
<solnic>
since you don't have one canonical relation instance that you work with, you can have plenty of instance with various ops applied
<solnic>
I thought your memory adapter would be that mutable wrapper to be honest :)
<solnic>
but as you said, it would not be inline with how axiom relation works
<dkubb>
it is mutable, but it just follows the Axiom::Relation interface
<solnic>
we could still hide it behind rom's interface
<dkubb>
or rather the gateway does
<solnic>
we'll see
<dkubb>
the adapter just has a single thread safe hash
<dkubb>
the other thing about a gateway is that it doesn't always sit at the same "position" within the object graph
<dkubb>
with the memory adapter it basically sits at the leaves, propagating the writes to the adapter
<dkubb>
in an RDBMS adapter it actually sits near the root
<solnic>
yes I noticed that difference
<dkubb>
so if you had it be mutable, two object graphs would actually have different behaviours
<dkubb>
assuming otherwise identical graphs
<solnic>
otoh it just feels weird that you have a relation representing something in the datastore and when you insert something you get a new relation and the first one doesn't include that inserted thing
<dkubb>
every adapter is going to have different behaviour with where the gateway sits. a gateway would basically be as close to the root as possible, meaning that everything between it and the leaves is being handed to the adapter when the operation is performed
<solnic>
how would that work in DO adapter btw?
<solnic>
if you insert something, it will perform an INSERT
<solnic>
so once you read from the same relation you WILL get the data back
<solnic>
which goes back to what I said initially - it doesn't work like memory adapter
<dkubb>
yeah, that's totally a side effect of implementation.. I'm not sure how to resolve it
<solnic>
I'm having hard time imagining a solution :)
<dkubb>
if we can't resolve it then every adapter is going to have different behaviour when you actually insert then read the data back
<dkubb>
which makes me sad
<solnic>
maybe inserting should change the actual relation
<solnic>
I mean, really, that's exactly what I expected from memory adapter
<solnic>
I'm not saying it's better, I dunno if it is
<solnic>
it was natural for me that it should work like that
<solnic>
maybe adapter should be that mutable layer
<solnic>
but yeah, inconsistency worries me
<solnic>
you wouldn't be able to test things using axiom relations in the same way like using adapter gateways
<dkubb>
yeah it would be a critical blocker for me
<dkubb>
we *cannot* have variable behaviour
<solnic>
I agree
<dkubb>
because then what's the point. I could just use ARel underneath everything
<solnic>
but it's hard to imagine working it like that with a real-db adapter
<solnic>
because once you inserted something, it's there
<solnic>
so if you read it again, you will get it back
<dkubb>
the interface totally makes sense if you're used to working with immutable objects
<dkubb>
it's basically the same as what hamster does
<solnic>
but you know
<solnic>
the database is pretty much mutable :)
<solnic>
that's where we're hitting a wall now
<dkubb>
I suppose all datastores are mutable
<dkubb>
well, in the sense that you can add to them and get back a snapshot of the system at a specific point in time
<dkubb>
so if I did make this change, axiom itself would probably propagate writes down to materialized relations
<dkubb>
and there may not even be a need for an axiom-memory-adapter
<solnic>
heh
<solnic>
when I think about it now
<solnic>
I don't understand what's exactly the purpose of memory adapter :)
<solnic>
I could just use axiom relations
<dkubb>
I'd have to be really careful about thread safety because I wrote all that immutable logic stuff with the assumption a relation is going to be global and shared between multiple threads
<dkubb>
I think the thread_safe gem provides a thread safe Array
<dkubb>
I could also do it myself, but I'd rather use something simple to start
<dkubb>
hmm.. I think even if I have a materialized relation become mutable I would still have an axiom-memory-adapter, if only because it provides a place to ask for the named relations you've setup
<dkubb>
it would probably be mostly gutted though
<dkubb>
I like the #[] and #[]= interface on the adapter, because with rdbms or others, it gives a place to retrieve reflected relations from the schema, which I plan to use for migrations as well as bootstrapping
<solnic>
right
<dkubb>
solnic: I'm going to think about this today and a bit over the weekend, but I think I may test this out and see if it works well. I may have to have a special case for an immutable materialized relation (like if an Axiom::Relation is created for a frozen Array or Set), but otherwise I think it would be fairly straight-forward
<solnic>
I was more looking from a rom pov, now memory adapter doesn't give me much
<solnic>
because it works exactly like when we just use Axiom::Relation
<dkubb>
yeah
<dkubb>
and that's why I kind of scratched my head when it was discussed as being critical :)
<dkubb>
but I needed a reference adapter
<solnic>
we assumed mutable behavior (me and snusnu)
<dkubb>
especially because writes haven't been done in any adapters, I needed to know the interface
<solnic>
that piece was missing and we thought memory adapter will provide this
<solnic>
what would be straight-forward? making materialized relations mutable?
<dkubb>
it would be easy to do with a proxy that holds a @relation, and then applies the #insert and #delete operations to it, and updates @relation with the return value
<solnic>
yes I thought memory adapter would do just that :)
<solnic>
so every time I access a relation I get the current state
<dkubb>
it could do @relation = @relation.public_send(method, *args, &block).materialize .. assuming method is either :insert or :delete
<dkubb>
if that's all we needed I could've easily written a shim to allow you guys to continue on with your work
<solnic>
yes please
<solnic>
I think we will have that behavior for datastore adapters anyway
<dkubb>
we can remove the shim later once materialized relations are mutable
<solnic>
I mean, how would it work otherwise?
<solnic>
ok cool
<dkubb>
yeah, the main thing is that Axiom::Relation needs to be updated to have the same interface
<solnic>
let's discuss it a bit more with snusnu too :)
<dkubb>
yeah
<dkubb>
but yeah, a shim would probably be really simple I think
<solnic>
I'm almost certain he had similar expectations though
<solnic>
but I prefer to check it with him anyway
<dkubb>
we can throw it away later
<solnic>
sure, makes perfect sense
<dkubb>
when everything else behaves like it
<solnic>
so that shim would be on top of the memory adapter?
<solnic>
replacing its relations everytime you insert/update/delete right?
<dkubb>
solnic: you could just use it with straight Axiom::Relation objects I think
<snusnu>
solnic: yeah, quickly read through them, sorry for not having more time to comment, i have to leave very soon
<snusnu>
solnic: i wouldn't bother with the different end user api (wrt to ops returning relations) … but i agree that we MUST be consistent for all adapters (memory included, and probably also raw relations
<mbj>
snusnu: Added me to rubygems and permission to release? (client does not accept :git sources)
<snusnu>
mbj: you're a gem owner too
<snusnu>
mbj: also, yeah, you can release whenever you think it fits
<mbj>
snusnu: thx
<mbj>
snusnu: 0.0.9, patch bump
<solnic>
snusnu: well, the only thing that's problematic is that insert/update/delete returns a new relation and if you access the "stored" one in the registry you won't see the changes you made
<snusnu>
mbj: yeah, next up is 0.0.10 tho
<solnic>
what we want, probably, is to actually make the change
<snusnu>
solnic: right, i see that problem
<solnic>
how it will do it under the hood is not important from the ROM user pov
<snusnu>
solnic: i have no really strong opinion i guess, altho i'd like to push the immutability as far as possible .. i was kinda surprised reading that axiom would now need mutable relations
<snusnu>
solnic: i agree
<solnic>
snusnu: dude, I tried to imagine how it would work with any database adapter
<solnic>
and I failed
<solnic>
just think about it
<mbj>
solnic, snusnu: I'd accept we have to do "copy update" style for axiom-api consumption.
<mbj>
My adapters would work here.
<mbj>
(with some refactoring)
<snusnu>
solnic: yeah, i guess i know what you mean .. the fact that any database is some form of persistent collection .. if you change it, next time you ask it, it must be changed, otherwise it wouldn't be persistent :
<solnic>
so basically
<solnic>
env[:users].insert(user) will insert a user into the db
<solnic>
next time you access env[:users] you will see that user in the collection
<solnic>
now, with memory adapter, it won't work like that
<solnic>
because you'd get a new relation with insertion op
<snusnu>
lol, we're leaving to party with a friend who's getting married .. now the guy who picks us up forgot …. *the beer* haha
<snusnu>
i have a few more minutes ...
<solnic>
THAT relation will return the inserted user
<solnic>
original will not
<solnic>
with such interface people will *hate* ROM
<solnic>
:D
<solnic>
snusnu: lol :)
<solnic>
snusnu: ok so let's talk over the weekend, this is pretty important :)
<solnic>
have fun at the party
<mbj>
solnic, snusnu: Before commenting here I'll read the backlog.
<snusnu>
solnic: no dude, i'm leaving *for* the weekend ;)
<solnic>
snusnu: rite
<solnic>
snusnu: so Monday
<mbj>
But I think we can have both. Immutable axiom relations and the ROM interface you described.
<solnic>
oh wait, that'll be hangover
<solnic>
ok, tuesday
<snusnu>
solnic: yeah, in the meantime tho, rest assured that i trust you guys ;)
<snusnu>
haha
<snusnu>
actually monday should be fine ...
<snusnu>
:p
<mbj>
snusnu: have fun!
<snusnu>
thx!
<solnic>
mbj: yes of course it's possible, it's just that we'd like to have consistent behavior for adapters
<solnic>
not just that we want, we MUST have consistent behavior
<snusnu>
solnic: yeah, i guess, initially i thought that we could "wrap" the persistent relations in something on top of the adapter .. but then the adapter would expose a different api .. not good either
<snusnu>
solnic: everything should quack like a relation
<solnic>
snusnu: dkubb said he could built a shim for memory adapter that would expose that "mutable" interface
<solnic>
so basically once you inserted something the new relation will replace the previous one
<solnic>
gtg guys
<snusnu>
solnic: yeah
<solnic>
snusnu: have a great weekend then and talk to you on Monday :)
<snusnu>
solnic: rite, thx!
<snusnu>
mbj: what are your plans with substation? fix specs and kill mutations?
<mbj>
snusnu: fix specs, kill mutations rlease
<snusnu>
mbj: i dunno if i should say it, but i'd somehow like it if all substation releases pass our ci
<snusnu>
mbj: that'd mean at leasts docs for a few things
<snusnu>
mbj: i'm fine with whatever reek exclusions or rubocop issues
<mbj>
snusnu: So you ask me to update docs?
<mbj>
snusnu: I'm fine to do a .beta release :D
<snusnu>
mbj: there are like 5methods that need yard docs, nothing serious
<snusnu>
mbj: hehe
<mbj>
snusnu: Hehe, I'll add yard docs, always.
<snusnu>
mbj: update the old ones, nah, i'll get around to that
<snusnu>
mbj: cool then :)
<mbj>
snusnu: But I'm not able to produce snusnu grade README :D
<snusnu>
mbj: haha
<snusnu>
mbj: if i think about it tho … maybe we're fine releasing without full yardstick cov .. the most important things is mutant obviously
<snusnu>
mbj: do as you please / as time permits
<mbj>
snusnu: I always release with full YARD docs.
<snusnu>
mbj: even better then
<snusnu>
bye guys
snusnu has quit [Quit: Leaving.]
solnic has quit [Quit: Leaving...]
solnic has joined #rom-rb
solnic has quit [Quit: Leaving...]
knowtheo1y has joined #rom-rb
knowtheory has quit [Ping timeout: 276 seconds]
dkubb has joined #rom-rb
<dkubb>
good morning
cored has quit [Ping timeout: 246 seconds]
<mbj>
dkubb: morning
<mbj>
Still not had the time to review the discussions and the adapter.
<mbj>
Only saw solnic and snusnu thinking loud about immutable updates and mutable relations :D
<dkubb>
hehe
<dkubb>
yeah solnic and I had a discussion late last night and I suspect that snusnu and he continued today. I haven't checked the history yet
g0bl1n has joined #rom-rb
zekefast has quit [Quit: Leaving.]
g0bl1n has quit [Ping timeout: 240 seconds]
g0bl1n has joined #rom-rb
g0bl1n has quit [Ping timeout: 240 seconds]
mbj has quit [Ping timeout: 246 seconds]
mbj has joined #rom-rb
postmodern has joined #rom-rb
<mbj>
dkubb: hola, how do you run parameterized rspec tests. For example running twice once with mocked gateway and once with real integrated gateway.
<mbj>
dkubb: I'd hate to use ENV variables for all the options.
<mbj>
I'd love if rspec itself would allow to pass such flags, maybe I just dont know that flag :D
knowtheo1y has quit [Quit: Computer has gone to sleep]