<snusnu>
mbj: it's time to get an idea for a more unified interface to axiom adapters
<snusnu>
mbj: the data_objects adapter exposes #read, the memory adapter not, but #[] instead, etc
<snusnu>
mbj: i hope to be able to push an early spike for axiom-schema later today, that is somewhat "ast based"
<snusnu>
mbj: i.e. i separate the classes from an internal representation that models the participating entities, and allows a builder to construct the proper axiom objects based on those definitions / ast_nodes
<snusnu>
separate the DSL classes i mean
<snusnu>
eventually, an immutable Axiom::Schema object is being built from the ast
<mbj>
snusnu: First we shoud write an axiom-relation to AST converter.
<mbj>
snusnu: And use that relation-ast as primary query interface ;)
<snusnu>
nope we should not, this has absolutely nothing to do with that imo, it would only halt dev for another what, x months?
<mbj>
snusnu: Far easier to handle / introspect / walk than Axiom::Relation trees.
<mbj>
snusnu: Just read your first message
<snusnu>
mbj: remember, i'm simply doing a DSL that allows to nicely define axiom relations
<snusnu>
i don't care about the internals at this point
<mbj>
snusnu: hehe
<snusnu>
also, i said "ast like"
<mbj>
snusnu: I only wrote out what was on my head after reading your first message ;)
<snusnu>
i'm not doing a perfect ast, i'm simply encapsulating info necessary to build the proper axiom objects
<snusnu>
i know ;)
<snusnu>
and in fact, i knew you were going to say what you said ;)
irclogger has joined #rom-rb
<snusnu>
mbj: so, to be more concrete, i'm not building a real ast, i'm building a dsl, that exposes data capsules (after being instance_eval'ed), that can be used to build up a set of Axiom::Relation(::Base) instances, a few Axiom::Schema::Repository instances .. that are then put together to form an immutable Axiom::Schema instance which basically only exposes #[](relation_name)
<snusnu>
mbj: those intermediate data capsules, are what i was referring to as ast .. probably not very wisely
<mbj>
snusnu: AST as IR between DSL and impl ist a good pattern AFAIK.
irclogger has quit [Ping timeout: 245 seconds]
<snusnu>
mbj: yeah, i have that feeling too
<mbj>
snusnu: you know general AST design rules?
<mbj>
snusnu: especially with whitequark/ast ?
<snusnu>
mbj: nope, and i don't really care about it atm, i want to keep it special case first
<mbj>
snusnu: okay
<mbj>
snusnu: I'll review your AST and point you to problems I can identify.
<snusnu>
mbj: sure, that'd be great, we just have to keep in mind that really it's nothing (at this point) that needs to support different kinds of processors
<mbj>
heh
<mbj>
snusnu: I hate DSLs.
<mbj>
snusnu: Also to use them ;)
<snusnu>
i don't, i really don't
<mbj>
snusnu: Totally happy with writing s expression style ;)
<snusnu>
that's no silver bullet tho
<mbj>
no
<mbj>
both no
<mbj>
DSL is not silver bullet, AST neighzter.
<snusnu>
right
<mbj>
Just my personal preference.
<snusnu>
i probably confused things a bit when mentioning the ast .. what i'm really doing, is building an IR that allows a builder to construct axiom objects .. at least atm .. once axiom itself is more ast based .. the approaches may merge
<mbj>
snusnu: jo
<snusnu>
i just want to have no real blockers that would allow for further processing of the IR
<snusnu>
so i don't need it to be a proper ast right now, but i want to separate concerns correctly
<snusnu>
so that later on, a refactoring to a real ast will be easy
<mbj>
jo
dkubb has joined #rom-rb
<snusnu>
hey dkubb
<snusnu>
dkubb: mind reading the last 20min or so of logs? it's only a few lines anyway
<snusnu>
between mbj and i
<snusnu>
dkubb: i've started on axiom-schema locally, and hope to be able to show a first spike later today or tomorrow
<snusnu>
and wanted to know wether you feel i'm working in the right directin
<dkubb>
snusnu: good morning
<dkubb>
snusnu: oh cool. lemme check it out
<dkubb>
snusnu: with axiom-schema, try to use the Axiom::Relation/Axiom::Header objects to describe individual relations if you can..
<snusnu>
dkubb: yes, i do that
<mbj>
dkubb: hi
<dkubb>
if they don't carry enough information for describing one, then we probably need to update them
<snusnu>
dkubb: we definitely should cut out schema definition from rom-relation and make it available for raw axiom relations
<snusnu>
dkubb: hence my work on axiom-schema
<dkubb>
yeah, as much as we can push down the better
<snusnu>
dkubb: in my current code, i build Axiom::Attribute, Axiom::Relation::Header and Axiom::Relation(::Base) instances .. additionally, i've added preliminary "ast nodes" for Constraint::Key::{Primary, Foreign} and Constraint::Unique
<dkubb>
snusnu: so does "ast.base_relation(:people) do" return an actual Axiom::Relation::Base object?
<dkubb>
actually, I guess since it's a command method it would return the ast instance, but I would imagien it creates a base object somewhere
<snusnu>
dkubb: nope, it adds (a somewhat poorly named) Axiom::Schema::AST::Node::Realtion::Base object to the "ast"
<snusnu>
naming's still poor
<mbj>
dkubb: Sidenote, I had a nice discussion with postmodern yesterday.
<dkubb>
ok
<mbj>
dkubb: About how mutant handles explicit returns at the end of a block.
<mbj>
dkubb: there is an ast difference for "def foo; return 1; end" and "def foo; 1; end", but NO semantic difference.
<dkubb>
snusnu: it's quite possible we'll be able to collapse that into an Axiom::Relation::Base at some point, but it's probably a good idea to keep them separate for now while you get the ideas down
<mbj>
dkubb: Its one edge case where the AST is inflected by style.
<mbj>
dkubb: I'm thinking about stripping explicit returns (only at the end of a block) before mutations.
<dkubb>
mbj: yeah, we have a few places we inject our style
<mbj>
dkubb: Because mutant is NOT a style mutator, its a semantic mutator.
<snusnu>
dkubb: yeah, those are my thoughts .. also, since there are still objects missing in axiom (like the constraint stuff) .. i needed a place to put it all together for now .. that can be taken by a builder object to produce the proper axiom object instances
<dkubb>
snusnu: yeah, what you can do is create those objects and get it all working, and then we can push them down further into the core stuff
<snusnu>
exactly
<dkubb>
snusnu: in your foreign_key example, if I use "people.id", but the people relation doesn't exist yet, does it create some kind of placeholder for it anyway?
cored has quit [Ping timeout: 240 seconds]
<snusnu>
dkubb: not yet, and it probably won't for a while, as this complicates the code (because we need futures)
<snusnu>
dkubb: it can and should be added eventually tho
<snusnu>
dkubb: i was even thinking of starting out with "external" constraint definition, because it's simpler .. but it's too "ugly"
<snusnu>
dkubb: so i thought, make it work nicely *if* all prerequisites are met
<dkubb>
snusnu: btw, generally we would call something a "key" not a "primary key". a "primary key" is an arbitrary distinction for a unique key. I would generally collapse primary_key and unique into a single key() method, and have a primary option, eg: key :id, primary: true
cored has joined #rom-rb
<snusnu>
dkubb: well, hmm, for a proper ast, we would have to make the distinction in the objects we create anyway .. and i'm not too sure if i like :primary as an option .. accepting a hash of options, with only one allowed key, always feels weird
<snusnu>
also, primary_key and foreign_key have the same number of chars ;)
<dkubb>
snusnu: there are 3 kinds of constraints at the attribute, relation and database level .. and technically you can represent any constraint "below" a level you want. so you can have a database level constraint that applies to a single attribute. you generally want to push them down if you can
<dkubb>
snusnu: yeah, but primary_key and unique are actually exactly the same thing in the relational model
<dkubb>
snusnu: I'm talking about the proper model, not the RDBMS one that we've all been infected with
<snusnu>
dkubb: i know, but when serialized to DDL they differ, and that's where we eventually want to go with that ast?
<snusnu>
yeah
<snusnu>
dkubb: i do agree tho, that the DSL shouldn't be considered final at all at this stage .. i'm not at all against changing it to where we find consensus
<dkubb>
snusnu: yeah, a key being primary is completely arbitrary. it has no real significance to the model
<snusnu>
dkubb: thinking more about it, i might just accept that :primary option and have #key
<dkubb>
snusnu: I would remove primary_key and unique and just have key
<snusnu>
fair enough
<dkubb>
also, keep in mind a key can be multi-attribute
<snusnu>
code's going to be a bit more ugly, but i can live with that
<snusnu>
yeah
<dkubb>
so can a foreign key
<snusnu>
i know, i still need to account for that
cored has quit [Ping timeout: 272 seconds]
cored has joined #rom-rb
<dkubb>
yeah, you may be repeating information if you do: foreign_key :people_name, :people_number, people.name, people.number
<dkubb>
since it specifies people.* twice
<dkubb>
I would be more likely to use a Hash for that
<snusnu>
yeah, i remember thinking about that for a while "back in the days" … i need to refresh my thoughts and code snippets on that topic
<dkubb>
it's still duplication, but at least you don't have to worry about positional args
<snusnu>
yeah, iirc i have a dsl spike that uses a hash lying around somewhere
<dkubb>
it wouldn't be wrong to use the term "database" instead of "repository" though. we got hung up with using repository() in DM1, but database is true, as long as we make it clear we're not talking about RDBMS specific databases
vsorlov has quit [Ping timeout: 246 seconds]
<snusnu>
dkubb: yeah, i like that
<snusnu>
dkubb: while getting closer to database .. wdyt can/should be done wrt adapter api unification?
<dkubb>
with DM1 it's almost like we got scared of calling ti a "database"
<snusnu>
dkubb: currently the do adapter exposes #read and the memory one #[]
<snusnu>
heh true
<dkubb>
the #read and #[] methods are different
<mbj>
dkubb, snusnu: I dont have time to follow discussion, will catch up later.
<snusnu>
i know
<dkubb>
#[] and #[]= allow you to get/set the base relations in the "schema"
<snusnu>
yeah, and i don't really like the "set" part
<snusnu>
the adapter could be initialized with all the (base) relations it handles?
<dkubb>
when you update a relation, you're effectively replacing one relation with another
<snusnu>
yeah
<dkubb>
how it's implemented is different
<snusnu>
the relvar, right
<dkubb>
yeah
<snusnu>
dkubb: i'm currently facing the "problem" that repositories/databases encapsulate their own set of relations (at least the memory one), and the "unified" schema spans n of those
<dkubb>
#read is more about giving a non-materialized relation to the adapter and saying "yield each tuple for this relation from the database"
<snusnu>
dkubb: so i need a unified api to access one particular base relation, no matter the adapter
<dkubb>
what I would probably do is implement a database-level object for tracking all relations in a specific database for a specific adapter, and then have a system-level object that simply proxies to the database level objects based on the names
<snusnu>
dkubb: i could of course special case, but that'd be ugly and involve otherwise unneeded repository/database subclasses
<dkubb>
unless we want to do cross-database foreign keys, which I guess we could.. hmm
<snusnu>
and should ;)
<snusnu>
as FKs will form the base for cross repository join (sugar)
<snusnu>
and in general, relationships
<snusnu>
the way i see it, "all that is needed" for cross repo joins, is meta info for "on what attribute" to join .. the join itself will have to happen in memory anyway
<snusnu>
thus the FK can be purely "logical", nowhere to be persisted/serialized
<snusnu>
s/FK/FK constraint/g
<dkubb>
we should have the same convention as we did in dm-constraints, a FK should default to "restrict"
<dkubb>
I got bitten in AR a few weeks ago by foreigner, since it doesn't default to anything strong
<snusnu>
absolutely, that's the only feasible default
<dkubb>
so I had to redo all my FKs to be strong because they were merely annotions otherwise
<snusnu>
and with most rdbms, it is?
<dkubb>
I don't think it is. it's dumb though
<snusnu>
meh, yeah
<dkubb>
snusnu: I'm not sure how to represent constraints yet though. in axiom-types, you can represent a constraint as a value that has predicates applied to it. attribute-level constraints could (should) be represented as a type, while relation and database constraints should be normal predicates from axiom
<dkubb>
snusnu: if we use the existing predicates then we can use the existing SQL generation code to serialize them to read CHECK constraints should we want to
<snusnu>
dkubb: yeah, we should definitely work towards that, atm, the DSL is only there to be able to describe the relations as completely as possible, and to have all the knowledge needed to serialize (and ideally *validate*) the constraints at some point
<snusnu>
dkubb: i fully expect that actual FK support will still take a while, that's fine
<snusnu>
dkubb: for now, i just want a schema i can use with rom, and work with the ducktrap mapper
<dkubb>
yeah
<dkubb>
we already also mostly know what we need for the future, and it is unlikely to change too much
<dkubb>
the relational model has been mostly unchanged in the last 30-40 years
<dkubb>
I've always wanted to follow the model, and then figure out how to adapt it to each database vendor's system
<dkubb>
snusnu: also, I probably wouldn't bother with the distinction base_relation() in the DSL.. just have it be ast.relation() we can figure out whether to use an Axiom::Relation::Base or Axiom::Relation internally
<dkubb>
from the pov of the user, they shouldn't care too much what's a base relation and what's a "view".. they should be as close as possible, and ideally completely indistinguishable
zekefast has quit [Quit: Leaving.]
<snusnu>
dkubb: a base relations must be tied to a repository tho, no?
<snusnu>
dkubb: and a virtual one can't
<snusnu>
dkubb: that at least was my initial motivation .. make the #repository method only available for base relations, as it doesn't make sense in the virtual one
<snusnu>
dkubb: also, i like the somewhat "established" distinction between tables and views
<snusnu>
dkubb: in general, i took the approach that lead to as few as possible ambiguities (in the DSL code) .. preferring an explicit name/method over conditionals within the method body
<snusnu>
dkubb: (hence the initial primary_key/unique distinction)
<snusnu>
dkubb: also, only base relations need to expose a "relation builder dsl" … virtual ones don't need any of that, they simply are constructed by applying RA ops to other (base)relations
<snusnu>
dkubb: i'd rather have people remember 2 method names and then expose only stuff that matters in the respective contexts
<snusnu>
dkubb: as compared to making all DSL available, only to have people apply them wrong'ish
<cored>
howdy
<dkubb>
snusnu: ok, you changed my mind. I agree with you. as long as the methods available within the blocks differ to make it harder to use them wrong
<snusnu>
dkubb: yeah, the goal is to expose only methods that make sense in the context, nothing else
<snusnu>
and hey cored :)
<dkubb>
cored: hello
<dkubb>
snusnu: also, it's just me, but instead of ast = Axiom::Schema::AST.new I would probably (eventually) want schema = Axiom::Schema.new, and then use schema.base_relation(:people) do, etc
<snusnu>
dkubb: i agree, but that's a matter of naming .. i want the eventual Axiom::Schema object to be immutable, constructed using the info stored in what i currently call ast
<snusnu>
dkubb: there will surely be a Axiom::Schema.build method tho, that wraps up this construction in one step
<dkubb>
yeah, good idea
<snusnu>
dkubb: i wanted to make sure, that the definition of the schema can be split into multiple files tho
<snusnu>
dkubb: but once the app "booted/got required" i want an immutable schema
<cored>
hi dkubb snusnu
<cored>
snusnu: can you check out my PR for devtools, I would like to finish it by the end of this week
<cored>
probably I need more guidence to figure out what to fix/add next
<dkubb>
snusnu: it's hard to get away from the concept of finalization
<snusnu>
cored: i can do that, yes, but only later today evening
<snusnu>
dkubb: i know right? ;)
<snusnu>
dkubb: but we should push as far as we can
<snusnu>
dkubb: the way i currently see it, finalization is only really needed when dealing with circular references or when "forward declarations" should be avoided (for aesthetical reasons)
<snusnu>
dkubb: that mean that eventually, we'll probably need it, but yeah, let's go as far as possible without .. and then look at the tradeoffs
<snusnu>
dkubb: as a side effect, it also forces you to restrict mutability to very few places
<snusnu>
dkubb: so once finalization/futures get added, the impact on where the mutability is, is kept low
irclogger has joined #rom-rb
irclogger has quit [Ping timeout: 272 seconds]
<cored>
snusnu: please do, and leave me some comments/tasks in the same PR so I can follow
<snusnu>
cored: will do so
<cored>
thanks
snusnu has quit [Quit: Leaving.]
<mbj>
dkubb: I think its possible to get a way the finalization. At the price the DSL must be rethought.
snusnu has joined #rom-rb
<dkubb>
mbj: somewhere, something needs to be mutable, but I think it may be possible to push that to something really tiny
<mbj>
dkubb: So its basically a graph.
<mbj>
dkubb: If you wanna connect two nodes you need the nodes prior to the edge.
<mbj>
dkubb: A DSL like
<mbj>
graph = Graph.build do
<mbj>
node :a
<mbj>
node :b
<mbj>
connect :a, :b
<mbj>
end
<mbj>
works without any finalization. Because you dont need the concept of a future entity.
<mbj>
So maybe we should define relations without foreign keys until we have all relevant ones, and than allow later connection.
<mbj>
This way we could build that thing in full functional style.
<dkubb>
by putting them in a block you can defer evaluation
<mbj>
yeah, but even than. Getting it correct with "late connection" is more easy.
<mbj>
We can always add more sophisticated DSL later.
<mbj>
Because we can easily add an fk to a relation. later.
<mbj>
Not with an RA operation.
<mbj>
Just rebuild the relation with FK added.
<mbj>
dkubb: Internally we'd have a builder that returns a "full build" schema each time.
<mbj>
dkubb: And some methods returning a new schema with the new node / edge each time you call such a method.
<mbj>
dkubb: Pure functional.
<dkubb>
there's still the need for a mutable registry, but it can be isolated
<mbj>
why, the schema could be the registry?
<mbj>
I think we have a naming problem.
<dkubb>
well, you can replace the registry with a new registry, but the pointer to it needs to be mutable
<mbj>
okay
<mbj>
yeah
<dkubb>
for me a registry is just a name mapping to some object
<mbj>
But the pointer is the "very tiniest" part that is mutable ;)
<dkubb>
you can minimize the mutability, but at some point you need something that points to an object, and you sometimes need to replace the object it points to with an updated version of it
<dkubb>
yeah, exactly
<dkubb>
and I'm fine with that
<mbj>
Also if you collect an AST from the above dsl you can also walk this DSL while building the schema. And having that pointer in an lvar ;)
<dkubb>
you minimize the place where something can go wrong
<mbj>
dkubb: I do this with my ducktrap evaluators.
tpitale has joined #rom-rb
<dkubb>
you can also use optimizers on these structures to collapse things when the history isn't relevant
<mbj>
yeah
<mbj>
dkubb: Did I mentioned my morpher spike has evaluation with history and evaluation without history.
<tpitale>
I asked a question over in #datamapper about data_objects and jruby. Okay for me to ask that question in this room?
<mbj>
tpitale: Lets say it this way you attention, switching to #datamapper
<mbj>
tpitale: my english sorry ;) You got a reply.
<tpitale>
No worries, thanks for the reply!
<dkubb>
oh, I forgot to mention I successfully replaced the memoization code in adamantium with memoizable
<dkubb>
the extraction is almost complete
<dkubb>
so when this is done the memoization code is on-part with adamantium's memoization, allowing you to use memoizable anywhere you like, even outside immutable objects
<dkubb>
although, obviously, be careful memoizing methods that depend on mutable state
<mbj>
dkubb: I saw it. Nice work.
snusnu has quit [Quit: Leaving.]
irclogger has joined #rom-rb
snusnu1 has joined #rom-rb
cored has quit [Ping timeout: 246 seconds]
cored has joined #rom-rb
irclogger has quit [Ping timeout: 246 seconds]
postmodern has joined #rom-rb
snusnu1 has quit [Quit: Leaving.]
jgaskins has quit [Quit: This computer has gone to sleep]