orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Ping timeout: 265 seconds]
orivej has joined #picolisp
_whitelogger has joined #picolisp
orivej has quit [Read error: Connection reset by peer]
orivej_ has joined #picolisp
mtsd has joined #picolisp
orivej_ has quit [Ping timeout: 256 seconds]
orivej has joined #picolisp
orivej has quit [Ping timeout: 258 seconds]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Ping timeout: 260 seconds]
orivej has joined #picolisp
orivej has quit [Ping timeout: 264 seconds]
orivej_ has joined #picolisp
orivej_ has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Ping timeout: 264 seconds]
orivej_ has joined #picolisp
<beneroth>
hi all
<beneroth>
Regenaxer, I actually have a use for (+Ref +List +Symbol) :-)
<Regenaxer>
Hi beneroth
<Regenaxer>
ok :)
<beneroth>
but had to figure out that (collect 'foo '+Bar '((Bla Blubb))) works, while (collect 'foo '+Bar '(Bla Blubb)) doesn't
<beneroth>
because collect uses (pair) :-)
<Regenaxer>
hmm
<beneroth>
(if (pair X1)
<beneroth>
(setq X2 (append X2 T))
<beneroth>
(setq X1 (cons X1) X2 (cons X2 T)) )
<Regenaxer>
true, it is a bit unorthogonal
<beneroth>
from collect source
<Regenaxer>
the pilog funs accept pairs as ranges
<Regenaxer>
collect takes a from-till
<beneroth>
aye
<beneroth>
same with (iter)
<Regenaxer>
yes
<beneroth>
another funny one.. if I remember correctly, (+Ref +Joint) is not really working. But apparently in an older project I did (+Ref +Joint +Link) and it works beautifully xD
<beneroth>
I'm not sure if I did that on purpose or accidentally.. the result (having an indexed joint) was certainly desired
<Regenaxer>
I would say (+Ref +Joint) is not intended, so not really supported
<Regenaxer>
I never tried
<Regenaxer>
But there is no reason why it should not work
<Regenaxer>
it is a subclass of +Link
<beneroth>
T
<Regenaxer>
Where is an indexed +Joint useful?
<Regenaxer>
Looking up the index with the object is more expensive than using the object directly with 'get'
<beneroth>
aye, but what if you don't have the object
<beneroth>
once per index, once per object, different access paths
<Regenaxer>
Then you cannot index
<Regenaxer>
you cannot make a range of objects
<Regenaxer>
So you can only traverse the *whole* index
miskatonic has joined #picolisp
<beneroth>
it was something like (class +Foo) (rel bars (+List +Joint) foo (+Bar)) and (class +Bar) (rel foo (+Ref +Joint +Link) NIL bars (+Foo))
<beneroth>
to be capable to get a list of +Bar sorted by 'foo
<beneroth>
directly, without iterating over any +Foo
<Regenaxer>
But the sort is useless
<Regenaxer>
it is object block addresses
<beneroth>
T
<Regenaxer>
For me (+Ref +Link) and (+List +Joint) are equivalent
<beneroth>
I try to grok the thinking for that project again
<Regenaxer>
:)
<beneroth>
+Bar had no other index, so this was used to get all +Bar, I think....
<Regenaxer>
Anyway good to know that even such unexpected class combinations work :)
<beneroth>
yeah you are right
<beneroth>
(of course)
<beneroth>
or at least I cannot find the sense in it anymore
<Regenaxer>
:)
<beneroth>
(except for getting around loading of the object..but.. that... is not a good reason)
<Regenaxer>
true, that would be a reason
<Regenaxer>
loading the index is more expensive though
<beneroth>
probably
<beneroth>
multiple blocks
<Regenaxer>
yeah, depends
<beneroth>
if loading the object is so expensive, then it's time to learn about +Swap
<Regenaxer>
indeed
orivej has joined #picolisp
orivej_ has quit [Ping timeout: 256 seconds]
orivej has quit [Ping timeout: 258 seconds]
orivej has joined #picolisp
<rob_w>
good morning chan
<Regenaxer>
Hi rob_w
<beneroth>
hi rob_w
<rob_w>
hey guys
orivej has quit [Ping timeout: 265 seconds]
orivej_ has joined #picolisp
orivej_ has quit [Ping timeout: 246 seconds]
orivej has joined #picolisp
<aw->
hi rob_w Regenaxer beneroth
<razzy>
hi all
<beneroth>
hey aw-, razzy
<Regenaxer>
Hi aw- & razzy
orivej has quit [Ping timeout: 258 seconds]
<aw->
i will release a new OSS project
<aw->
today or tomorrow, just finishing up the README now ;)
<beneroth>
yay, exciting
<beneroth>
what is the topic?
orivej has joined #picolisp
<aw->
secret ;)
orivej has quit [Ping timeout: 260 seconds]
orivej has joined #picolisp
<Regenaxer>
:)
orivej has quit [Ping timeout: 260 seconds]
orivej_ has joined #picolisp
orivej_ has quit [Ping timeout: 240 seconds]
orivej has joined #picolisp
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
mtsd has quit [Quit: Leaving]
<razzy>
aw- will it be picolisp project?
orivej_ has joined #picolisp
orivej has quit [Ping timeout: 246 seconds]
orivej_ has quit [Ping timeout: 256 seconds]
orivej has joined #picolisp
orivej has quit [Ping timeout: 260 seconds]
orivej has joined #picolisp
orivej has quit [Ping timeout: 258 seconds]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej_ has joined #picolisp
orivej has quit [Ping timeout: 265 seconds]
orivej_ has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
razzy has quit [Quit: Connection closed]
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Ping timeout: 240 seconds]
orivej has joined #picolisp
miskatonic has quit [Quit: ERC (IRC client for Emacs 24.5.1)]
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
<beneroth>
Regenaxer, btw. how would you do a read transaction? I mean like (dbSync) (rollback) but without rollback issueing 'wipe
orivej_ has joined #picolisp
orivej has quit [Ping timeout: 264 seconds]
<Regenaxer>
A read transaction does not need a sync
<Regenaxer>
What is the purpose?
<beneroth>
multiple objects
<beneroth>
a clean read. otherwise in the middle another process makes a write to objects I haven't read yet
<Regenaxer>
You have almost always multiple objects
<beneroth>
T
<Regenaxer>
The point is that the writer cannot start to write as long as somebody is reading
<Regenaxer>
Reading makes a read lock
<beneroth>
for how long?
<Regenaxer>
Only for the moment of the read
<Regenaxer>
But when a writer starts to write, read is impossible
<Regenaxer>
i.e. you get no read lock
<Regenaxer>
So the sync is not necessary
<Regenaxer>
or, better, it *is* synced
<beneroth>
yeah, so (db .. '+Foo) (db ... '+Bar)... if the other process happens to write in between this too, I get an updated +Bar without an updated +Foo, no?
<Regenaxer>
Yes, possible
<Regenaxer>
But then the other made a bad transaction
<Regenaxer>
when writing
<Regenaxer>
If foo and bar depend on each other, the *write* needs to sync the transaction
<Regenaxer>
*writer*
<beneroth>
yes, but even when the writer does that
<beneroth>
the (tell) is not read in our process
<beneroth>
or?
<Regenaxer>
It is read
<beneroth>
when? where is the (wait) ?
<beneroth>
during disk IO?
<Regenaxer>
It is all in waitFd
<beneroth>
ah so during (db ... '+Bar) ?
<Regenaxer>
Speaker, Hear, Talking etc.
<Regenaxer>
no, db is just a read
orivej_ has quit [Ping timeout: 256 seconds]
<Regenaxer>
it does no telling
<beneroth>
then where is the (wait) in our process?
<beneroth>
when we just do a long list of seemingly disconnected (db) s
orivej has joined #picolisp
<Regenaxer>
Reading always gives you *some* state between writing transactions
<Regenaxer>
You can't assume anything else
<Regenaxer>
The writer must put the DB into a consistent state
<beneroth>
yes, and lets assume the writer does so
<beneroth>
can a parallel reader expect sets of objects to be consistent without any (wait) ?
<Regenaxer>
What do you mean with (wait)?
<Regenaxer>
The Lisp level wait function?
<Regenaxer>
The reader does not need to call (wait)
<beneroth>
yes, or a implicit wait which also processes the (commit 'upd) send from other process
<Regenaxer>
He does not even need to be a family member
<Regenaxer>
ah, sorry, no!
<Regenaxer>
Must be a family member of course to get the tell messages
<beneroth>
yep
<Regenaxer>
But the reading itself does not need it
<beneroth>
and must process the tell messages
<Regenaxer>
It is read locked with fcntl()
<Regenaxer>
Yes
<beneroth>
does it process tell messages when reading database files (implicitly) ?
<beneroth>
I'm currently holding the believe (!) that it doesn't
<Regenaxer>
Not while reading
<Regenaxer>
But the read blocks
<beneroth>
yeah so if we have process R: and process W:, and we do the following steps: 1) R: (db .. '+Foo) -> SomeFoo 2) W: (dbSync) (put> 'SomeFoo ..) (put> SomeBar ..) (commit 'upd) 3) (db ... '+Bar) -> SomeBar
<beneroth>
3) being in the Reader process R:
<Regenaxer>
If the reader did not call wait, it is ok
<beneroth>
doesn't than Reader have SomeFoo WITHOUT updates from Writer in its memory and a SomeBar WITH updates from Writer?
<Regenaxer>
Then he has some older (consistent) state
<beneroth>
exactly
<Regenaxer>
The point is that he never reads an inconsistent state
<Regenaxer>
Of course each process may lag behind
<beneroth>
per object, but isn't it inconsistent when looking at all objects (blocks) ?
<Regenaxer>
But eventually it will be synced
<beneroth>
in my example
<beneroth>
doesn't Reader hold SomeFoo without updates from Writer, and hold a SomeBar with updates from Writer?
<beneroth>
until Reader does do (wait) or something which implicitly does (wait) ?
<beneroth>
then SomeFoo gets wiped and refreshed on next access
<beneroth>
Regenaxer, do you see my example or should I put it in code?
<Regenaxer>
SomeFoo was valid before the sync
<Regenaxer>
No, it is fine
<beneroth>
in most cases this would be no problem, but in some cases (application-specific) you want to be sure that you have the most recent version of both SomeFoo and SomeBar
<beneroth>
the way to do that would be (dbSync) ... (commit)/(rollback) even when in this process no writing happened, no?
<Regenaxer>
I'm not sure atm
<Regenaxer>
When R goes into (wait) later, he will receive all updates
<beneroth>
yes, but not before that
<beneroth>
basically what we want
<beneroth>
is a writelock without wanting to write anything
<Regenaxer>
I'm not sure if I remember all details atm
<Regenaxer>
You can (dbSync) and then (commit)
<beneroth>
yes
<Regenaxer>
It will write nothing if nothing is dirty
<beneroth>
yes
<Regenaxer>
a bit heavy though
<beneroth>
yes
<beneroth>
this is a major difference to how other database systems usually do it.. which is legitimate
<Regenaxer>
I think (commit) also would work only for reading a single object
<Regenaxer>
Object read unlocks after reading an object
orivej has quit [Quit: No Ping reply in 180 seconds.]
<beneroth>
hm
<beneroth>
I come to the conclusion (dbSync) ... (commit 'upd) is the right thing, even for that here
<Regenaxer>
hmm, I think it is not enough
<beneroth>
how so?
orivej has joined #picolisp
<Regenaxer>
Usually you want to read several objects
<beneroth>
it is costly, would be nice if there would a cheaper way... but should be enough
<Regenaxer>
SomeFoo AND SomeBar
<beneroth>
yes
<beneroth>
so you do (dbSync) (db...) -> SomeFoo, (db ..) -> SomeBar ... (commit 'upd)
<Regenaxer>
But the DB machinery readlocks AND unlocks for each object
<Regenaxer>
Each (db ..))
<beneroth>
yes
<beneroth>
the read locks are separate
<Regenaxer>
Yes
<Regenaxer>
So in between some other process can grab the db, no?
<beneroth>
?
<Regenaxer>
Not sure atm :)
<beneroth>
this is a major difference to most other database systems.. which is basically a consequence of pilDB doing in-place updates while others don't
<beneroth>
they use a WAL, which picolisp also have, but only for transaction integrity or replica journaling
<Regenaxer>
It would require major rewrite of the machinery
<beneroth>
reads in other systems are then based on WAL, this way they can read a snapshot over multiple objects consistently without blocking parallel writes
<beneroth>
yes. and the pilDB approach is not wrong, just different, with other advantages
<beneroth>
it's certainly much simpler
<Regenaxer>
It assumes some cooperative behavior
<beneroth>
aye
<beneroth>
and puts a limit on parallelism because all transactions are serial
<beneroth>
real serial
<beneroth>
which is the strongest and best isolation
<Regenaxer>
Read transactions are fully parallel
<Regenaxer>
Only when *one* process writes all other reads and writes are blocked
<beneroth>
what is a read transaction? you mean a database file access, practically, no?
<Regenaxer>
writes == commit
<beneroth>
yeah
<Regenaxer>
Yes, db read
<Regenaxer>
(val Obj)
<beneroth>
different layers of concern, I would say
<beneroth>
database file: read transaction as you name it, and blockage of concurrent writes
<beneroth>
logical data level: (dbSync) ... (commit)
<Regenaxer>
Yes, but only while (commit) runds
<beneroth>
a read transaction on the "logical data level" (my term, not a real term afaik)
<Regenaxer>
After (dbSync) reading continues
<beneroth>
yes but parallel writes are blocked
<Regenaxer>
Yes
<beneroth>
so nothing can change between multiple reads
<Regenaxer>
yes
<Regenaxer>
After (dbSync) reading means: Reading blocks and receiving modifications from other processes
<beneroth>
well strictly we only receive modifications within the (dbSync) call, right? it's just there will be no new ones issued until we remove our (lock)
<Regenaxer>
We receive them also in other (wait) calls
<Regenaxer>
i.e. when idle
<Regenaxer>
Well, not really "receive"
<Regenaxer>
just (wipe)d
<beneroth>
so theoretically we could also just at the beginning lock all the objects we want to work with, which would prevent parallel writes on those objects but would allow parallel writes to unrelated objects?
<Regenaxer>
Yes, this is fine
<beneroth>
still would need a way to find out all the involved objects, and all processes would need to lock on object levels instead of global for this to work
<Regenaxer>
(lock Obj)
<Regenaxer>
right, this is not trivial
<beneroth>
ok
<beneroth>
you confirmed my understanding
<Regenaxer>
as tree changes may involve many objects
<beneroth>
I hope I haven't confused you too much
<beneroth>
T
<Regenaxer>
Nono, good questions
<beneroth>
I need to go
<beneroth>
dinner is ready!
<Regenaxer>
So, in summary, (dbSync) .. (commit 'upd) makes sense, but a bit overhead of nothing was modified
<beneroth>
thanks for the discussion :-)
<beneroth>
T
<Regenaxer>
Thanks too! :)
<beneroth>
but in some cases a necessary overhead!
<Regenaxer>
I think typically is that you modify something if you do such business
<beneroth>
to ensure total consistency over multiple objects we work with, when we need that
<beneroth>
T
<Regenaxer>
Otherwise, just idling, things change all the time anyway
<beneroth>
but e.g. creating a report
<Regenaxer>
OK, enjoy dinner! :)
<beneroth>
a large report
<Regenaxer>
Yes
<beneroth>
which must be internally consistent
<beneroth>
then you have to block writes to anything involved
<Regenaxer>
right
<beneroth>
okay
<beneroth>
thanks
<beneroth>
:)
<beneroth>
afk
orivej has quit [Quit: orivej]
razzy has joined #picolisp
freemint has joined #picolisp
razzy has quit [Quit: Connection closed]
orivej has joined #picolisp
freemint has quit [Ping timeout: 260 seconds]
rob_w has quit [Quit: Leaving]
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
freemint has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 210 seconds.]
orivej has joined #picolisp
orivej_ has joined #picolisp
orivej has quit [Ping timeout: 240 seconds]
orivej has joined #picolisp
orivej_ has quit [Ping timeout: 264 seconds]
orivej has quit [Ping timeout: 264 seconds]
orivej has joined #picolisp
miskatonic has joined #picolisp
miskatonic has quit [Quit: ERC (IRC client for Emacs 24.5.1)]
orivej has quit [Read error: Connection reset by peer]