ChanServ changed the topic of #picolisp to: PicoLisp language | Channel Log: https://irclog.whitequark.org/picolisp/ | Picolisp latest found at http://www.software-lab.de/down.html | check also http://www.picolisp.com for more information
_whitelogger has joined #picolisp
andyjpb has quit [Ping timeout: 264 seconds]
karswell has joined #picolisp
karswell has quit [Remote host closed the connection]
karswell has joined #picolisp
karswell has quit [Remote host closed the connection]
karswell has joined #picolisp
karswell_ has joined #picolisp
karswell has quit [Ping timeout: 265 seconds]
karswell_ is now known as karswell
karswell has quit [Remote host closed the connection]
karswell has joined #picolisp
_whitelogger has joined #picolisp
<aw-> hi all
<Regenaxer> Hi aw-!
<aw-> i'm a bit confused about the "Persistence (external symbols)" doc
<aw-> it shows how to (put) stuff in the *DB
<aw-> but it doesn't show how to retrieve them
<aw-> other than using (show)
<aw-> I figured i need (get) to retrieve, but the syntax is unclear
<Regenaxer> There is no explicit retrieval
<Regenaxer> Just use it
<Regenaxer> One level above the raw external symbols is the E/R level
<Regenaxer> There you have classes, and indexes to find things
<aw-> ?
<aw-> i have no idea what that means
<aw-> i (put) a few things in the DB
<aw-> how do I retrieve them?
<aw-> ex: (put *DB 'a 1) (put *DB 'b 2) ... how do I get the value 2 from 'b ?
<aw-> (get *DB 'b) doesn't work
<Regenaxer> You always know the root object *DB == {1}
<Regenaxer> yes
<Regenaxer> the put's and get's look fine
<Regenaxer> you need (commit) before you close the session
<aw-> yes actually i'm using (put!)
<Regenaxer> (put *DB 'a 1) is the same as (put '{1} 'a 1)
<Regenaxer> ah, ok
<Regenaxer> So it is all correct
<aw-> hmmm.. ok i must be doing something wrong
<aw-> one sec
<Regenaxer> What means "(get *DB 'b) doesn't work" ?
<Regenaxer> gives NIL?
<aw-> ok it works
<aw-> I was doing (get *DB (pack Var1 Var2)) .. doesnt work
<Regenaxer> yes, needs internal symbols
<aw-> yeah i saw
<Regenaxer> pointer equality in 'get'
yunfan has quit [Ping timeout: 248 seconds]
yunfan has joined #picolisp
<aw-> hmmm
<aw-> (unless (get *DB 'a) ...) doesn't work either
<aw-> weird
<Regenaxer> Must work
<aw-> hmmm
<aw-> this doesn't work with strings?
<aw-> ex, if a property is "a" instead of 'a ?
<Regenaxer> As I said above, properties in general *should* be internal symbols
<aw-> i need to intern each property?
<aw-> why?
<Regenaxer> transients work, but only in the same scope
<Regenaxer> pointer equality in 'get'
<Regenaxer> on pil64 also short numbers work, they are also "pointer equal"
<Regenaxer> I usually avoid numbers as properties though
<aw-> ok so this means, if I intern the symbols then i can't have duplicates ?
<Regenaxer> How do you mean that?
<aw-> in normal databases you can (optionally) have multiple keys with the same name
<Regenaxer> You mean non-uniqe indexes?
<Regenaxer> Nothing to do with properties of symbols
<Regenaxer> You can't have 2 entries in a C struct with the same name either
<Regenaxer> properties are like structs in some sense
<Regenaxer> I think you misunderstood something basic about pil DBs
<Regenaxer> The properties are not to *search* the DB
<Regenaxer> they are attributes of objects
<aw-> oh ok! i see
<aw-> yes i don't know about pil DBs.. first time using
<Regenaxer> good, take time :)
<aw-> hmmm
<aw-> ok so if they are not to search the DB, how do I search the DB?
<Regenaxer> I think you should read the tutorial
<aw-> i did
<Regenaxer> and doc/select.html
<Regenaxer> select is the main workhorse of searching
<Regenaxer> But you find also objects with (db 'var '+Cls ...)
<Regenaxer> or (collect 'var '+Cls ...)
<Regenaxer> or via direct +Link's and +Joint's between objects
<Regenaxer> There is also 'init' + 'step', and 'iter' and 'scan' on the lower levels (direct B-Tree access)
<aw-> i don't see why i can't use the properties of a symbol for searching.. is it slow?
<Regenaxer> no, the fastest
<Regenaxer> +Link's and +Joint's use them
<aw-> if I do: (put *DB 'a 1) (put *DB 'b 2) (put *DB 'c 3) .. what's wrong with using (get *DB 'c) ?
<aw-> you said: "Regenaxer: The properties are not to *search* the DB"
<Regenaxer> Sigh, it *does* work
<Regenaxer> this is not a search but a direct access
<Regenaxer> Searching means to find an object by the *value* of its property
<aw-> oh.. no i'm not doing that
<aw-> i'm "searching" by property name
<Regenaxer> *DB is known, no need to search for it
<aw-> would be it be inefficient to have say... 1000 symbols in my DB? better to have 1 symbol with 1000 properties?
<Regenaxer> 1000 is nothing
<Regenaxer> talk about 1000000000
<Regenaxer> Ah, 1000 properties?
<Regenaxer> That's bad
<Regenaxer> They are searched linearly
<Regenaxer> It means an *object* with 1000 attributes, not a good idea
<aw-> yes 1000
<aw-> interesting..
<Regenaxer> But a typical application has millions of objects typically
<Regenaxer> Thats is no problem
<aw-> ok so (put *DB 'abcdef 1000) <-- (1000th entry) would be bad?
<aw-> what do you mean by "objects" and "attributes" ?
<aw-> these are not the words in the doc
<Regenaxer> "objects" = "symbols" in pil
<Regenaxer> and "attributes" are properties
<Regenaxer> Both words are used I think
<aw-> so 1000 objects = OK, 1000 attributes = NO
<Regenaxer> A symbol is called an "object" if it has classes
<Regenaxer> yep
<Regenaxer> You can try
<aw-> ok so back to my example
<aw-> ok so (put *DB 'abcdef 1000) <-- (1000th entry) would be bad?
<Regenaxer> such an object would just be big
<Regenaxer> takes time and space to load into memory
<aw-> hmmm
<Regenaxer> I don't understand: (put *DB 'abcdef 1000) <-- (1000th entry)
<aw-> but i thought interning many symbols is considered bad in pil
<Regenaxer> *external* symbols
<Regenaxer> not internal
<aw-> well.. that's ripe for confusion .. it's not obvious what is the meaning of "interning external symbols"
<Regenaxer> You should forget about "(put *DB" something
<aw-> ahahaha
<Regenaxer> not done in applications
<aw-> ??
<aw-> it's in the tutorial!
<Regenaxer> Well, yes, external symbols have their symbol table in memory
<Regenaxer> but these are only for the *currently* loaded symbols
<aw-> i see..
<Regenaxer> "(put *DB" is the low level, yes
<aw-> so if I "forget" about (put *DB) , what should I use instead?
<Regenaxer> Entities and relations
<aw-> why?
<aw-> i dont need relations
<Regenaxer> I think you did *not* read the tutorial
<aw-> i just want to maintain a list of key/value across parent/child processes
<aw-> this is become too complex for what i want
<Regenaxer> If you have only key/value pairs then you only need a tree
<Regenaxer> and use 'store' and 'fetch' directly
<aw-> i'm not trying to do a DB maintenance or relationship/entity mapping or any fancy stuff here
<Regenaxer> Then I can't help you
<aw-> well yes actually, i think 'store and 'fetch are the functions I was looking for
<Regenaxer> you can use NIL for the tree if the root of the thee should be *DB
<Regenaxer> in store and fetch
<tankf33der> aw-: good startpoint:
<Regenaxer> In trees the keys can be anything, not only internal symbols
<Regenaxer> tankf33der, thanks, but perhaps still too complicated example
<tankf33der> top task
<Regenaxer> Just (store NIL "key" "value" (1 . 256))
<Regenaxer> (fetch NIL "key")
<tankf33der> school marks db on lists and db
<Regenaxer> yes, very nice
<Regenaxer> Uses E/R so not what aw- wants to use it seems
<tankf33der> ok
<tankf33der> latest ios 11.4, safari crash on picolisp documentation frames
<Regenaxer> oh
<tankf33der> i need tricky steps how to read reference fetch func
<Regenaxer> 'fetch' is a low-level function
<Regenaxer> Normal apps use it seldom
<Regenaxer> I do have some raw trees in applications, mainly for logging
<Regenaxer> Where it is too expensive to build full objects
<tankf33der> i wish sometimes inmplement all this on picodb
<aw-> ok i think (fetch) and (store) with a NIL tree is simplest
<Regenaxer> tankf33der, yep :)
<Regenaxer> aw-, yes, seems fit for your use case
<aw-> Regenaxer: thanks
<Regenaxer> np :)
<aw-> can you also confirm if it will be slow to retrieve 1 of 1000 keys using (fetch) ?
<Regenaxer> 1000 is not much
<Regenaxer> You should use a larger block size if you expect large trees
<Regenaxer> default is only 256
<Regenaxer> so if keys and values are long, only a few nodes fit into a block
<Regenaxer> But 30 should fit, so searcng 1000 only involves 2 levels
<aw-> 256 what?
<aw-> 256 bytes?
<Regenaxer> ideally reading 2 blocks
<Regenaxer> yes, these are bytes
<Regenaxer> see ref of 'pool'
<Regenaxer> you specify the block size there
<aw-> "defaulting to (2) (for a single file with a 256 byte block size"
<aw-> what does this mean?
<aw-> 2 for a single file?
<Regenaxer> no, 2 means 256
<aw-> so 4 would be 512 ?
<Regenaxer> a single file with 1024 blocks would be (4)
<aw-> with (store) do i also need to specify that same blocksize used in (pool) ?
<Regenaxer> Unfortunately yes, as 'store' does not know the context (as the E/R system does)
<Regenaxer> So in an application I create a dummy-relation for such trees
<Regenaxer> eg (rel log (+Swap +Any))
<Regenaxer> But in your simple case just pass it to 'store'
<aw-> yeah... tbh i don't really understand the concept of block size for the DB
<aw-> if my key/value pairs can have a max length of 1KB (1024 bytes), what should I use?
<aw-> (pool "db" (8)) ?
<Regenaxer> External symbols are simply written to blocks in PLIO format
<Regenaxer> oh, thats very long for a single key/value
<Regenaxer> The max size does not matter, but the average
<Regenaxer> It is good if 30 or perhaps.100 key/value pairs fit into a block
<Regenaxer> It is a B-Tree
<Regenaxer> one block is one node ideally
<aw-> 30-100 seems arbitrary
<Regenaxer> yes
<Regenaxer> can be more
<aw-> if each key is 512bytes and each value is 512 bytes, how would I choose the value for (pool) ?
<Regenaxer> *inside* a tree node the search is with 'rank', aka linear
<Regenaxer> Then 32 k
<Regenaxer> would be (9)
<Regenaxer> Do you really have such long keys??
<Regenaxer> If the *values* are big, then better use separate objects to store them
<Regenaxer> to avoid having to load *all* for every search
<Regenaxer> And the keys are better broken down if speed is really an issue
<Regenaxer> Not for 1000 entries perhaps
<aw-> i see
<Regenaxer> How long are the keys on average?
<aw-> no idea
<Regenaxer> You talked of 512 bytes
<aw-> what's the max length of a filesystem dir/path ?
<Regenaxer> I see, then I would index the path elements
<aw-> 255 bytes is max length on XFS
<Regenaxer> But for a first try, just use, say, 6
<aw-> so probably the default 256 bytes is fine for the path, assuming it's used as a key
<Regenaxer> I think it can be more than 256 on modern OSs
<Regenaxer> I think a typical path length is below 100
<aw-> oh sorry, 255 bytes is the filename length
<aw-> not the full path
<Regenaxer> So with (6) or 4 KiB you get enough space
<Regenaxer> still 20 or 40 nodes
<Regenaxer> My recommendation is (6) :)
<aw-> (pool "db" (6)) ?
<Regenaxer> yes
<Regenaxer> Feels ok
<Regenaxer> If the block size is too big, too much unnecessary stuff is loaded on each access
<aw-> oh i see
<aw-> yeah... kinda hard to predict
<Regenaxer> and if too small some nodes may overflow to 2 blocks needing more reads
<Regenaxer> In practice no big difference perhap
<Regenaxer> s
<aw-> hmmmm
<Regenaxer> Depends how often accessed etc
<Regenaxer> I would not worry too much here
<aw-> perhaps I can set an artificial limit to 256 bytes, so i'll never have a key longer than 256?
<Regenaxer> You can tune later
<aw-> or is the 256 key+value ?
<Regenaxer> yes both
<Regenaxer> I would inspect the tree with typical data with (edit *DB)
<Regenaxer> then click on the objects down into the tree nodes
<Regenaxer> I'm doing that often to get an impression
<aw-> i don't know if that'll be worth the effort for now
<aw-> i'll start with smaller values and wait a while before checking if it should be higher
<Regenaxer> yes, good way
alexshendi has quit [Ping timeout: 264 seconds]
karswell has quit [Read error: Connection reset by peer]
karswell has joined #picolisp
<beneroth> aw-, a bit dated and incomplete, but still a good start: http://www.prodevtips.com/2008/04/23/simple-oodb-in-pico-lisp/
<Regenaxer> I think aw- wants something more low-level
<Regenaxer> only key/value pairs
<beneroth> probably, though he should now there is more :)
<Regenaxer> true
<beneroth> key/value stores are a fashion. but all in all they're weaker then DBs with schema. and pilDB has the flexibility of a dumb key/value store while having graph db features, I would say :P
<aw-> it's not a fashion statement
<Regenaxer> correct. He could probably extend his model later if needed
<beneroth> sure. that was not a saying about you
<beneroth> I guess you have a key/value use case, like config or so?
<aw-> i'm doing cache lookups, how else would you store that?
<beneroth> so you have another backend behind it?
<Regenaxer> Sometimes raw tree access is useful. I do that occasionally too
<aw-> ? just picolisp fetching/storing keys from a cache
<aw-> fetching values based on the key
<aw-> i can't think of a more optimal storage mechanism, no need for relational complexity
<beneroth> not without looking at the data, T. I think I would try to maybe split it up in different types/groups, if it makes sense with the data
<aw-> no
<aw-> it's just key/value
<aw-> and it doesn't even need to persist, so i'm using it exclusively in memory, without commit
<beneroth> what about (idx) (not DB functionality) ?
<aw-> if i want to (fork) then i can commit and fetch from the fork
<beneroth> T, good concept
<Regenaxer> Ah, not persistent? OK, understand
<Regenaxer> Such an tree also makes sense (over idx) if it does not fit into memory in total
<beneroth> T
<beneroth> though this use case is a bit of a weakness, afaik you can't just (wipe) the longest-not-used external symbols from memory (because access is not tracked)
<beneroth> special case
<Regenaxer> But you can clean the tree with 'prune'
<Regenaxer> I do that for very big imports
<beneroth> hm right. how does (prune) track accesses ?
<Regenaxer> yes
<beneroth> ah I see. inbuilt into fetch and store, when *Prune global exists
<beneroth> ahnice
<Regenaxer> right
<beneroth> so tracking can easily be enabled. be it with prune or another scheme.
<beneroth> another = custom
<Regenaxer> yep
<beneroth> yeah perfect for aw-s scenario.
<beneroth> thanks, you too
<Regenaxer> I call prune after commits, and then sometimes (gc 0) to free heap space
<beneroth> thanks, you two
<beneroth> yeah
<Regenaxer> But prune is seldom needed, only in long periods of imports
<Regenaxer> During normal operation the automatic gc cleans up also all tree nodes
<Regenaxer> In an import they are always referred to, so the they don't become garbage
<Regenaxer> afp a while
<aw-> beneroth: no need to wipe
<beneroth> T
orivej has quit [Ping timeout: 245 seconds]
orivej has joined #picolisp
orivej has quit [Ping timeout: 260 seconds]
orivej has joined #picolisp
andyjpb has joined #picolisp
<aw-> quick q
<aw-> in the ref docs: (let sym 'any . prg) -> any .. what does "-> any" mean ?
<aw-> shouldn't it be the return value?
<Regenaxer> The 'let' expression returns anything
<Regenaxer> yes
<aw-> but return value of 'let' is prg, not any
<Regenaxer> no, prg runs and gives any
<aw-> oh right right
<aw-> ok my bad
<aw-> yes i forgot that
<Regenaxer> np :)
<aw-> thanks
<aw-> is there a shorthand for doing something like: (Let Myvar Lst (fun1) (fun2) (fun3) Myvar) ?
<aw-> err (let*
<Regenaxer> yes, in some cases 'prog1'
<Regenaxer> (prog1 (foo) (println @) (inc @))
<Regenaxer> will return the result of (foo)
<Regenaxer> same as
<Regenaxer> (let X (foo) (println X) (inc X) x)
<Regenaxer> (let X (foo) (println X) (inc X) X)
<aw-> oh awesome!
<aw-> that's exactly what i was looking for
<aw-> thanks!
<Regenaxer> :)
<aw-> in (prog1), @ is always bound to the result of 'any1 ?
<Regenaxer> yes
<Regenaxer> Must go to the station, fetch my daughter
<Regenaxer> afp
<aw-> ah yes it's in the docs!, case, casq, etc...
<aw-> ok thanks
<aw-> see you!
<Regenaxer> :)
<beneroth> "If your application uses GPGME your application is safe. Fortunately
<beneroth> users should make sure to use "set crypt_use_gpgme".
<beneroth> "
<beneroth> most modern mail readers use GPGME, including GpgOL and KMail. Mutt
beneroth has quit [Quit: Verlassend]
andyjpb has quit [Quit: Leaving.]
clacke[m] has quit [Ping timeout: 256 seconds]
fwirt[m] has quit [Ping timeout: 276 seconds]
alexshendi has joined #picolisp
fwirt[m] has joined #picolisp
clacke[m] has joined #picolisp
orivej has quit [Ping timeout: 256 seconds]
kakobrekla has joined #picolisp
orivej has joined #picolisp
alexshendi has quit [Ping timeout: 256 seconds]