ChanServ changed the topic of #picolisp to: PicoLisp language | Channel Log: https://irclog.whitequark.org/picolisp/ | Check also http://www.picolisp.com for more information
_whitelogger has joined #picolisp
razzy has joined #picolisp
rob_w has joined #picolisp
<tankf33der> Regenaxer: what if after leak fix it can run in 8k stack
<Regenaxer> It cannot run in 8k now?
<Regenaxer> I depends what you call
<Regenaxer> I have not searched for the places yet where 'save' is called in some kind of loop. Maybe not so many
<Regenaxer> In those places it depends how often the loop is executed. It uses more and more stack in each iteration
<Regenaxer> only a few bytes
<Regenaxer> Perhaps I can look this evening
twelvemonkeys has quit [Ping timeout: 246 seconds]
twelvemonkeys has joined #picolisp
<tankf33der> this is llvm or your leak
<Regenaxer> My leak of course
<tankf33der> huge.l can be run in 8k for years
<tankf33der> yea
<Regenaxer> Good
<Regenaxer> The heap size is not an issue
<Regenaxer> Only how often a loop is executed
<Regenaxer> Perhaps there are not as many cases as I was afraid of
<Regenaxer> The 'del' function is the one I saw yesterday
<Regenaxer> if you pass a var with a very long list
<Regenaxer> each iteration uses 16 byte
<beneroth> hi all
<Regenaxer> Hi beneroth
<beneroth> Regenaxer, what do you think about +Key and partial string indexing?
<beneroth> would be (+IdxFold +Key +String) okay?
<beneroth> partial indexing for partial search
<Regenaxer> OK if they don't collide
<beneroth> ah, the +Key than applies to all parts?
<beneroth> so substrings?
<Regenaxer> There is no 'select' filter predicate for such a combination
<Regenaxer> ah
<Regenaxer> sorry
<beneroth> the whole string is unique, but parts collide
<Regenaxer> +IdxFold AND +Key
<Regenaxer> You could try
<beneroth> what do you recommend? staying with just (+IdxFold +String) or (+Key +String) instead?
<beneroth> it's kind of a value table
<Regenaxer> I don't know atm whether the above combination works. Perhaps it does
<beneroth> ok, I'll find out
<beneroth> unless you say it's stupid :)
<Regenaxer> Not sure ;)
<beneroth> another option would be splitting it into 2 properties
<beneroth> one for +Key and one for partial string search
<Regenaxer> yeah
<beneroth> but kinda... spacey
orivej has joined #picolisp
<Regenaxer> I think it may work. (+Sn +Idx... is something similar
<Regenaxer> as (+Key +IdxFold ...
<beneroth> I think it will not work
<beneroth> I want (+IdxFold +Key +String). But I guess either way the substrings are expected to be unique
<beneroth> hmmm I don't have many values here
<beneroth> forgot that. I just keep it (+Key +String) and make a full scan for partial matches
<Regenaxer> ok, saves space at least
<beneroth> but I will think a bit more about the +Key +Idx story
<beneroth> aye
mario-go` has joined #picolisp
lodsw has quit [Ping timeout: 265 seconds]
mario-goulart has quit [Remote host closed the connection]
Regenaxer has quit [Ping timeout: 246 seconds]
lodsw has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej_ has joined #picolisp
orivej has quit [Ping timeout: 256 seconds]
orivej_ has quit [Ping timeout: 246 seconds]
orivej has joined #picolisp
Regenaxer has joined #picolisp
orivej has quit [Ping timeout: 265 seconds]
orivej_ has joined #picolisp
<beneroth> Regenaxer, what is the best way to check if a symbol has (any) properties or not?
<beneroth> getl ?
<tankf33der> would say, yes
<tankf33der> beneroth: you dont have big list to choose from
<beneroth> rofl
<beneroth> yeah
<beneroth> thanks tankf33der :))
<beneroth> I just wondered if there might be another way
orivej_ has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
libertas has joined #picolisp
<Regenaxer> Hmm, getl is not good, as it conses a list
<Regenaxer> But there is no direct function
<Regenaxer> Could check with 'adr' whether the tail is a number
<Regenaxer> But I think this is not a good idea anyway
<Regenaxer> There might be other, non-related, properties
<Regenaxer> So the list is not empty despite your app did not put any
<Regenaxer> Better keep track in some other way
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
<beneroth> Regenaxer, yeah I guessed that getl is constructing
<beneroth> it is about a (box), so there will be no properties beside the ones I added, I believe
<Regenaxer> Probably, but the principle of properties is that they are freely added or removed
orivej has quit [Quit: No Ping reply in 180 seconds.]
<Regenaxer> Are these DB symbols? If so, there is 'lieu'
orivej has joined #picolisp
orivej_ has joined #picolisp
<beneroth> no (box), used to collect values during import, before validating the values and store them in the database
orivej has quit [Ping timeout: 256 seconds]
<Regenaxer> ok
<Regenaxer> Why got an association list, or idx?
<Regenaxer> why *net
<Regenaxer> grr
<Regenaxer> *not
<Regenaxer> (I'm *lying* on the couch and can't write ;)
orivej has joined #picolisp
orivej_ has quit [Ping timeout: 260 seconds]
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
<Regenaxer> Oh, closer inspection shows thatd'del' is ok
orivej_ has joined #picolisp
twelvemonkeys has quit [Ping timeout: 260 seconds]
orivej has quit [Read error: Connection reset by peer]
twelvemonkeys has joined #picolisp
<tankf33der> added non crypto hash functions to pil21-test repo
<Regenaxer> Good
<Regenaxer> I'm stupid! The 'save' stack leaks are not a problem :) Most cases where 'save' is in a loop are correct, eg in the map functions. The only bad one is in 'repl', will fix
orivej_ has quit [Ping timeout: 260 seconds]
orivej has joined #picolisp
rob_w has quit [Quit: Leaving]
<Regenaxer> done
razzy has quit [Quit: Connection closed]
<Regenaxer> Now investigating the gc bug during build
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #picolisp
<Regenaxer> It is not a gc bug
<tankf33der> not a bug?
<Regenaxer> It is a bug
<Regenaxer> But -"gc 400" does not change anything
<Regenaxer> I strange bug
<Regenaxer> Only a single line in base.ll is different
<Regenaxer> a comment line
orivej has quit [Ping timeout: 260 seconds]
orivej has joined #picolisp
<Regenaxer> http://ix.io/2pWw
<Regenaxer> Args is printed without namespace prefix in pil21
<beneroth> yeah yeah that namespacing... ;-)
<beneroth> Regenaxer, mostly to make use of the built-in "redef" warning
<Regenaxer> Now I see that it is in fact not a bug
<beneroth> idx with a (Key . Val) could also be used, aye
<beneroth> whats your common pattern for such tasks?
<beneroth> Regenaxer, oh what is it then? conceptual error?
<Regenaxer> No
<beneroth> ordering?
<Regenaxer> It is just that when building with standard pil it *does* have a symbol Args
<beneroth> is the stuff already present you refer to at that stage?
<Regenaxer> which overshadows that in priv
<Regenaxer> So it print correctly with a prefix :)
<Regenaxer> beneroth, which stuff?
<Regenaxer> Anyway, the printed comment is correct
<Regenaxer> So the question is if there is still another bug in build, which just does not show up currently
Guest83967 has quit [Quit: irc_error]
Guest83967 has joined #picolisp
<beneroth> stuff = internal symbols. I have no idea what I'm talking about
<Regenaxer> :)
<Regenaxer> well, yes
<Regenaxer> at least one symbol, 'Args'
<beneroth> yeah ;-)
<Regenaxer> It is in standard pil, but not (yet) in pil21
<beneroth> ah
<beneroth> I see
<Regenaxer> So everything correct
<Regenaxer> printing without prefix in the comment
<Regenaxer> I print the Lisp expression as a comment into src/base.ll during build
<Regenaxer> I think it comes from lib/pilog.l in standard pil
<Regenaxer> in pil21 lib/pilog.l has its own private namespace
<Regenaxer> voila
Guest83967 is now known as Blue_flame
<beneroth> has pil21 (nsp) yet? :P
<Regenaxer> yes
<Regenaxer> I used it now to find out why the output is different
<beneroth> :)
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
<Regenaxer> tankf33der, what do you wish next in pil21?
<Regenaxer> Should we start with 'native'? I is a tough issue, I avoided until now ;)
<tankf33der> did you fixed everything related to gc?
<Regenaxer> no idea
<Regenaxer> no symptoms atm
<tankf33der> then try read issue.md file
<Regenaxer> Perhaps it was fixed in the course of other changes, perhaps not
<Regenaxer> OK, I did
<Regenaxer> that's why I fixed the map functions
<Regenaxer> # eval does not apply
<Regenaxer> no more env offsets
<Regenaxer> the test must be changed
<Regenaxer> only '@' is handled by the env offset
<Regenaxer> Same for 'run'
<Regenaxer> doc/diff:
<Regenaxer> - Lambda-binding environment offset for 'eval' and 'run' only for '@'
<Regenaxer> 'dm' is not implemented at all yet, see src/flow.l
<Regenaxer> (de _dm (Exe)
<Regenaxer> $Nil )
<Regenaxer> So perhaps 'dm' next?
<tankf33der> implement comparing
<tankf33der> that look at left
<tankf33der> < and so on
<Regenaxer> Comparing is there, only the high level funs are missing
<Regenaxer> they are all the same
<Regenaxer> So better not too early I though, they give no now insights
<Regenaxer> '>' and '>=' is there
<tankf33der> dm then, better fix from top to bottom
<Regenaxer> OK. But then perhaps I should really try 'native' to see if everything works out
<tankf33der> will play with sha512, why it crashes
<Regenaxer> ok, great
<Regenaxer> Crashes are the most interesting things atm
<tankf33der> (gc 64) saves, so this is gc issue
<Regenaxer> they may indicate a general problem
<Regenaxer> I see
<Regenaxer> Perhaps we can find that
<Regenaxer> Can you try to test the functions used by sha512 individually?
<Regenaxer> To see *which* one
<Regenaxer> I do 'dm' meanwhile
<tankf33der> all crypto is ok and quite the same, except sha512 so this is gc or cons
<Regenaxer> yeah
<Regenaxer> cons is simple, probably ok
<Regenaxer> (if not a fundamental systematic error)
<tankf33der> but gc 64 saves then this is not an any function issue
<Regenaxer> You cant say so
<Regenaxer> gc 64 just causes gc *not* to run later
<tankf33der> what is simplest way generate lists and run gc often?
<tankf33der> idea to generate lists of different length
<tankf33der> also get backtrace from core file
<Regenaxer> But then you always test 'list' or 'cons' which are probably ok
<Regenaxer> Does sha512 use any special functions?
<Regenaxer> any other?
<tankf33der> no
<tankf33der> cut conc list
<Regenaxer> hmm
<tankf33der> i will try first find the test where crash happens
<Regenaxer> Can you point me to the source?
<Regenaxer> thx
<tankf33der> it happens after latest mapcan
<tankf33der> iirc
<tankf33der> or inside it, or inside list creation for mapcan
<tankf33der> afk.
<Regenaxer> When it happens it is already a long time after the bug
<Regenaxer> gc is just the symptom usually, not the reason
<Regenaxer> unless, of course, gc itself is buggy
<Regenaxer> Let's see ...
<Regenaxer> nice
<Regenaxer> One more question? How do you call (d512 ...)?
<Regenaxer> (d512 (range 1 9)) works ;)
<Regenaxer> (d512 (range 1 999999)) also
<Regenaxer> As for the issues, 'finally' is also not implemented yet
<Regenaxer> (de _finally (Exe)
<Regenaxer> $Nil )
<Regenaxer> Also coroutines are just a skeleton, a proof of concept initially
<Regenaxer> So as I said, you tested too early
<Regenaxer> the 'need' issue is fixed
<Regenaxer> 'memq' too
<Regenaxer> So the first item in issues is 'fill'
<Regenaxer> I fix that now
orivej_ has joined #picolisp
orivej has quit [Ping timeout: 265 seconds]
<Regenaxer> 'fill' is fixed
<Regenaxer> oops, not the second one
<Regenaxer> ok, now second issue too
<Regenaxer> released
orivej_ has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #picolisp
twelvemonkeys has quit [Ping timeout: 260 seconds]
twelvemonkeys has joined #picolisp
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
<tankf33der> http://ix.io/2pXo
<tankf33der> so debug flag matters.
<tankf33der> playing with sha512
<Regenaxer> Debug mode loads more things, so the point of error moves
<tankf33der> when i insert println in sha512 function crash disappears
<Regenaxer> Yes, that's normal with this kind of bug
<Regenaxer> gc must run just at the right moment
<Regenaxer> Just playing does not help, we must locate the faulty function(s)
<Regenaxer> It can be anything
<Regenaxer> sha512 calls dozens of functions
<Regenaxer> arithmetics, lists functions etc
<tankf33der> its latest mapcan
<tankf33der> proven
<Regenaxer> Can be *any* function
<Regenaxer> even 'let' if it destroys the stack
<Regenaxer> I asked above:
<Regenaxer> One more question? How do you call (d512 ...)?
<Regenaxer> (d512 (range 1 9)) works ;)
<Regenaxer> (d512 (range 1 999999)) also
<Regenaxer> How to call it?
<tankf33der> via test function
<tankf33der> read official test vectors and run and comparer
<tankf33der> read official test vectors and run and compare
<tankf33der> you could clone repo or download related files.
<tankf33der> this is not mapcan because mapcar crashed too
<tankf33der> this is not mapcan because mapcar crashed too
<Regenaxer> ?
<Regenaxer> "this" ?
<tankf33der> mapcan crashed and mapcar too
<tankf33der> so this is something else inside
<Regenaxer> All map functions have very similar code
<Regenaxer> If one is wrong, all are
<Regenaxer> As a few days ago
<Regenaxer> The bug with the atomic tail
<Regenaxer> I changed
<Regenaxer> all map functions, see repo
<Regenaxer> So I thin map is not the problem
<Regenaxer> I thought very much about it several times
orivej has quit [Quit: No Ping reply in 180 seconds.]
<Regenaxer> It can be any function in sha512.l, or some other function called e.g. by 'hex'
orivej has joined #picolisp
<Regenaxer> Most calls are arithmetics, so the probability is high that it is one of the arithmetic functions
<Regenaxer> The calculations are all bignum, cause 64 bit don't fit into a short
<tankf33der> if i change something inside latest mapcan crash disappears
<Regenaxer> This does not mean anything
<Regenaxer> Change something else too and it may re-appear
<Regenaxer> How do you know that mapcan crashed?
<Regenaxer> What we need is a more isolated test, with fewer functions
<tankf33der> changing (let R in mapcan for
<tankf33der> (let R (list 1) ...
<Regenaxer> Remember: If it is really a corrupted stack or heap, the crash occurs at some random point much later
<tankf33der> but when i change (for (I 17 (>= 80 I) (inc I))
<tankf33der> to
<tankf33der> (for (I 17 (>= 20 I) (inc I))
<tankf33der> it appears again.
<tankf33der> :)
<tankf33der> yea.
<Regenaxer> yeah
<Regenaxer> We need a simpler test :)
<tankf33der> it will disappear
<Regenaxer> On the other hand have we tested arithmetics a lot with bigtest(s)
<tankf33der> yes, tested. in general they are all ok.
<tankf33der> even 20k digits.
<tankf33der> even on 20-30k digits.
<Regenaxer> good
<tankf33der> even if i call sha512 from test-sha512.l like this it crashes:
<tankf33der> (sha512 (make (do 64 (link (rand 0 255)))))
<tankf33der> so I dont use K and C read from test vector file.
<Regenaxer> ok
<tankf33der> sha512 and sha256 so equal, use the same functions. like copy-paste brothers
<tankf33der> but sha256 dont crash.
<Regenaxer> Hmm, here it does not crash
<Regenaxer> (loop (sha512 (make (do 64 (link (rand 0 255))))) (msg (inc (0))))
<tankf33der> i meant use in test-sha512.l file
<tankf33der> in standalone of course its ok.
<tankf33der> if i call sha512 function from test-sha256.l file it dont crash.
<Regenaxer> hmm
orivej has quit [Ping timeout: 260 seconds]
orivej_ has joined #picolisp
<tankf33der> if i remove even one test vector from file crash disappears
<Regenaxer> Yeah
<Regenaxer> Heisenbug
<tankf33der> maybe it is possible to run it under debug output
<Regenaxer> This will give no hint what is wrong
<Regenaxer> The only way is to find simpler tests
orivej_ has quit [Ping timeout: 264 seconds]
orivej has joined #picolisp
<Regenaxer> ideally with one single function ;)
<tankf33der> no way to do it
<Regenaxer> Looks like we must modify src/gc.l again
<Regenaxer> so that it crashes more near or inside the function
<tankf33der> even if i add one (gc) call crash disappears.
<Regenaxer> Then add more and more until it appears again ;)
<Regenaxer> But no, adding gc on the top level does not help
<Regenaxer> Then the bad function is always called with enough memory and will not call gc
<tankf33der> http://ix.io/2pXJ
<Regenaxer> In the past it helped to force gc in each cons etc
<tankf33der> backtrace from gdb
<Regenaxer> I'm walking the fields now and cannot test well
<Regenaxer> ok
<Regenaxer> _chop
<Regenaxer> Perhaps it is chop
<Regenaxer> Can you produce more such dumps in various situations?
<Regenaxer> To see if it is always chop
<Regenaxer> I think mkChar is safe
<Regenaxer> Could also be _for
<Regenaxer> _for is very complicated
<tankf33der> it is always the same if I crash it in the same place.
<Regenaxer> good!
<Regenaxer> So either for or chop
<tankf33der> if I switch crash it inside latest mapcan backtrace will be different
<Regenaxer> I take a look when I'm home
<tankf33der> ok
<tankf33der> afk.
<Regenaxer> thanks!!
orivej has quit [Ping timeout: 260 seconds]
orivej has joined #picolisp
<Regenaxer> Intersting. It *does* crash in 'hex'
<Regenaxer> Cause 'chop' inside 'for' is nowhere in sha512
<Regenaxer> Of course it does not guarantee that 'for' or 'chop' is faulty.
<Regenaxer> Can also be some function called before that
<Regenaxer> But 'hex' is very early, only at load time, right?
<tankf33der> [mpech@lambda crypto]$ cat test-sha512.l | grep chop
<tankf33der> (for (L (chop H) L (cddr L))
<tankf33der> chop inside for!
<tankf33der> afk, use my nickname to ping me.
<Regenaxer> I find no such place in sha512
<Regenaxer> (for (L (chop H)
<Regenaxer> I have only
<Regenaxer> (for (L (chop H)
<Regenaxer> oops
<Regenaxer> (for (I 17 (>= 80 I) (inc I))
<Regenaxer> and
<Regenaxer> (for I 80
<Regenaxer> You have a different version
<Regenaxer> No hurry with pinging
<Regenaxer> I'm still walking
<Regenaxer> and then will shower
<tankf33der> it is in test-sha512.l
<tankf33der> thats why call sha512 function from test-sha256.l dont crash
<tankf33der> at the same time test-sha256 also have for-chop loop
<Regenaxer> I see
orivej_ has joined #picolisp
orivej has quit [Ping timeout: 240 seconds]
orivej_ has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Ping timeout: 265 seconds]
orivej_ has joined #picolisp
orivej_ has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #picolisp
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #picolisp
orivej has quit [Ping timeout: 240 seconds]
orivej has joined #picolisp