ChanServ changed the topic of #picolisp to: PicoLisp language | Channel Log: https://irclog.whitequark.org/picolisp/ | Check also http://www.picolisp.com for more information
_whitelogger has joined #picolisp
_whitelogger has joined #picolisp
orivej has quit [Ping timeout: 265 seconds]
DKordic has quit []
_whitelogger has joined #picolisp
aw- has quit [Quit: Leaving.]
aw- has joined #picolisp
patrixl has quit [Remote host closed the connection]
<Regenaxer> app/ crashes in ht:Prin
<aw-> hi Regenaxer
<Regenaxer> hi aw-
<aw-> i have another question about (native) if you have a minute
<Regenaxer> ok
<aw-> ok thanks.
<aw-> i want to read 4 bytes from a filedescriptor with (native "@" "read").. but those 4 bytes represent an Integer
<aw-> what is the buffer I need to pass? I tried '(Buf (4 . i)) like in the native docs
<aw-> but it doesn't work
<aw-> err.. '(Buf (4 . I))
<Regenaxer> 4 bytes are good
<Regenaxer> yes
<aw-> i get back an unusually large integer
<aw-> oh.. is this big/little endian problem?
<aw-> i'm reading from a file: (out "a" (wr 0 0 0 13))
<aw-> then (open "a") and (native "@" "read" 'N @ '(Buf (4 . I)) 4)
<aw-> returns -1
<aw-> err.. sorry, bad file descriptor
<aw-> anyways if i pass the correct FD, the result in Buf is wrong
<Regenaxer> let me try here
<aw-> (let Fd (open "a") (native "@" "read" 'N Fd '(Buf (4 . I)) 4) (println Buf))
<Regenaxer> I tried, this works:
<Regenaxer> (let F (open "a") (%@ "read" 'I F '(X (4 . I)) 4) (close F))
<Regenaxer> Looks very similar! :)
<Regenaxer> even "a" ! :)
<Regenaxer> I get the correct value in 'X'
<Regenaxer> Return value of read is int, so 'I' is better
<Regenaxer> but should not matter on 64 bits
<aw-> yeah.. return value of read() is just the length of the bytes read
<Regenaxer> T
<Regenaxer> But what goes wrong on your side?
<aw-> i get this
<aw-> 218103808
<Regenaxer> this is correct
<Regenaxer> big endian
<Regenaxer> : (hex 218103808)
<Regenaxer> -> "D000000"
<Regenaxer> BTW, remember that you should (use Buf ...) around your code
<Regenaxer> in all such native usages of return variables
<aw-> yes i did that
<aw-> here it's just an example
<aw-> (hd "a")
<aw-> 00000000 00 00 00 0D ....
<Regenaxer> good
<Regenaxer> Anyway, your read call works
<aw-> hmmm...
<Regenaxer> (wr 0 0 0 13) is big endian of 13 decimal or D hex
<aw-> right
<aw-> so why am I getting D000000 instead of 000000D ?
<Regenaxer> big endian
<Regenaxer> you wanted big endian
<Regenaxer> but the machine seems to be little endian
<Regenaxer> the 13 is at the highest byte
<Regenaxer> buffer + 3
<aw-> ok so if i'm sending the 4-byte integer '13' over the network, (0 0 0 13) is correct, right?
<Regenaxer> network order is which endianess?
<aw-> big endian
<Regenaxer> yes, you send 13 last then
<aw-> yes
<aw-> and my protocol should read the bytes: (0 0 0 13)
<aw-> it does
<Regenaxer> yeah
<aw-> but why when I use '(Buf (4 . I)) i get 218103808 instead of 13?
<Regenaxer> and 13 is at the highest address
<aw-> because my CPU?
<Regenaxer> yes, Arm and x86 are little endian
<aw-> ohshit
<aw-> well that explains it
<aw-> in this case I can't use (4 . I) .. better to use (4 B . 4) and then convert to Int manually?
<Regenaxer> hmm, what did I do in lib/net.l?
<Regenaxer> call htonl()
<Regenaxer> these are C macros usually
<Regenaxer> no, in pil64 I call htons()
<Regenaxer> why not in lib/net.l ?
<aw-> in pil21 you shift bytes by 8
<aw-> left shift
<Regenaxer> right
<aw-> ahh yes
<Regenaxer> it is only 2 bytes for a short
<aw-> that makes sense
<Regenaxer> but you can call htonl()
<aw-> i've been using (rd 4) since the beginning.. and because pil knows my cpu arch, it handles the conversion
<Regenaxer> 8 bytes I presume
<Regenaxer> no, pil does not know
<Regenaxer> it uses what you tell
<Regenaxer> (rd -4) reads little endian
<aw-> ohhh ok
<aw-> so i've been reading in big endian all this time without realizing
<Regenaxer> as you know the data are big endian, all is ok
<aw-> "it just works" for network data
<aw-> yes exactly
<Regenaxer> yes
<aw-> haha oops
<Regenaxer> So all good here
<aw-> ok well I can fix this now, no problem
<aw-> thank you
<Regenaxer> call htonl()?
<Regenaxer> or shifting?
pointfree has quit [Read error: Connection reset by peer]
pointfree has joined #picolisp
<aw-> htonl probably.. but maybe I don't to do it because the code might run on systems which are already big endian.. dont want to break for those systems
<aw-> maybe i don't want* to
<Regenaxer> yeah, htonl is 32 bit
<Regenaxer> It wont break
<Regenaxer> on such systems htonl does nothing
<Regenaxer> it is precisely for such portability
<aw-> oh ok
<aw-> wait, reading manpage now ;)
<aw-> ok works!
<Regenaxer> perfect
<aw-> (use Buf (let Fd (open "a") (native "@" "read" 'I Fd '(Buf (4 . I)) 4) (println (native "@" "htonl" 'I Buf))))
<Regenaxer> Looks good
<Regenaxer> Good to know htonl is not a macro
<Regenaxer> Thats why I avoided in lib/net.l I think
<aw-> oh
<aw-> well it could save you a bit of code ;)
<Regenaxer> yes, shifting and oring
<Regenaxer> no big thing
<Regenaxer> and faster than a native call
<Regenaxer> also for 4 bytes
<Regenaxer> So your own (de htonl (N) ... might make sense
<aw-> perhaps but then i need to check the arch
<Regenaxer> no
<Regenaxer> ok, for efficiency
<Regenaxer> to avoid the shifting
<Regenaxer> you can read the int if you know the arch is big endian
<Regenaxer> But you can always read 4 bytes and combine them
<Regenaxer> regardless of arch
<Regenaxer> As @lib/net.l does with the short 2 bytes
<aw-> hmmm
<aw-> yes with 4 bytes i would need to shift -16 and -24 as well
<aw-> ok easy, i'll try that
<Regenaxer> yes
<Regenaxer> I debug ht:Prin
<Regenaxer> can't believe it is wrong, must be something fundamental
<aw-> oh right
<Regenaxer> Have not much time today, let's see
<aw-> i got this:
<aw-> (| (>> 0 (; Buf 4)) (>> -8 (; Buf 3)) (>> -16 (; Buf 2)) (>> -24 (; Buf 1)))
<aw-> it seems to work, but it feels wrong
<aw-> the buffer is '(Buf (4 B . 4))
<Regenaxer> (>> 0 is not needed, but '|' is missing
<Regenaxer> oops
<Regenaxer> no, '|' is there :) Sorry
<Regenaxer> I would do (++ Buf)
<Regenaxer> (| (>> 24 (++ Buf)) (>> 16 (++ Buf)) .. (car Buf))
<Regenaxer> all negative of course
<aw-> oh i see.. pop from the list
<Regenaxer> Just a little shorter, and no repeated traversal of the list
<aw-> i like it
<aw-> thanks
<Regenaxer> :)
<aw-> (| (>> -24 (++ Buf)) (>> -16 (++ Buf)) (>> -8 (++ Buf)) (>> 0 (++ Buf)))
<aw-> much better
<Regenaxer> Leave out the (>> 0
<Regenaxer> and the last needs no '++'
<Regenaxer> just runtime overhead
dexen has quit [Ping timeout: 260 seconds]
<Regenaxer> >> constructs a new Lisp number
<Regenaxer> short, so no garbage, but still quite some machinery
<Regenaxer> and ++ assigns the Buf symbol, additional memory access for nothing
<Regenaxer> Remember we have an interpreter. No compiler optimizing away nonsense code
<Regenaxer> It is under your control :)
<aw-> so wouldn't it be more efficient to use htonl() ?
<Regenaxer> Not sure, but I suspect the 'native' call overhead is bigger
<Regenaxer> you could measure with 'bench' in a loop
<Regenaxer> both versions
<Regenaxer> Would be interesting to know
<aw-> ok i'll do it in a bit
<Regenaxer> great, thanks!
<aw-> ok
<aw-> htonl is faster
<aw-> wait.. let me try again without error checking
<aw-> Regenaxer: htonl is 'marginally' faster.. (bench (do 100000 ...) difference of less than 100ms
<Regenaxer> can you paste the two bench calls?
<aw-> but there's also latency from open/close the same file 100000 times
<Regenaxer> No need to open a file to test htonl
<Regenaxer> My problem with ht:Prin is deeper
<Regenaxer> ht:Out is called nested, which should not be
<Regenaxer> So it is the event handling
<aw-> oh
<Regenaxer> Might also concern the stress error yesterday
orivej has joined #picolisp
<aw-> not sure how i can help with that
<Regenaxer> I must find myself. It is one of the heaviest parts in pil
<Regenaxer> No, don't test read
<Regenaxer> only htonl
<Regenaxer> ok
<Regenaxer> like that:
<Regenaxer> (let L (0 .) (bench (do 1000000 (| (>> -24 (++ L)) (>> -16 (++ L)) (>> -8 (++ L)) (car L)))))
<Regenaxer> and:
<Regenaxer> (bench (do 1000000 (native "@" "htonl" 'I 0)))
<aw-> i se
<aw-> i see
<Regenaxer> The first is 0.121 and the second 0.163
<Regenaxer> Not so much difference
<aw-> htonl is _much_ faster on my computer
<Regenaxer> but native is slower
<aw-> in these tests, 2x faster
<Regenaxer> but you test other stuff
<aw-> no no
<aw-> with the tests you pasted above
<Regenaxer> ah
<aw-> i ran them multiple times
<Regenaxer> it is also Arm64 ?
<aw-> yes, 6-core arm64
<Regenaxer> I'm on my phone
<Regenaxer> let me test on an x86 server
<aw-> probably same CPU though
<aw-> dual 1.8GHz Cortex-72 and quad 1.4GHz Cortex-A53
<Regenaxer> on x86 8 core I get 0.108 vs 0.099
<aw-> 1st one, average: 0.184 sec, 2nd one, average: 0.070 sec
<Regenaxer> interesting
<Regenaxer> So native is very fast
<aw-> on here yes
<aw-> seems like it
<Regenaxer> yeah
<Regenaxer> Something for tankf33der to test also on other systems :)
<aw-> maybe it's slower because of the circular list? and ++ ?
<Regenaxer> This makes no difference
<Regenaxer> the circ list I mean
<Regenaxer> the interpreter just pops it
<aw-> ok
<Regenaxer> I try %@
<aw-> oh right, i forgot to test on pil21
<aw-> oh wow
<aw-> 0.200 sec
<aw-> 0.271 sec
<aw-> pil21 native is slower
<Regenaxer> I tested all on pil21
<Regenaxer> maybe thats the reason!
<aw-> my previous tests were on pil64
<Regenaxer> pil21 uses ffi lib
<Regenaxer> that's why!
<Regenaxer> it is a lot slower :(
<aw-> indeed
<Regenaxer> I try pil64 now
<aw-> almost 4x slower than pil64
<aw-> (for this test)
<Regenaxer> yeah! Same here
<Regenaxer> pil64 is a lot faster in native
<Regenaxer> Hmm, we have to live with that
<Regenaxer> 0.1 sec for a million calls
<aw-> haha
<aw-> not a big deal tbh
<Regenaxer> Yes
<Regenaxer> Great, we have a new insight
<Regenaxer> ffi adds some interpretation overhead
<Regenaxer> thats the price for portability
<aw-> i dont plan to make a million calls to htonl.. at least not within 1 second
<Regenaxer> :)
<aw-> anyways you can focus on optimize at a later time
<Regenaxer> right, but ffi is given
orivej has quit [Ping timeout: 240 seconds]
dexen has joined #picolisp
<Regenaxer> ret
<Regenaxer> Meanwhile I understood the problem with ht:Prin
<Regenaxer> As I suspected, the problem is in the event handler, in waitFd()
<Regenaxer> it is not reentrant
<Regenaxer> Somehow I ignored that in pil21
<Regenaxer> (it is reentrant in pil32 and pil64)
<Regenaxer> So if waitFd()s are nested as in the GUI, it crashes
<Regenaxer> Will fix
orivej has joined #picolisp
<aw-> great! glad you figured it out
<Regenaxer> Hmm, well, yes, but I made it reentrant now, and it still does not work ;)
<Regenaxer> Either did it wrong or there is another reason
<Regenaxer> I'm close
<Regenaxer> yes, I did it wrong. Must redesign
<aw-> nice you get to spend so much time coding
<aw-> night
aw- has quit [Quit: Leaving.]
<tankf33der> pilbox on top page of hackers news
<Regenaxer> oh, wow!
<Regenaxer> It is the old link. Who posted that?
<tankf33der> tell me new, i will reply
<Regenaxer> There is no new doc :(
<tankf33der> see
<tankf33der> i already post it via reddit many years ago, maybe here too
<Regenaxer> cool
<Regenaxer> Anyway, nice
<Regenaxer> I have no progress with pil21 today. No time and peace
<tankf33der> i spent all day with darcs.
<Regenaxer> Fixed the waitFd reentrant issue, but that was not the problem. Removed it again
<Regenaxer> What is darcs?
<tankf33der> like git but different idea behind
<tankf33der> all dcvs are darcs and all others
<dexen> darcs differs in having a different approach to change ordering
<dexen> in git, hg, etc., the changesets are strictly ordered, by parent-child relationship
<dexen> in darcs there's more flexibility, as long as the changes are reorderable, you can reorder them.
<dexen> this makes rebase & merge less of a chore / less common
<tankf33der> yea :)
<tankf33der> i like it a lot
<tankf33der> mirror.
<tankf33der> afk.
<dexen> i would probably enjoy it, tho for now i prefer git because if its very natural & sensible *storage* format. given my filesystem interest, the storage format is very natural to me.
_whitelogger has joined #picolisp
<Regenaxer> I see, thanks!
_whitelogger has joined #picolisp
<dexen> generally, how does one implement a very simple DSL in picolisp?
<dexen> i want to interpret RPG style dice throw expression, similar to "3d10+5"