razzy has quit [Remote host closed the connection]
razzy has joined #picolisp
Viaken[m] has quit [Read error: Connection reset by peer]
freeemint has joined #picolisp
Viaken[m] has joined #picolisp
rob_w has joined #picolisp
freeemint has quit [Ping timeout: 240 seconds]
freeemint has joined #picolisp
alexshendi has joined #picolisp
alexshendi has quit [Read error: Connection reset by peer]
freeemint has quit [Ping timeout: 240 seconds]
mtsd has joined #picolisp
<mtsd>
Good morning!
<Regenaxer>
Hi mtsd! :)
<mtsd>
Hi Regenaxer! All well, I hope?
<Regenaxer>
Yesss :)
<mtsd>
Happy to hear that :)
<Regenaxer>
I hope you are well too!
<mtsd>
Yes, I'm well too. A bit busy, but well :)
<Regenaxer>
perfect
<mtsd>
Not as busy with Picolisp as I would like, unfortunately. But I hope that will change soon :)
<Regenaxer>
ok :)
<razzy>
Good morning :]
<Regenaxer>
Hi razzy
alexshendi has joined #picolisp
<Nistur>
mornin'
<Regenaxer>
Hi Nistur!
<Nistur>
hulloooooo :)
<Regenaxer>
:)
alexshendi has quit [Ping timeout: 264 seconds]
<Nistur>
Regenaxer: my graphics question from a few days ago, I think I know how I'm going to do this :P I think the problem is that I haven't done much with that sort of level of data access in lisp at all, and the way I learn is by, basically, attacking the problem from a low level, so I understand it, before trying to figure out the higher level stuff... I could try to dig into picolisp source to try to figure
<Nistur>
out what I want to do there, I have some ideas, but I think that pil will actually distract me from learning what I want to, to begin with... so instead I have a sort of minimalist lisp in my CV project, which I've written from scratch. It has pretty much zero functionality, but because I wrote all of it, I (in theory) know how it all plugs together, so I'll try and solve my problem in that, or at least
<Nistur>
figure out what the questions I want to ask are properly, then if I can do that, then I'll try to figure out how to do it in pil :)
<Regenaxer>
OK
<Nistur>
I just don't quite know enough information yet about what I want to do :)
mtsd has quit [Ping timeout: 264 seconds]
Viaken[m] has quit [Remote host closed the connection]
Viaken[m] has joined #picolisp
illo has joined #picolisp
DKordic has joined #picolisp
ubLIX has joined #picolisp
ubLIX has quit [Ping timeout: 245 seconds]
razzy has quit [Ping timeout: 246 seconds]
razzy has joined #picolisp
<razzy>
um, if i want to trigger my own function for "cleanup" when part of my SW reach certain size. and i do not want periodically (size) huge chunks of memory. is there some trigger before (gc) is run?
ubLIX has joined #picolisp
orivej has quit [Ping timeout: 245 seconds]
<DKordic>
razzy: IMHO it is in pil.asm. Source Code is the "Configuration".
<razzy>
i would expect some *Variable holding functions that are executed when signal of (80% of memory exhausted) is send. or something like that
<razzy>
do-it-yourself is also an option :]
<DKordic>
razzy: Have You seem Suckless.org?
<DKordic>
Not that it is original.
<razzy>
DKordic: i would like to SuckLess :]
<Regenaxer>
razzy, you could call 'heap' periodically to measure free space, but it is very un-precize (MiB resolution)
<beneroth>
hi all
<Regenaxer>
Hi beneroth
* DKordic
:
<Regenaxer>
razzy, but 'gc' is probaly a lot faster than such measurements
<beneroth>
Hi Regenaxer o/
<beneroth>
razzy, don't try to do things yourself badly as long as the existing solutions are good enough...
<Regenaxer>
T
<beneroth>
or go full deep and implement it fully yourself
<beneroth>
you can't do this just a little bit.
<beneroth>
that is not how it works.
<Regenaxer>
Why not doing the cleanup all the time
<beneroth>
DKordic, o7
<DKordic>
Greetings beneroth, Regenaxer :3 .
<Regenaxer>
Hi DKordic
<Regenaxer>
razzy, clean up when you don't need the stuff any more
<Regenaxer>
gc runs probably much more often
<Regenaxer>
(typically a few times per second)
<DKordic>
razzy: How is Your goal different from _Generational_ GC?
<Regenaxer>
Looks like you are doing optimization of things you don't understand
<razzy>
Regenaxer: because when doing evolution, i want to keep as much information as possible, before cleanup is needed. i agree, it is propably silly, better for management is fixed-size population
<Regenaxer>
Have you tried?
<razzy>
beneroth: i agree that doing bad solutions yourself is not wanted
<beneroth>
razzy, cleaning up is only happening when stuff is out of memory scope...
<DKordic>
Generational GC is very interesting but beyond me ATM. I haven't understood Mark&Sweep yet.
<razzy>
i do not want to implement GC.
<beneroth>
DKordic, you properly overthinnk the stuff, I think conceptually they are pretty simple
<beneroth>
razzy, then don't bother with it.
<razzy>
i just want to delete parts of my program before (gc) is run
<beneroth>
razzy, you're question is about implementing your own GC
<beneroth>
what do you mean by delete? overwrite with special values? or de-allocate?
rob_w has quit [Remote host closed the connection]
<beneroth>
picolisp has not really a concept of deleting memory on the user level, I would argue... Regenaxer ?
<beneroth>
except when using (gc 'num) .. with a number argument
<DKordic>
razzy: WeakRef?
<razzy>
i just want to remove pointers to it like (delete)(remove) function do.
<beneroth>
razzy, in picolisp, memory is handled as a big pool if tiny memory cells. picolisp is built upon those memory cells. cells out of use get reused as soon as needed.
<beneroth>
no pointers in picolisp
<razzy>
i meant adresses
<Regenaxer>
right, freeing memory manually is not possibl
<beneroth>
well,., (let Pointer ...here the pointer exists) after (let) it doesn't exist anymore
<Regenaxer>
e
<Regenaxer>
You can release data structures you don't need
<beneroth>
just make use of let :)
<Regenaxer>
then gc cleans them up
<Regenaxer>
right
<Regenaxer>
eg. the DB lets you do things like 'prune'
<beneroth>
(let) is all the memory management in picolisp you need. and practically the only tool to control memory management besides explicitly calling (gc) yourself. and it's all you need.
<Regenaxer>
T
<beneroth>
Regenaxer, I would not mix (prune) into memory management.. its more like DB caching control. you are not wrong, but 'prune is only about DB...
<beneroth>
right?
<beneroth>
razzy, what other programming languages do you know?
<beneroth>
e.g. (let) is comparable to using(var foo = new bar....){...} in C#
<beneroth>
or scoping with { .. } in general (in C#, C/C++ etc.)
<Regenaxer>
No, it is not only caching
<Regenaxer>
it is to remove references to memory objects (tree nodes)
<beneroth>
yes I now
<beneroth>
but the tree nodes is practically the DB cache, I would argue? :)
<Regenaxer>
The gc handles external objects as weak references
<beneroth>
that means?
<razzy>
i will ask differently, i will formulate that
<Regenaxer>
All externals are cached, yes
<Regenaxer>
but will not be gc'ed as long as referenced
<beneroth>
T
<Regenaxer>
And a tree references a lot
<beneroth>
prune works in combination with gc, it's not a higher layer than gc or such.
<Regenaxer>
if the root node is in memory ('lieu'), the whole tree is never freed
<beneroth>
it's an addition
<Regenaxer>
right
<Regenaxer>
removes references
<Regenaxer>
like 'let' when done
<beneroth>
prune is DB memory/cache control, prune is not gc
<beneroth>
that I meant :)
<Regenaxer>
correct
<beneroth>
but I believe razzy is not working with the DB, right razzy ?
<razzy>
no DB
<Regenaxer>
and how big are your data?
<Regenaxer>
larger than RAM?
<razzy>
not larger than RAM yet.
<Regenaxer>
So don't worry at all
<razzy>
i just want to exhaust RAM to see where it is going.
<Regenaxer>
good
<Regenaxer>
But gc is not the right place
<Regenaxer>
it is called very often, and it makes no sense to clean up at that moment
<Regenaxer>
Release data *when* you don't need them, *not* when gc runs
<razzy>
Regenaxer: ok, what if i want to keep maximum amout of data?
<razzy>
from randomly generated process?
<Regenaxer>
Why do you want to keep?
<Regenaxer>
either you need them or you don't
<Regenaxer>
Throw away what you don't need
<razzy>
i think it is false dichotomy
<DKordic>
"There are only 3 hard problems in CS: naming and cache invalidation. "
<razzy>
Regenaxer: i want throw away data i need the least only when it is neccesarry
<Regenaxer>
WHEN is it necessary? How do you know?
<razzy>
and i was under belief (gc) is run only when memory is exhausted
<Regenaxer>
right
<razzy>
Regenaxer: when memory is full
<Regenaxer>
but if frees only stuff not referenced
<Regenaxer>
It is not trivial to know that data freed in this way are indeed nowhere referenced
orivej has joined #picolisp
<Regenaxer>
you may have pointers in variables, eg in callers of that code
<Regenaxer>
So gc may not free the space now, but at some later point
<Regenaxer>
As I said, either forget about gc, or think of it running *all* the time
<Regenaxer>
ie very often
<Regenaxer>
Depending on the situation, it may run hundred times per second
<Regenaxer>
No good idea to hook into that
<razzy>
my other idea is to run (size) very often on 5GB list of stuff and when it is more than 5GB i de-reference something.
<razzy>
which seem ugly
<Regenaxer>
'size' is too expensive, it traverses all data
<Regenaxer>
As I said above, use 'heap'
<razzy>
ok, will do.
<Regenaxer>
If the heap gets too large, throw away something
<razzy>
i was not aware about heap being less expensive.
<razzy>
thx
<Regenaxer>
Yes, heap is fine, as it calls allocated chunks, not cells
<Regenaxer>
1-MiB-chunks
<razzy>
cool
<Regenaxer>
You can shrink the heap if you need to, with (gc 0)
<Regenaxer>
This *tries* to free empty chunus
<Regenaxer>
chunks
<razzy>
nice :D but i think i do not need it
<razzy>
propably
<Regenaxer>
ok
<Regenaxer>
My mistype above: heap is fine, as it calls allocated chunks
<Regenaxer>
s/calls/counts
orivej has quit [Ping timeout: 250 seconds]
<razzy>
and (gc) is run asynchrounously, yes? on external trigger?
<Regenaxer>
Not asynchrounously in that sense
<Regenaxer>
it runs when a new cell needs to be allocated and all heap segments are full
<Regenaxer>
It runs asynchrounously seen from the user program
<Regenaxer>
Usually deep in some nested execution
<razzy>
thx very much.
<beneroth>
DKordic, I know it was "There are only 3 hard problems in CS: naming, and off-by-one errors"
<beneroth>
s/was/as
<Regenaxer>
haha
<beneroth>
well, it's true :D
<razzy>
that one i get :]
<beneroth>
let's just categorize all the estimations in the off-by-one category :)
<razzy>
i do need (gc 0) :D
<Regenaxer>
good
alexshendi has joined #picolisp
<razzy>
human-forced-breakpoint (C-c C-c) is called when exactly? some simple instructions seem to ignore for a while.
<razzy>
*ignore it
<Regenaxer>
right
<Regenaxer>
Signals are checked for *between* primitive calls
<Regenaxer>
(do 10000) will not break
<Regenaxer>
but (do 10000 (inc 'N)) does
<razzy>
simple instructions can take long while, when they are forcing (gc) all the time :]
ubLIX has quit [Ping timeout: 246 seconds]
<Regenaxer>
possible
<Regenaxer>
But not many take long, only loops like 'do', and they are usually not empty
<Regenaxer>
Which instructions take long in your case?
<razzy>
almost not worth mentioning. when testing data performance (setq A (need 10 NIL (range 1 100000000))) nil
<razzy>
Signal check between GC run could help?
<razzy>
idk
alexshendi has quit [Read error: Connection reset by peer]
<Regenaxer>
Yes, a large 'range' is always a bad idea
<Regenaxer>
And no, you cannot break execution inside a primitive
<Regenaxer>
There may be inconsistent states
<razzy>
ah, yes.
<Regenaxer>
Instead of 'range', better use 'make'
<Regenaxer>
And call (gc 400) or so *before*
<razzy>
yes, i do :]
<Regenaxer>
ok
<Regenaxer>
Hmm, well, both 'range' and 'make' are same bad ;)
<Regenaxer>
The bad idea is to build such big lists
<Regenaxer>
In most cases this is not necessary
<Regenaxer>
(though sometimes is, my favourite example is Vip)
<beneroth>
(vip could be improved regarding that)
<Regenaxer>
Not really, we *want* to keep the edited lines in memory, no?
<beneroth>
yes, but maybe not all, just the edited? :)
<Regenaxer>
right
<Regenaxer>
Makes it more complicated
<beneroth>
T
<beneroth>
and vip is intended mainly for pil source code.
<Regenaxer>
For source files, logs, email it is sufficient
<beneroth>
if such a file becomes so big, then you got other problems
<Regenaxer>
exactly
illo has quit [Quit: WeeChat 2.2]
<razzy>
i like the idea of streamed data. vip looks soo cool. shame i like emacs :]
<Regenaxer>
Right, so better stream instead of a big list
<Regenaxer>
eg with coroutines
<razzy>
but big lists are necessary with big evolution algorithms :]
<razzy>
imho
<Regenaxer>
perhaps, yes
<Regenaxer>
or big other structures
<razzy>
big databases maybe?
<Regenaxer>
yeah, but thats slower probably (unless bigger than memory anyway)
<beneroth>
if it needs to be so big, then use DB.
<beneroth>
it's unlikely that you need all this big stuff in memory all the same time. it's surely multiple parts, multiple steps you can process rather independently.
<Regenaxer>
afp
<beneroth>
or divide it further in smaller problems which you distribute over multiple programs.
<beneroth>
cu Regenaxer
* razzy
like idea of infinite list. reminds him of infinite-tape turing-machine