<freemint>
a list of chars is 31 times as big as the file it was drawn from
<Regenaxer>
I don't understand what you mean
<Regenaxer>
If you mean with "drawn from" operations like (line) or (till), it is 64 times as big
<yunfan>
afternoon :D
<Regenaxer>
morning :)
shpx has joined #picolisp
<freemint>
Regenaxer, i load a file via (nil (setq *Unfil (in "datasets/google/lilianngoyice.mbox" (till))))
<freemint>
and divide the ram usage by the loaded file size
<freemint>
(I also check that till loads the file completly)
<freemint>
F.E. 198 mb of ram vs 63,61130 mb file
<beneroth>
Good morning all
<Regenaxer>
Hi beneroth
<Regenaxer>
freemint, why do you load the whole file into memory?
<Regenaxer>
I would not do that
<freemint>
to see when it breaks
<Regenaxer>
ah, ok
<freemint>
me neither but you got to know when you break
<beneroth>
under realistic loads ;)
<aw->
hi all
<beneroth>
hi aw- o/
<aw->
freemint: wouldn't it be better to convert the mbox to Maildir?
<aw->
i remember back in the day, mbox was such a pain.. particularly when it gets corrupted
<freemint>
I have not seen much about Maildir ... maybe possible but i do not see why
<freemint>
yeah
<aw->
Maildir puts each email into its own file
<aw->
avoids many problems
<aw->
no locking issues
<Regenaxer>
I feel the mbox format is fine
<Regenaxer>
Simply delimited by '\nFrom '
<aw->
it's fine except when you have 30,000 emails and the file is multiple GB
<aw->
with Maildir it's incredibly easy to move the oldest mails to archive storage
<Regenaxer>
true
<aw->
with mbox you need to trim the file from the top.. while it's not being accessed/locked/etc
<freemint>
i will read the file and delete it
<aw->
freemint: what are you making btw?
<cess11_>
It's fairly easy to import in chunks by incrementing a global or object field to keep track of where one was last time the file was open.
<freemint>
reading mbox, storing things in a database make it a mail archive
mario-go` has joined #picolisp
<cess11_>
Though I agree that mbox is a horror.
<cess11_>
Small ones are OK but once they get huge they get troublesome. Especially if someone sent in mail where sentences on new lines start with "From ".
<Regenaxer>
Yeah
<Regenaxer>
I keep my own mails in mbox files, but not too big ones
<Regenaxer>
going back till 1994
mario-goulart has quit [Ping timeout: 240 seconds]
<Regenaxer>
So I can easily edit them, or grep through them
<Regenaxer>
one file per contact, broken down by years if they get large
<aw->
wow nice
<aw->
cess11_: correct, you need to store the locations
orivej has joined #picolisp
<aw->
probably good for freemint to implement a db that knows where in the mbox is located certain things
<aw->
oh, you delete the file afterwards? so just parse once?
<beneroth>
yeah good idea, instead of copying the content into the DB, store the location/offset
<freemint>
aw-, nah i put each message in blob
<aw->
SQL database?
<aw->
or pil db?
<freemint>
why? you can index it way easier ... i can compress base64 escaped data in to binary data - pildb ofcourse
<freemint>
beneroth, why store the data location in addition to the data
<beneroth>
not in addition, instead of. but blob approach might be good, too
<aw->
thinking out loud here ;)
<rob_w>
so, freemint , do you know hagl or any biogas stuff ?
<freemint>
I saw their webserver online
<rob_w>
ok
<freemint>
some biogas stuff
<freemint>
(like the controller website)
<freemint>
i know it from seeing it online ...
<freemint>
rob_w, why do you ask?
<rob_w>
Regenaxer told me about that you found something, so i wondered if you where related or even a customer of mine ;-)
<freemint>
No i used shonan to look up picolisp servers online
<rob_w>
shonan ?
<freemint>
shonan.io a port scanning service which allows to search it's result
<freemint>
(they scan the ports routienly and store the results)
<freemint>
(like google indexes html)
<beneroth>
search engines for hosts. good to find IoT devices to infect, unprotected Application servers and thelike. :D
<beneroth>
and usage statistics
<freemint>
rob_w, why was pil chosen for process control?
<rob_w>
i use it as "cheap n fast" webserver for either headless controls or simple views
<freemint>
how did you convince them to use it?
<rob_w>
i am the decision maker ;-)
<rob_w>
so our customers take what we give am , as long as it works 24/7
<freemint>
ah so "selbständig"
<rob_w>
also i ve a good support contact for any pil or even c-related problems M-)
<freemint>
me too
<Regenaxer>
exactly! ;D
<rob_w>
^ ^ hehehe
<freemint>
then again why pil?
<freemint>
rob_w, why choose pil for "cheap n fast" webserver
<rob_w>
differnent reasons, i had some bash cgi driven stuff , but that was sooo ugly when it grew
<rob_w>
then i talked to alex and switched to pil, more powerfull for library integration ( had some own c-libs which i integrated)
<rob_w>
so the webserver now has a direct hw access , ssl support and pil is so leightweight, as those are all embedded linux boxes
<rob_w>
some had very little ram and cpu contraints etc
<freemint>
i will make a presentation about picolisp ... so i try to get diverse views so i can give a good overview
<rob_w>
also alex did some work directly at some projects ( as we did collaborate on other projects before)
<rob_w>
my key points where , leightweight compile-time requirements, leightweight run-time concerns, powerfull library extensions and long-years of expierence of the project
<rob_w>
cross-compilation was easy , deployment to the field is even more easy and the active comunity is a must for our products
karswell has quit [Remote host closed the connection]
bruno1 has joined #picolisp
<beneroth>
tankf33der, awesome
aw- has quit [Quit: Leaving.]
bruno1 has quit [Ping timeout: 268 seconds]
freemint has quit [Ping timeout: 240 seconds]
freemint has joined #picolisp
freemint has quit [Ping timeout: 248 seconds]
freemint has joined #picolisp
mtsd has quit [Remote host closed the connection]
rob_w has joined #picolisp
<Regenaxer>
What do you guys think of Mastodon?
<Regenaxer>
I'm getting fed up with twitter
<beneroth>
how so?
<beneroth>
afaik being fed up with twitter is the main reason people switch to Mastodon. Which means slowly all the people/behaviours which made people switch to Mastodon are also showing up there :)
<Regenaxer>
Too noisy, too much ads
<beneroth>
ah this might work better on Mastodon
<Regenaxer>
The people are all right
<beneroth>
afaik Mastodon is implemented on top of GNU social, but incompatible with it (superset)
<Regenaxer>
yes, seems so
<Regenaxer>
Anybody tried it?
<Regenaxer>
I'm now registering
<beneroth>
bbl
<Regenaxer>
cu
bruno2 has joined #picolisp
<Regenaxer>
Wow, cool! clacke[m] *is* in mastodon :)
bruno2 has quit [Quit: WeeChat 2.0.1]
<clacke[m]>
I have been on the Fediverse for a decade
<clacke[m]>
Mastodon is just the latest node implementation for it.
<clacke[m]>
GNU Social, Pleroma and a few others implement the OStatus protocol, as does Mastodon, but Masto is doing a bit of embracing and extending.
<clacke[m]>
Lately Masto started supporting ActivityPub too, a newer less layered protocol which does the same thing but different, and allows private messages across nodes. Pleroma will soon support it too.
<clacke[m]>
once pump.io implements ActivityPub, it will be able to talk to Pleroma and Masto too. AP was based on the pump.io protocol, but pump needs to catch up with the new spec.