<Regenaxer>
perhaps, or like the 'tr' example in (doc 'pipe)?
<tankf33der>
tr is not portable then
<Regenaxer>
?
<Regenaxer>
you could en/decrypt here
<tankf33der>
tr is unix command, no ?
<Regenaxer>
yes
<tankf33der>
encryption and decryption is picolisp codde
<Regenaxer>
(pipe (in Sock (decrypt)) (rd))
<tankf33der>
aaaa
<tankf33der>
and (pipe (out Sock (encrypt)) (pr))
<tankf33der>
?
<Regenaxer>
if (decrypt) writes to stdout
<Regenaxer>
no (pipe (pr) (out Sock ..
<tankf33der>
ah
<tankf33der>
i see
<Regenaxer>
The first arg to pipe is an exe running in a child
<Regenaxer>
Just experiment a little ;)
<tankf33der>
yea. i will
<Regenaxer>
:)
<beneroth>
yeah plio encryption would be interesting
<beneroth>
one could also inject it into the pil DB, encryption per database block, so its encrypted on disk
<beneroth>
I would like that, e.g. for encrypting user-data tied to the user-login (of course, password reset is then not possible anymore, or only with a escrowed backup key)
<Regenaxer>
I use @bin/ssl normally
<beneroth>
you are talking about transport, as did tankf33der as far as I understood. :)
<Regenaxer>
right
<Regenaxer>
not to files
<beneroth>
I'm talking about encryption on rest (disk). Granted, its not end-to-end-encryption (though that would be than possible, if a pil-DB runs on the client device as e.g. in pilbox), but I think it could be some protection on the server SaaS front. Illegitimate access to stored files on disk is easier than illegitimate access to a program and changing that program to store a password/backdooring the encryption. Similar gov enforcing delivering of files is prob
<beneroth>
ably a lower barrier than gov forcing a company to add a backdoor to their program code.
<Regenaxer>
ok
<beneroth>
at best the encryption would be happening in the plio layer of pilDB, and it should be fast.
<Regenaxer>
I think it is not worth the effort
<beneroth>
I agree for your ERP applications. a trusted server is the best.
<Regenaxer>
yeah
<beneroth>
I argue it might be worth it for shared hosts where data of multiple customers are on the same server, or the server might be in the cloud (no protection against the cloud provider, but some protection against other customers of that cloud provider running on the same node)
<beneroth>
I agree that security-wise the whole concept of shared hosting (multiple users/clients/untrusted actors on the same physical server) is bullshit in the end :)
<beneroth>
the shared CPU Level 3 buffer kills it all :D
ubLIX has joined #picolisp
<freemint>
beneroth encryption for picolisp at pilIO level is an idea i like but will not find useful anytime soon
<freemint>
unless the encryption is asymmetrical
<razzy>
encryptin is not usefull for me neither yet :]
<freemint>
but that i just tied to an particular use case
<freemint>
Am i right to assume that you encrypt the complete dB beneroth?
<freemint>
Also why not encrypt blobs which contain pilIO with the tools provided by the OS that would me more modular and more performant
<beneroth>
I'm not, but I will have some use cases where this might be useful in the nearer future.
<beneroth>
In some contexts you might not have an OS in that sense (e.g. application container on cloud infrastructure)
<razzy>
i hope for end to end encryption channel inside 2 picolisp instances :]
<freemint>
razzy that is called ssh
<beneroth>
encrypting on the level of database symbols (well, better be blocks) in picolisp would allow to only partially encrypt parts of the database, or to re-use existing functionality to replicate/stream/read the database.
<beneroth>
freemint, right
<freemint>
beneroth how you you encrypt pilIO partially? How would the DBGC work then? Is data leaked by index structures?
<beneroth>
I would encrypt the related index blocks too - or not, depending on the specific use case.
<beneroth>
just encrypt a block before it gets written to disk.
<freemint>
Is it encrypted then indexed or indexed then encrypted?
<freemint>
Does the overall pointer structure in the index stay the same?
<beneroth>
indexed then encrypted, though the index might also be encrypted when it is saved.
<beneroth>
yes.
<beneroth>
pointer would be the same.
<beneroth>
well the external symbols, {12} etc.
<beneroth>
all DB operations happen on the unencrypted data.
<freemint>
Ok that allows for some extraction of information of an partially encrypted database
<beneroth>
well if the indices are also encrypted, than you can only say something about the number of records in the database, not more, without decrypting.
<beneroth>
the relationship between records lives within the records themselves (+Link etc) and in the indices - if both are encrypted, the relationship cannot leak
<freemint>
If it is B-Tree you can say way more in particular if you can get app to insert entries at you wish
<beneroth>
the indices in pilDB are B-Trees.
<beneroth>
the attack to linked above does not apply here in my understanding.
<freemint>
ok then i have a bad understanding what you are doing
<beneroth>
I'm not doing it, I'm just thinking about it :)
<freemint>
but in your thread model you should assume that the attacker can get service on insert new
<beneroth>
I'm talking about encrypted files - think TrueCrypt etc. But doing it instead for the whole DB file, doing it for blocks (they're all the same size within one DB-file), so different encryption keys are possible for different database records.
<beneroth>
what do you mean by "can get service on insert new" ?
<freemint>
insert new entries in to the db
<beneroth>
whats the threat? that someone, having a copy of the db files, can insert a new record? that doesn't give the attacker any information.
<freemint>
I can infer slot about a balanced B-tree just by looking how the connections change after inserting stuff
<beneroth>
well you obviously can't edit a B-Tree when it happens to be encrypted.
<freemint>
yes
<beneroth>
if you only encrypt the data objects and not the indices, then yes you can infer some information from the index. no need to edit the index, just reading it is enough.
<beneroth>
I'm not talking about homomorphic encryption.
<freemint>
the threat model I propose is: They can read the encrypted DB and they can talk with front-end over the normal interface can affect the encrypted db that way
<beneroth>
I don't think homomorphic encryption is worth something, unless maybe in some very specific cases. Generally I think most information lives in the connections between records, and this information cannot be hidden by homomorphic encryption as far as I understand it. Some banks started using homomorphic encryption, but this is probably more of an insurance fig-leaf than any real protection.
<freemint>
I am talking homomorphic at all
<freemint>
*not
<beneroth>
well if they have the access credentials of an user, then yes of course they can impersonate that user and encryption is no protection.
<freemint>
I am not talking about homomorphic encryption
<freemint>
lets say I want to check whether abb is in an indexed slot in the database. I get he app to add aba and and abc and look if those end up next to each other in the B-tree
<beneroth>
the scheme would be a protection against someone getting a copy of the stored database, not someone able to watch or even modify the running system prolonged undetected.
<beneroth>
you only get access to the B-Tree if you have the valid credentials, else not.
<beneroth>
yes this means there are no indices which index data from multiple so separated users (separated encryption keys)
lodsw has quit [Ping timeout: 240 seconds]
<freemint>
Is your threat model: They can get one encrypted copy of the DB and they have no way to influence the content in the db?
<beneroth>
thread model: They can get one encrypted copy of the DB
<beneroth>
they can indeed not influence the content of the encrypted db without being able to decrypt it.
<beneroth>
like: you have an encrypted file.
<beneroth>
can you edit it? well you can add stuff or edit part of the encrypted data, but you cannot edit anything which is encrypted.
<beneroth>
does not matter.
<freemint>
Beneroth I am not saying they edit the file. I am saying the talk to application and the application behaves normally and writes stuff to the db
<beneroth>
can you watch the program doing the encryption while it is doing a real and legitimate encryption (or decryption) ? well yes, than you lost, in any case. well hardware encryption modules can be a protection (well then the encryption/decryption-step ought to not be observable), but its questionable if they can be trusted.
<beneroth>
ah, you mean someone who CANNOT observe the application, but can observe the files in real-time during real legitimate actions?
<freemint>
beneroth maybe you can not watch the program but just the file which is read only from that user they control
<freemint>
lets say they get a second copy of the hourly backup
<beneroth>
I now understand what you mean!
<beneroth>
you cannot tell about the content, just that some unknown record has property X because index for X got edited in the same step.
lodsw has joined #picolisp
<freemint>
something like that
<beneroth>
but yes, if the index is split over multiple symbols which don't change if only one branch/leaf is edited, then yes one could maybe interfere some range of values the indexed property must have.
<beneroth>
good thinking freemint
<freemint>
thanks
<freemint>
how might I have said that clearer?
<beneroth>
no idea :D
<beneroth>
its a complex topic
<freemint>
good
<beneroth>
maybe stating more clearly from the beginning which parts the attacker is able to observe and which not.
<beneroth>
but yeah, nvm
<freemint>
I want to play through a scenario which would be interesting to see how your current model behaves
<freemint>
is that fine with you?
<beneroth>
if such an attack would result in practical results is questionable, but good thinking. the scenario would be one of those this scheme should protect against, granted.
<beneroth>
sure
<beneroth>
that's why I'm talking about it here.
<beneroth>
I like my ideas to be questioned.
<freemint>
lets imagine a social network where each user has a name a list of likes and a list of friends
<beneroth>
so a graph of friends with like relations/edges between
<freemint>
the attacker can create many new accounts and befriend people
<freemint>
yes
<freemint>
If attacker1 is friend with alice1 and attacker2 is friend with bernd2 and the attacker now at which offset there structure is stored in the file
<freemint>
can they infere that bernd2 is friend with alice1?
<freemint>
my reasoning that they should is: they can follow the entries in there friend list which still is a list of links which point to encrypted profiles
<freemint>
and those encrypted profile to contain a list of links and you can see whether alice1 list has a link that points to benrd2.
<freemint>
The attacker knows what a link to bernd2 looks like since they have a link to them
orivej has joined #picolisp
<freemint>
can you follow beneroth?
<freemint>
if I understand it correctly that property is equivalent to the DBGC works without knowing anything about passwords
<beneroth>
They cannot. But you are right, DBGC cannot work.
<beneroth>
clever freemint
<freemint>
GC works without being aware there is encryption going on in the DB <=> attacker can infer anonymized meta data about connections
<beneroth>
T
<beneroth>
I didn't think of DBGC not working. right.
<beneroth>
this is a big point. throws the whole thing back to Regenaxers approach of "don't do this overhead, its no protection anyway, just keep your instance under control".
<freemint>
attacker can infer anonymized meta data about connections AND they can create new connections <=> they might deannomyze everybody who is connected to somebody to whom they may be able to make new connections
<beneroth>
yeah. would still protect some data, but probably not be relevant.
<freemint>
roughly
<freemint>
well it makes things harder definitely
<beneroth>
so to be useful (gdbc working) and secure, the scheme boils down to encrypt each database individually and using an individual database per user/encryption-level.
<beneroth>
but then again simple standard file-encryption applies.
<freemint>
I think you would need to throw out cyclic structures ... DBGC with reference counting should be able to work
<beneroth>
thanks freemint. you just convinced me of Regenaxers view in this topic :D
<freemint>
Beneroth I am not convinced Regenaxer's view is right or proper
<beneroth>
T
<beneroth>
central unencrypted counters would work.
<freemint>
even encrypted counters when they encryptable to the GC
<beneroth>
but then again, they leak information about which values exist and how much they're used, so enough to probably guess meaningful data.
<Regenaxer>
I was concerned mainly about the overhead in work
<freemint>
Regenaxer we know
<beneroth>
we discuss if there is a benefit for the overhead cost, or if the benefit is not worth it.
<beneroth>
Regenaxer, your view in my understand is: "generally benefits are in most cases smaller than hypothesised and not worth it, so just keep with KISS damn."
<freemint>
I think the encryption scheme you propose makes a slightly stinger thread model
<beneroth>
stinger?
<freemint>
stronger
<beneroth>
ah yes.
<beneroth>
but overhead and additional code not only reduces performance but also development cost/maintainability
<Regenaxer>
right, perhaps with maintainability the biggest part
<beneroth>
T
<beneroth>
a often undervalued value, because it is a hidden longterm cost with a big upfront cost
<beneroth>
big upfront price
<freemint>
I just wanted to make you think about processing attackers data with bernd2 data in a non trivial way and then encrypting has non obvious consequences
<Regenaxer>
yes, it is in the way during development, testing etc.
<beneroth>
yeah, that is my argument against homomorphic encryption
<beneroth>
that = freemint last message
<freemint>
beneroth glad to help you but I got to do some stuff for university.
<beneroth>
haha, same. Thanks for the discussion!
<freemint>
no problem
<freemint>
I am not sure that argument applies in full strength to homomorphic encryption
<beneroth>
T, my understanding of homomorphic encryption might be incomplete or wrong. But at the moment I cannot imagine it to work against that argument.
<freemint>
I can see ways. But those make some tradeoffs
<beneroth>
of course it increases the barriers for attackers, but not to an implausible level.
<freemint>
Tradeoffs I do want to entertain now but can not responsibly do so.
<beneroth>
Resources might be better spent in having trust-able hardware with proper physical security, if one really requires that level of security.
<beneroth>
ok, then go and do your work!
<freemint>
beneroth there is something to that but on the other hand I am not sure if it is really that simple
<freemint>
the problem is "trustable" hardware becomes very hard in many threat models
<freemint>
have a nice day
<beneroth>
you too
<razzy>
i would be happy with simple mantainable function for end-to-end streamed encryption. you could imagine stored data as frozen stream.
<razzy>
*end-to-end encrypted stream
<beneroth>
that exists. TLS.
<tankf33der>
all you need handshake and chacha
<tankf33der>
both in monocypher
<tankf33der>
lets wait for announce and when handshake will be verification proven
* beneroth
is excited for it :)
<razzy>
the encryption would not have to be strongest in the world :]
<beneroth>
why not?
<beneroth>
btw razzy what are you doing with picolisp? any projects?
<razzy>
learning and chess :]
<beneroth>
okay :)
<razzy>
and distributed OS in future
<razzy>
emacs and MIT-lisp-machine are closest competitors :]
<razzy>
if you have encrypted, distributed, ever-shifting database, you do not need super strong encryption. you just want some honey-pot traps and possibility to increase encryption
<tankf33der>
so "mike" in plio is a list (12 6d 69 65)
<tankf33der>
so "mike" in plio is a list (12 6d 69 6b 65)
<tankf33der>
i need pass this list to chacha
<tankf33der>
encrypto, send to socket and do reverse on other side
<tankf33der>
again question: how get access to list (12 6d 69 65)
<tankf33der>
?
<Regenaxer>
It is not a list
<Regenaxer>
you want a list of bytes?
<tankf33der>
yeap
<tankf33der>
because chacha can work on lists
<tankf33der>
via native
<Regenaxer>
(make (in "x" (while (rd 1) (link @]
<tankf33der>
aaaa
<Regenaxer>
If speed is critical, I would mmap() the file and pass it to 'native'
<Regenaxer>
instead of converting every byte
<beneroth>
if you can convert picolisp number in the native wrapper, you could also use (rd 'num) with a higher number of bytes, kinda using a picolisp number for an array
<beneroth>
right, Regenaxer ?
<beneroth>
mmap would still be even better of course
<tankf33der>
now i can encrypt and send over network (out Sock (pr List))
<Regenaxer>
Good, but I don't understand the second one
<Regenaxer>
Should always be (>> -64 12)?
<Regenaxer>
ah, haha, I'm confused
<Regenaxer>
good
<tankf33der>
just code.
<tankf33der>
show that i want always be in plio on both sides of sockets
<Regenaxer>
ok
<tankf33der>
client will send "mike" in plio and server replies "MIKE", all encrypted and all inside PLIO
<beneroth>
nice!
<Regenaxer>
yeah
<razzy>
is picolisp system more like inhereted tags?
<razzy>
is picolisp system more like
<beneroth>
can't you find out yourself?
<razzy>
i have suspicion, and if somebody already using it, would be efficient if said somebody confirm it, or deny it
<beneroth>
I have no clue what you mean by "inhereted tags"
<beneroth>
don't suck up other peoples time without trying yourself
<beneroth>
trying yourself is more efficient globally.
<razzy>
ok, np
<razzy>
trying yourself is more efficient globally. not always
<beneroth>
always, lack of communication overhead. you can't correctly prise in the opportunity cost of other people.
<razzy>
it is up to them
<beneroth>
disturbing them to notice your message is already a cost.
<razzy>
it is not like i steal busy channel
<razzy>
i could ask stupider questions only if there is no talk in half a day
<razzy>
would it be more acceptable?
<beneroth>
you could ask better question benefiting people.
<beneroth>
compare with better, not with worse.
<razzy>
better questions means more work. there is balance
<beneroth>
aye. My point is the amount of work you put into your questions vs. the amount of work other people pay by reading (and optionally answering) them is off balance. put in some more effort.
<beneroth>
You can do it! Thanks.
<beneroth>
btw (rebuild) behebt das Problem. Ich versuch noch rauszufinden warum es ueberhaupt entsteht.
<beneroth>
read your link properly, it clearly differentiates between "aye" and "aye, aye". (<> "aye" "aye, aye")
<beneroth>
Klugscheisser.
groovy2shoes has quit [Quit: moritura te salutat]
<razzy>
should i read it as that you do not like me?
<beneroth>
I don't like your insubstantial remarks.
<Regenaxer>
yes razzy, it is not personal, but you should think a little before you write
<razzy>
thx
<freemint>
razzy I was in a similar situation as you are a few years ago. A good way to reset the annoyance level is write a neat piece of code without help. Show it to the irc and then vanish for a week or two.
<Regenaxer>
Nice tip freemint :) Though vanishing is not necessary ;)
orivej has quit [Ping timeout: 268 seconds]
groovy2shoes has joined #picolisp
<freemint>
That's the way I didn't annoy the hell out of in 2014/2015
<Regenaxer>
You were really young back then ... 16?
<freemint>
oh my god I started picolisping when I was 15/16 that was a long time ago. 8th or nineth grade
<Regenaxer>
T
<freemint>
the earliest email about PicoLisp I could find was Januar 2015 but I hang around in IRC before I joined the mailing list
<beneroth>
I'm excited to see what you will work on in a few years, freemint. I'm pretty sure it will be awesome stuff :)
<freemint>
oh that is appreciated
<freemint>
to bad the irc logs do not reach that far back
<freemint>
Hey everybody including lurkers what are the oldest logs you have?
<Regenaxer>
Unfortunately we did not know about _whitelogger, or it did not exist yet
<beneroth>
I only have logged some small parts, usually from technical discussions or when something got explained.
<freemint>
oh sorry misread I joined the mailing list on the 22.10.15
<freemint>
beneroth if you are bored could you grep me in the older logs?
<beneroth>
sorry, nothing there except the "better keep picolisp code pure in style"-discussion you had (mainly) with Regenaxer some days ago. Saved that to remind me of the arguments in favour for purity for when I happen to write pil tutorials or thelike
orivej has joined #picolisp
<beneroth>
most logs/notes I have are about pilDB internals :)
<freemint>
I see mhh
<beneroth>
and a longish discussion I had where raydeejy enlightened me about not worrying about hidden advantages of popular languages I might have overseen - they're not there. popularity is a self-reinforcing property disconnected to virtue (after rising over some initial threshold)
<Regenaxer>
very true
<beneroth>
I was troubled if I just miss something about PHP. Now I can calmly look down on it, knowing that it main virtue was only to be more apprehensible than Perl when it was the only real option beside Perl. Nothing more, since then it just lives from its popularity :P
<freemint>
beneroth you might want to think about the following idea: "different things a virtuous for different people"
<freemint>
anyway walking over to my room woi
<beneroth>
that's obvious. No, I was worried that I would overlook something with would matter for me.
<freemint>
(without Internet)
<beneroth>
and I have a collection of links about forth from tankfeeder and joebo :)
<beneroth>
One day I will sort and clean and expand my notes and publish it.
freemint has quit [Ping timeout: 260 seconds]
freemint has joined #picolisp
<freemint>
what is the largest team that ever worked with PicoLisp?
<beneroth>
good question
<freemint>
I mean honestly until PicoLisp doesn't demonstrate it scales beyond a team of 20 people I will still considered it a toy language.
<beneroth>
toy language is more often measured by practical impact, on that scale picolisp delivers without any doubt!
<freemint>
a neat toy but still not something a group of adults can have fun without embarrassing themselves to death
<beneroth>
that I find somewhat insulting, tbh.
<Regenaxer>
toy has nothing to do with the team size
<beneroth>
T
<Regenaxer>
And you can have huge teams still everybody working on small factorized pieces
<freemint>
(that was just an idea that came to my head an I could not take it apart so I posted it here)
<Regenaxer>
In that comparison C is much worse than PicoLisp
<beneroth>
I would agree that picolisp is likely a bad fit for a larger team. but I would also question the sanity of large developer teams.
<Regenaxer>
and see what is written in C!
<beneroth>
true about C!
<beneroth>
and huge teams work with C :)
<Regenaxer>
yep
<beneroth>
I worked so far only with up to 2 people on individual picolisp projects.
<Regenaxer>
I had a max of 7
<freemint>
tell how it went.
<Regenaxer>
(it was TeaTime though)
<beneroth>
went well.
<Regenaxer>
yes, no problem
<freemint>
Regenaxer that sounds interesting... any insights how it scales?
<beneroth>
for working with large teams, you must have some sort of separation of the pieces the individual members work on.
<Regenaxer>
yes
<freemint>
beneroth I am aware
<Regenaxer>
freemint, I did not think much about it
<freemint>
Communication becomes the barrier
<Regenaxer>
higher levels of the project
<beneroth>
Java and C# enforce thus separations, picolisp doesn't, so separation must be produced through team organsiation and some discipline
<Regenaxer>
T
<freemint>
I am aware
<Regenaxer>
I think it is a delusion that eg. Java protects you
<Regenaxer>
The old wrong belief in the advantage of static checks
<beneroth>
freemint, read Fred Brooks "The mystical man month". The core issue with software teams is that complexity rises exponentially with every additional team member - as communication channels between the team members rise exponentially, and so misunderstandings. for software projects, the entire team has to share a common understanding of the vision to implement, and thats hard for pure reasons of communication and the lack of proper terms we often have in pr
<beneroth>
ogramming.
<freemint>
worse the exponentially
<freemint>
it is n!
<Regenaxer>
makes sense
<beneroth>
you can ease this strain on communication by splitting up the project into smaller sub-projects. this has the nice additional benefit that every developer gets their own piece where he/she reigns and is the expert - and then also feels responsible. when code pieces are not clearly associated with a respective person, than nobody feels responsible, and only minimum improvement/bugfixing will ever be done.
<beneroth>
this view is covered by stuff/experiences from others I read about, and also strongly supported by my personal experience. as I'm working as a freelancer/consultant, I have see quite some different companies and teams from the inside.
<beneroth>
though I never worked on a 100+ team.
<Regenaxer>
me neither
<Regenaxer>
The above 7 was the total max
<beneroth>
my former business partner didn, and his tales were pure horror/ridicule
<Regenaxer>
The key is to break a project into sub-project, and those into sub-sub-projects and so on
<beneroth>
in such big teams it's not much about the project anymore, everyone does only the minimal effort to take no blame, and the main activity is politics and intrigues.-
<beneroth>
also because doing more than the bare minimum gives more surface for attack for some other political player in the team.
<razzy>
i agree that ogranisational structure should be defined 1. by problem solved, 2.org-structure of company, 3. org structure of language. if it is enforced by language it is pain to look at.
<beneroth>
whats the largest programming team you worked on, razzy ?
<razzy>
5 people. but i worked at university lab with 100+ people
<razzy>
not on programming tho
<beneroth>
well academia is again quite a different world than big businesses or small hacker teams, I believe.
<razzy>
i think that large universities and large corporations are the same
<beneroth>
naturally overlaps might occur. but the main incentives are quite different.
<razzy>
only universities have longer time view
<beneroth>
I believe average people in academia work much more actively on their own careers than average people in enterprises
<razzy>
ussually
<freemint>
I got another question: Let's suppose a crypto millionaire finds PicoLisp neat and offers us an amount we can not refuse. What is the most interesting project (which makes back our salary) we could tackle with 20 people from the IRC?
<freemint>
oh and we are founded over a 2 year period
<beneroth>
we can refuse any offer, we one of the most stubborn bunch in the wide sea of FOSS/programming communities, I would believe :P
<beneroth>
and with 20 people and 2 years we can easily tackle multiple projects, I'm confident.
<beneroth>
though one should not forget, the code is the smallest part of a finished product.
<beneroth>
way more time goes into polishing, building an interest and network of customers etc.
<razzy>
beneroth: academia and enterprise work ethics imho depends on internal incentive structure
<freemint>
oh yeah and we got enough budget to hire people for non-programming tasks like bookkeeping, marketing...
<razzy>
cannot say where people work more
<freemint>
I mean honestly what projects could we do ~500 man months
<beneroth>
razzy, only partly, on a bigger scale they mostly depend on harsh realities of pre-defined systems. most people need a stable income, that requirement is way bigger than the internal incentive to not work for an organisation which does some questionable things. or well, yes you are right, but from those internal incentives the "having a salary" is the main driver.
<beneroth>
freemint, well I want and sort of have to (internal drive) to implement my vision for a flexible database (and second vision for a flexible easy-to-use web app builder-system).
<beneroth>
freemint, we could do pilOS. that would easily need many people and much time :)
<beneroth>
reviving the lisp OS days, where EVERY part of the software on a computer can be inspected and changed.
<Regenaxer>
right beneroth, PilOS would be the most interesting one
<freemint>
Regenaxer if you were given enough money to hire 20 picolisp programmers at average IRC quality at the condition that you end up with a sustainable profit turning enterprise after 2 years. what would that company work on?
<beneroth>
which gives some nice useful properties, but well its questionable if a good enough business case can be found.
<freemint>
and what would you do with PilOS?
<Regenaxer>
Embedded controllers
<beneroth>
same
<Regenaxer>
But I doubt the profit
<beneroth>
and secure web servers.
<freemint>
but I doubt the secure
<Regenaxer>
:)
<beneroth>
smaller code base is easier to keep secure :)
<freemint>
a dynamic language like picolisp is impossible to verify
<beneroth>
depends on what you mean by verify
<beneroth>
also the most powerful verify tool we have are testing oracles.
<Regenaxer>
Non-trivial setups are never verifyable
<beneroth>
smart fuzzers, basically.
<razzy>
university benefit from cheap-student-labour. corps benefits from looser law constrains about how to use money(corps are more agile with money, shorter response time). how do you motivate people is completely case by case dependent of culture, society. money motivates only if people do not have things to eat and where to sleep.
<Regenaxer>
freemint: Again the false belief in static advantages
<beneroth>
Regenaxer, T
<beneroth>
razzy, well from whose money are you living currently?
<Regenaxer>
Only the simple obvious parts are checked statically
<beneroth>
T
<beneroth>
often testing code (unit test etc) are mainly an alibi work.
<Regenaxer>
And worse, people then tend to rely on that
<beneroth>
T
<Regenaxer>
after it passed the compiler it is fine
<beneroth>
it even gets trickier when you cannot be sure if the compiler is to be trusted :)
<Regenaxer>
exactly
<Regenaxer>
you often canno
<Regenaxer>
t
<freemint>
I would not call SeL4 trivial and they had mathematically verified embedded kernel which they verified all the way down to assembly and later mathematically verified a small arm core it was running on
<Regenaxer>
we had trouble with some optimizing compilers
<Regenaxer>
Depends *what* is verified
<beneroth>
freemint, agreed, but most tests are not so thoroughly
<Regenaxer>
a system interacts with the real world
<razzy>
freemint: most sure option to work on would be web autonomous marketplace.
<beneroth>
I believe the organisation doing the best/most tested software work to be NASA, for decades. but that comes from that even when it costs huge amounts to do proper testing, its still cheaper than when the whole product fails in space.
<Regenaxer>
yep, and it was assembler all those years
<beneroth>
razzy, wtf should "web autonomous marketplace" mean?
<razzy>
and develop PilOS as side product
<freemint>
they proofed that that the model of the kernel provided full guarantees of isolation and scheduling. Later they proofed that there implementation was equivalent to the model and the assembly equivalent to the implementation.
<beneroth>
well they surely still had a lot of assumption about the hardware :)
<freemint>
other than cosmic rays flipping bits in ram
<beneroth>
yep
<beneroth>
or someone executing rowhammer :D
<Regenaxer>
And a kernel is a well-understood application
<freemint>
this was one of there most shaky assumptions
<freemint>
a Webserver is not?
<beneroth>
its a rather simple thing, compared to most business applications
<beneroth>
no, its not
<Regenaxer>
right
<Regenaxer>
kernel and web servers are trivial compared to that
<beneroth>
webservers should be tolerant about clients having mistakes or big errors in their HTTP request
<Regenaxer>
to applications modelling big company processes
<beneroth>
T
<beneroth>
and webservers are already pretty hard :P
<Regenaxer>
yeah
<freemint>
ashbys law?
<razzy>
beneroth: there was some product with legal darkweb distributed marketplace. i forgot the name
<razzy>
i think it died
<beneroth>
the hardest part about kernel is not the kernel itself but hardware drivers.
<beneroth>
razzy, "autonomous" and "web" together don't make sense
<razzy>
yeah, my bad
<beneroth>
most people are not interested on dark web, so you ignore a lot of potential customers
ubLIX has quit [Quit: ubLIX]
<beneroth>
freemint, yes, ashby's law!
<beneroth>
exactly that is the problem
<beneroth>
whats the difference to Goedelisierung?
<freemint>
a model system or controller can only model or control something to the extent that it has sufficient internal variety to represent it.
<beneroth>
ah what I mean is probably called "Goedel numbering" in english
<beneroth>
"The second incompleteness theorem, an extension of the first, shows that the system cannot demonstrate its own consistency. "
<razzy>
sorry, cannot find good example. and i need to do something usefull
<razzy>
software realtime control systems are also good buisiness
<freemint>
what the hell are software realtime control systems
<beneroth>
from a business perspective, the incompleteness theorem applied to software effectively means that completely tested/proven software costs much more than just producing a software which for the main cases seems to work without being completely tested.
<freemint>
why do think they are good business?
<beneroth>
are you talking to me?
<freemint>
nope razzy
<beneroth>
ah
<beneroth>
I see now
<beneroth>
well real-time systems in the context of software/OS usually means a system which gives some guaranteed fast and stable performance, and in which one component (e.g. process) cannot delay another one (within pretty small tolerances), afaik
<freemint>
beneroth what you just said is: "You can prove everything out of itself. Once you get sufficiently powerful you need to make assumptions at some point.
<beneroth>
but I have the impression razzy did not speak from experience or first-hand knowledge but just from some hearsay or whatever. Sorry razzy, but you haven't yet showed us what you got.
<beneroth>
freemint, cannot prove
<razzy>
freemint: what beneroth said. and by software i mean that you solve reliability on software part. you use i.e. 3 commercial raspberrypi (2 are as backup) and you test and swith them in software.
<beneroth>
and a unproven but functional system is in most cases a lot cheaper than a proven software
<beneroth>
that's why we cannot have nice software :)
<freemint>
if you want high reliability do not use the same hardware with chips from the same fab. These are common cause failures
<beneroth>
razzy, ah distribution. which adds A LOT of complexity and additional failure modes.
alexshendi has joined #picolisp
<freemint>
I bet you were even planning to use the same compiler too
<beneroth>
well clang hates us :P
<beneroth>
(well clang hates pil32)
<freemint>
that is closer to the truth
<freemint>
also the linux kernel is VLA free by now
<beneroth>
razzy, check out https://jepsen.io/analyses to get some ideas why distribution is not the silver bullet it seems to be at first
<beneroth>
freemint, I tend to believe that it makes sense for kernel, because in kernel you want to keep the stack small usually anyway
<beneroth>
but that's quite a different thing than straight out banning the compiler from be used with a big subset of possible VM implementations
<freemint>
I never understood why pil32 needs VLAs
<beneroth>
it fits the assumption that in C/C++ re-size-able stuff usually lives on the heap
<beneroth>
freemint, I don't know the exact reason, but basically I think to grow its stack/heap easily
<beneroth>
functionality which is available on ASM level, but not on C level without VLAs
<freemint>
why not use a tree or a skip list instead of an array?
<beneroth>
its doing that, but how to extend the tree/list ?
<beneroth>
without copying the current content around?
<razzy>
i will stop talking for a while :] my leaving agruments are: software-solved realtime reliable system build on cheap commercial parts have big potencial on money saves. it was not done because everybody use wrong software to do it. not simple enough
<beneroth>
OS/kernel already has the mechanics to map fragmented memory into a continues memory space for the application. it's a bit silly to do the same once more within the application.
<razzy>
picolisp with 500man/months could do that
alexshendi has quit [Ping timeout: 260 seconds]
<beneroth>
the whole IT business is at the moment (and for a few years now) based on building big expensive parts and then sell slices of it. thats basically the cloud and shared hosting (same thinking as mainframes). I agree that it would be better to use small optimized hardware pieces, fully separated systems. but claiming it is a better business model is a really big bet when all the smart dudes do something else.
<beneroth>
whole Oracle, Microsoft, IBM are on that train. it's a very big train.
<beneroth>
Google anyway, though they are mostly not really in IT business.
<tankf33der>
adv bureaue!
<beneroth>
T
<beneroth>
ah I forgot Amazon. the leading player in that game.
<razzy>
beneroth: and i am living from my debth, which as fun as it sounds
alexshendi has joined #picolisp
<beneroth>
razzy, yeah see, and when you finished your studies you will probably "want" to work somewhere which covers your livelihood and your debts.
alexshendi has quit [Ping timeout: 260 seconds]
alexshendi has joined #picolisp
<razzy>
propably, or die from unfairness and exhaustion
alexshendi2 has joined #picolisp
<beneroth>
not so likely, unless you develop a serious physical or psychical condition.
<beneroth>
humans are pretty resilient and can take a lot.
alexshendi has quit [Ping timeout: 252 seconds]
* beneroth
likes to remind everyone who is starting with pil DB to not overwrite the T method (constructor) of an +Entity class. just don't do (dm T ()) ever, You may customize it using (dm T @ ... (pass super)), but if you aren't sure how to do that properly better leave it.
<freemint>
beneroth I did that mistake in the past ... surprisingly nothing worked after that
* beneroth
just stumbled over some very old code of his where he replaced the T message of an entity. it's horrible. stupid stupid. bad influence of other languages.
<beneroth>
hehe, yeah, you can easily destroy the creation of objects that way
<beneroth>
coming from other languages you easily want to make a custom T method to handle a predefined parameter list, instead of using named parameters as in (.. 'prop value), but that makes (request) unusable with that entity, and thats a big big loss.
<beneroth>
I also have to say that (request) is one of the basic DB functions in pil DB I miss the most when working with other databases.
<freemint>
can you describe what you like with it?
<beneroth>
doing a reliable transactional (atomic) "create this record if it doesn't exist) is surprisingly hard to do correctly in SQL
<beneroth>
e.g. for data deduplication.
<beneroth>
well, mainly for data deduplication.
<tankf33der>
Regenaxer: here ?
<tankf33der>
i dont know how to convert list (8 0 8) back to number 1024
<tankf33der>
eh.
<beneroth>
how did you convert 1024 to (8 0 8) in the first place ? :O