<aeth>
If you're handling arbitrary user input and you don't particularly care about performance, then use read-line in a LOOP and parse it line-by-line. If it's interactive and you can't hang, you have to use READ-CHAR-NO-HANG and reinvent READ-LINE, unfortunately.
<White_Flame>
seok: there's also the reader macro #. which evaluates arbitrary expressions at read time
<aeth>
well that's just *read-eval* to NIL before calling READ
<White_Flame>
that can be disabled with ... yeah
<aeth>
(I say READ-LINE isn't the performance-oriented way because the fastest way to handle input would probably be regex on READ-CHAR or even READ-BYTE, but I don't think existing regex libraries support this)
<aeth>
(READ-LINE does have the problem that your user input could give you extremely large lines, so it's somewhat similar to the issue with interning, too)
<seok>
aeth don't quite understand. Is there a security risk in using READ over READ-LINE?
rumbler31 has quit [Ping timeout: 258 seconds]
<White_Flame>
making reading safe in general is quite large task with many modes of failure that would need to have thresholds specified and errors handled
<White_Flame>
and of course would likely significantly affect performance
<aeth>
seok: READ can run arbitrary code with #. (but that can be addressed) and can do some other undesirable side effects, like interning too many symbols until you run out of memory. Invalid syntax probably won't be handled gracefully, either.
<seok>
Yeah, it seems thats the gist of this
<White_Flame>
imagine simple READing from a character stream that infinitely outputs #\A characters
<aeth>
seok: Most of the time you're dealing with user input, you want READ-LINE and something like regex unless it's a multiline input (even then you could just collect lines into lists or something), but there's one issue I can think of there which is that the user could give you a line larger than memory or somethingl ike that.
<aeth>
(i.e. infinite #\A's with no #\Newline)
<seok>
I guess it is safer to take strings only and check for validity?
<aeth>
The fully robust way would be READ-CHAR or READ-CHAR-NO-HANG, but then you're mostly on your own without much help
<aeth>
READ-LINE is normally good-enough
<White_Flame>
it all depends where this info is coming from. file, socket, interactive user typing
<seok>
I imagined DDOS-style attacks would be handled by the limitations of HTTPS connection or CDN
<aeth>
Oh, and you might want to READ-BYTE, too, especially if it's some encoding that's not Unicode.
<White_Flame>
but yeah, everything above the single character reading functions are conveniences, and not all-emcompassing security sandboxes
<seok>
The user can't send string or stream of infinite length though, POST and GET requests don't allow that
Necktwi has quit [Ping timeout: 258 seconds]
<seok>
but it seems there are other hard to handle risks too
xkapastel has joined #lisp
<aeth>
Well, if it's line-oriented and there has to be either an EOF or a newline, then READ-LINE will work.
markong has quit [Ping timeout: 264 seconds]
<aeth>
I actually have thought about this problem and am working on a general solution that uses READ-CHAR or READ-CHAR-NO-HANG (and probably could be extended to use READ-BYTE) and an alternative to regex to parse input based on rules. Naturally, this is harder than it looks so I haven't gotten around to finishing it yet.
<aeth>
Since I hate the idea of using intermediate READ-LINEs and then running regex on it
<seok>
Actually it seemed pretty hard to me too xD just wondering if there was a solution
<aeth>
But READ-LINE does work if you don't want to waste 30ish hours yak shaving a custom READ-FOO macro.
<aeth>
At the cost of it looking like Perl and not being elegant :-p
<seok>
Is the reason you are running regex on each line
<seok>
to forbid evaluations?
thetabit has joined #lisp
<aeth>
(loop :for line := (read-line input nil nil) :while line :do ...)
<aeth>
And, yeah, your :do either parses the line manually with stuff like POSITION (and maybe e.g. PARSE-INTEGER), or it uses a regex library
<aeth>
Or if it's HTML/XML/JSON/etc. you pass it to a library that accepts input streams
<aeth>
I guess once you get past the HTTP header or however this works, I usually use web frameworks
Necktwi has joined #lisp
<seok>
Why would we want to build handler for JSON inputs?
<seok>
there are already json decode libraries?
<aeth>
heh
<aeth>
I meant if you have JSON, you don't use READ-LINE, you pass it off to a library, but... I'm not a big fan of the existing libraries last time I did a survey of the options, so I personally would write my own
<seok>
Right
<seok>
Yeah, I see what you mean. The json libraries are not really consistent with each other
<seok>
I just do with what we've got tho xD
<aeth>
Well, they mostly don't round trip
<aeth>
You have to make some sacrifices to round trip that no library when I checked a few years ago was willing to make.
<seok>
There isn't a clear winner amongst them, bit of a curve for newbies
<aeth>
If the library is poorly thought out, NIL<->false vs. NIL<->{} vs NIL<->[] is a major issue since NIL is also the empty list, and the library might prefer lists for [] and plists/alists for {}
<aeth>
Some might even do NIL<->null which is absurd because NIL is false, not null, despite the name.
<White_Flame>
:null, :true, :false, (json array), hashtables for {}, makes a fair bit of sense
<seok>
Ah that's difficult
<aeth>
The solution imo is to do NIL<->false and use a prefix for plists/alists/lists
<aeth>
somethingl ike this maybe: :null, NIL, T, (:json-array ...), (:json-object ...)
<seok>
That's probably right, since nil is not intended represent empty lists for most json
<aeth>
But that's not quite sufficient, either, since you want the user to decide between hash tables, plists, or alists for JSON objects.
<aeth>
You also want the user to specify ANY sequence for JSON arrays, not just keyword-prefixed lists (still need to disambiguate with NIL) and/or simple-vectors
<White_Flame>
having those options is difficult, because if you have nested objects, the consumer for the parent object might want arrays as lists, while the consumer for the child object might want arrays as CL arrays etc
<aeth>
So, I mean, obviously it won't quite roundtrip since more than one CL thing could become one JSON thing, unless you restrict the encoding to just one of the options
<aeth>
White_Flame: Right, to be fully general you need a schema
<aeth>
Or a way to select which sequence while parsing at runtime
<White_Flame>
you're always going to need a translation layer. It's nonsensical to canonically have JSON decode into idiomatic lisp, or lisp canonically encode into idiomatic json
<White_Flame>
so an unambiguous representation of JSON in lisp is the intermediate translation layer
<aeth>
Yes, but I'm okay with :null/T/NIL instead of :null/:true/:false
<White_Flame>
how do you give the json encoder an empty list?
<aeth>
You don't.
<White_Flame>
heh
<White_Flame>
well I mean an empty json array
<aeth>
(cons :json-array your-list)
<aeth>
If that's empty that becomes '(:json-array)
igemnace has joined #lisp
<seok>
White_Flame don't believe there is a good solution as long as nil === '() in lisp implementations
<aeth>
Technically, this conses so you might want a way around it
<aeth>
(Of course, the way around it might just be to give a simple-vector and hash-table instead of a list and a plist/alist)
<White_Flame>
seok: well, in my preference, anything that's listp should encode to a list
<White_Flame>
(array in this case)
elosant has quit [Quit: WeeChat 2.8]
<aeth>
White_Flame: And this is why there are so many JSON libraries, and I'll one day add another
<seok>
then how do you represent false in json?
<aeth>
Because people have like 10 different preferences when you combine all of the different issues, and they're all valid, and defaults matter so people will want one with the "correct" defaults
<White_Flame>
seok: the atomics would be the unambiguous keywords :null, :true, and :false
elderK has quit [Ping timeout: 256 seconds]
<aeth>
See, I don't like that. I'd prefer using T and NIL and complicate the representation of JSON arrays/objects
noface has joined #lisp
noface has quit [Client Quit]
<aeth>
Obviously :null is unavoidable unless you do NIL<->null which some libraries do and that's the least correct answer, maybe the only one that's objectively wrong.
<seok>
To accommodate tastes, the library should provide options for those translations then?
<White_Flame>
seok: and then that gets into the nesting problem
noface has joined #lisp
<aeth>
seok: The thing about software is that defaults matter. Most people will only ever use the defaults.
<seok>
What is the nesting problem?
<White_Flame>
where different nested JSON objects ideally want different translation options
elderK has joined #lisp
<seok>
Why would one want that?
<White_Flame>
because different code handles different objectrs
<White_Flame>
and the natural representation in lisp might differ between them
<seok>
Shouldn't one design their application so translations are consistent?
<seok>
hm
<White_Flame>
that would pierce abstractions
<aeth>
I mean, the real solution is to switch to .sxp and just use s-expressions
<seok>
aeth I'd like that
<aeth>
Here, you really do need to use "true" and "false", though, as a sort of "nobody wins" compromise between different Lisp dialects (e.g. Scheme's #t and CL's T)
<seok>
json -> lisp is no problem since json has both nil and [] right?
<White_Flame>
json's null and json's [] are separate. those 2 concepts are equivalent in CL
<aeth>
JSON is mostly oriented toward scripting languages that have, more or less, the same core data structures, like JavaScript, Python, Lua, etc. The mapping's pretty obvious there.
<seok>
and for lisp -> json the programmer can set the setting for each type of object no?
<aeth>
Languages with rich type systems like Common Lisp have more of an issue
<aeth>
e.g. CL has a whole numeric tower, JS just has a double. (well, JS finally added integers recently iirc)
<White_Flame>
and JS isn't JSON
<White_Flame>
JSON is much more limited
<seok>
but the problem here is actually that lisp has limited representation of '() and nil
<seok>
not that lisp has rich type system
<seok>
in lisp '() and nil are same concept, whilst in most other languages they are not
<aeth>
but that's only if you want [] to become a Lisp list
<aeth>
There's literally no way of knowing what the user wants [] to turn into.
<seok>
What other than an empty list would the user want to turn [] into?
<seok>
[] in json I mean?
<aeth>
The user probably wants a list or simple-vector, but maybe also e.g. (simple-array octet (*)) which is a 1D array of octets
<seok>
White_Flame mentioned empty array, that's one I guess
<aeth>
But imo there's basically no cost to prefixing a list.
<seok>
But making [] into an lisp array does not make sense
<aeth>
To write JSON, you'd just CONS :json-array and to read JSON, you'd just CDR the prefixed list
<aeth>
seok: why not? For all you know, [] represents a matrix and so you want it to become #2A()
<aeth>
People use JSON for everything.
<White_Flame>
to write JSON, you'd also have to know which values are intended to be json arrays
<seok>
what's the size of that array? 0?
<White_Flame>
and where that knowledge has to be is a weird situation
<aeth>
seok: no, it's (0 0) instead of 0
<aeth>
#() is 0
<White_Flame>
eg who is the you in "you'd just CONS up the thing"
<aeth>
(array-dimensions #2A()) => (0 0)
<aeth>
Actually #() is length 0 and array-dimensions (0) so it depends on how you approach it
<seok>
JS calls lists arrays, but they are actually lists, lol
<White_Flame>
they're sequences
<aeth>
White_Flame: and as I said, they're not necessarily sequences, since [] could mean #2A() if the context is matrices
<White_Flame>
(and really, they're just a shorthand for an object with implicit pseudo-numeric keys)
<White_Flame>
aeth: I meant json [] is a sequence
<White_Flame>
not a "list" or "array" necessarily
<seok>
but aeth, there isn't actually a way to represent arrays in JSON
<seok>
the parser / decoder for that array should be built into the application by the programmer
<seok>
how do you distinguish (0 0) and (0 0 0)
<aeth>
JSON has no Schema so it's possible [] is just short for [[],[]] in an API that otherwise expects e.g. [[1, 2, 3], [4, 5, 6]] to #2A((1 2 3) (4 5 6)) which is particularly problematic in CL where you get real multidimensional arrays
<aeth>
and a naive JSON parser would just give you '((1 2 3) (4 5 6)) which is nested lists...
<aeth>
seok: You don't distinguish (0 0) and (0 0 0) in my JSON example without knowing the context that the JSON is encoding matrices, which are by definition 2D
<seok>
one loses information of the array structure once it is in json, so there shouldn't be a native encode/decoder in aa library
<seok>
That's what I'm saying, that's application specific
<seok>
a library should handle general cases
<aeth>
Yes, it would be perfectly fair if [] was required to become a sequence (but anything more specific is too specific imo) and then the user would just have to turn that sequence into a 2D array manually
<seok>
yeah
<aeth>
Not efficient, but general enough
<White_Flame>
aeth: that also brings up the nesting issue of (:json-list (:json-list 1 2 3) (:json-list 4 5 6)) where the elements of the list aren't raw lisp objects, but a json intermediate format still
<seok>
It's a limitation of using JSON. one should not use JSON if that kind of performance is an issue
<seok>
but I guess what you are saying in writing your own JSON reader makes sense
<aeth>
White_Flame: That's fine for me. You can have a postprocessing function that removes the prefixes at the expense of reintroducing the NIL ambiguity
<aeth>
Or you can just have the user deal with it
<seok>
the next standard of lisp might want to have objects to represent [] and false only alongside '() to be able to communicate with other modern languages better
<seok>
?
shukryzablah has joined #lisp
<aeth>
but then you have two falsey objects and open up that can of worms
<seok>
[] = '() = false but [] != false?
<seok>
(= [] nil) => nil, (equal [] nil) => t?
<seok>
I dunno : P
johntalent has joined #lisp
vornth has joined #lisp
FennecCode has joined #lisp
shifty has joined #lisp
satousan has joined #lisp
EvW has quit [Ping timeout: 256 seconds]
stepnem has quit [Ping timeout: 264 seconds]
stepnem has joined #lisp
vornth has quit [Ping timeout: 240 seconds]
Kaisyu7 has quit [Quit: ERC (IRC client for Emacs 26.3)]
lucifer_h has quit [Quit: Connection closed for inactivity]
Kaisyu7 has joined #lisp
bitmapper has quit [Ping timeout: 246 seconds]
cosimone_ is now known as cosimone
space_otter has joined #lisp
dominic34 has joined #lisp
karstensrage_ has left #lisp ["Leaving"]
dominic34 has quit [Ping timeout: 240 seconds]
shukryzablah has quit [Quit: ERC (IRC client for Emacs 26.3)]
Oladon has joined #lisp
thetabit has quit [Ping timeout: 260 seconds]
torbo has quit [Remote host closed the connection]
ahungry has joined #lisp
isakovic has joined #lisp
isakovic has quit [Client Quit]
tucansam has joined #lisp
tucansam is now known as toucansam
toucansam is now known as tonythetiger
EvW has joined #lisp
jesse1010 has quit [Ping timeout: 240 seconds]
john__ has joined #lisp
gaqwas has quit [Ping timeout: 256 seconds]
nicktick has joined #lisp
satousan has quit [Quit: WeeChat 2.3]
<beach>
Good morning everyone!
<stylewarning>
beach: what are you working on
<beach>
Today, on a presentation for the online Lisp meeting.
<beach>
Also a specification for the extraction of first-class global environments to a separate library.
<beach>
And of course SICL bootstrapping. The other day I was able to load a large part of the portable condition system.
<beach>
stylewarning: You?
<stylewarning>
beach: usual quantum compiler stuff, how to specify and practically use a broader class of (quantum) compiler targets. Also thinking about trying to hack on Coalton a bit this evening; I still don’t understand everything I should about Lisp compilation, the environment that exists in different phases of compilation and execution, externalizable objects, etc
<jgodbout>
any interesting QIT papers catch your attention and help?
<beach>
stylewarning: Yeah, that's non-trivial stuff. I am working on that at the moment for the first-class global environments.
<stylewarning>
[the arxiv version exists but is a bit different and a bit lighter; can email the full one]
<jgodbout>
My university directs me to the arxiv /sigh
<stylewarning>
jgodbout: i can chat in detail about the quantum stuff in #qlisp, but the main idea is that compiler targets in quilc are somewhat limited; I want to be able to write efficient compilers for elements in SU(2) using an arbitrary (parametric) generating set, perhaps specified by a collection of rotations about off-axis vectors
<stylewarning>
jgodbout: there is a modicum of trouble in the compilation (math), but more trouble right now fitting it into the quilc framework, which really relies on things being canonically named
<stylewarning>
e.g., "RX" specifically and always refers to the the map theta -> {SU(2) elements rotating about x-axis}
<stylewarning>
jgodbout: (if you want to make a great contribution that's relatively self-contained, I would *LOVE* to see an implementation of Solovay-Kitaev or a better derivative, even though it's not always practical. A few folks have tried to implement it but have uniformly failed.)
Bike has quit [Quit: leaving]
tristero has quit [Ping timeout: 260 seconds]
toorevitimirp has joined #lisp
countvaj` has joined #lisp
toorevitimirp has quit [Ping timeout: 240 seconds]
toorevitimirp has joined #lisp
EvW has quit [Ping timeout: 260 seconds]
thetabit has joined #lisp
terpri__ has joined #lisp
terpri__ is now known as terpri
terpri_ has quit [Ping timeout: 260 seconds]
xkapastel has quit [Quit: Connection closed for inactivity]
countvaj` has quit [Remote host closed the connection]
countvaj` has joined #lisp
wxie has joined #lisp
gravicappa has joined #lisp
gravicappa has quit [Ping timeout: 240 seconds]
dominic34 has joined #lisp
wxie has quit [Remote host closed the connection]
v3ga has joined #lisp
Khisanth has quit [Ping timeout: 272 seconds]
cosimone has quit [Quit: Quit.]
shangul has joined #lisp
Khisanth has joined #lisp
narimiran has joined #lisp
nicktick has quit [Ping timeout: 256 seconds]
abhixec has quit [Ping timeout: 240 seconds]
Retropikzel has quit [Quit: Konversation terminated!]
red-dot has joined #lisp
countvaj` has quit [Ping timeout: 260 seconds]
wxie has joined #lisp
bhartrihari has joined #lisp
bhartrihari has left #lisp ["Disconnected: closed"]
Oladon has quit [Quit: Leaving.]
shangul has quit [Ping timeout: 265 seconds]
wxie1 has joined #lisp
wxie has quit [Ping timeout: 240 seconds]
wxie1 is now known as wxie
gxt has joined #lisp
dominic34 has quit [Ping timeout: 256 seconds]
tonythetiger has quit [Quit: Connection closed for inactivity]
shangul has joined #lisp
OpenZen has quit [Ping timeout: 256 seconds]
red-dot has quit [Quit: Going offline, see ya! (www.adiirc.com)]
shifty has quit [Ping timeout: 240 seconds]
Blukunfando has quit [Ping timeout: 272 seconds]
johntalent has quit [Ping timeout: 256 seconds]
<stylewarning>
beach (or anybody), can I pick your brain? I'm writing a functional programming language as a Lisp DSL that expands into Lisp code. For instance, the code (toplevel (define (f x) x)) expands into the CL code (DEFUN F (X) X). TOPLEVEL is the macro that processes the DEFINE forms; DEFINE itself is not a macro.
<stylewarning>
My trouble right now is that TOPLEVEL is also inferring the types of the functions defined by DEFINE and entering them into a global database, say a hash table mapping symbols (like F) to a representation of a type (say, a structure with a load form). It is doing this during macro expansion time.
<stylewarning>
It needs to do it at macro time so that all of the DEFINE's in the TOPLEVEL can be mutually type-inferred and type-checked.
<stylewarning>
My issue right now is that when the compiler produces a bunch of fasls, and my implementation (SBCL) later goes to load those fasls, the database is no longer known.
bocaneri has joined #lisp
<stylewarning>
Is the only way around this to macroexpand into toplevel SETFs so they get caught in the compiled output?
thetabit has quit [Ping timeout: 260 seconds]
red-dot has joined #lisp
gxt has quit [Quit: WeeChat 2.9]
narimiran_ has joined #lisp
<phoe>
stylewarning: I think so; there's also the issue of typechecking FASLs against the rest of the system
<phoe>
suppose you've made a FASL that depends on some earlier stuff and you're loading it, are you sure that it's still typechecked if the stuff it depended on was modified?
bsd4me has joined #lisp
narimiran has quit [Ping timeout: 246 seconds]
<stylewarning>
phoe: great point
Oladon has joined #lisp
<phoe>
I wonder if compilation-time is the best moment of performing that typechecking
<phoe>
because compilation-time doesn't exist for FASLs that are already compiled
<stylewarning>
load time seems like a good candidate, but it also seems too late. But maybe not all is bad because it seems that systems are rarely all compiled without loading
<phoe>
or if it's possible to design a system that performs the checks at compile time *OR* at load-time, but only if no compile-time check was made
<stylewarning>
more commonly seems that one does compile-load-compile-load-...
<phoe>
yes, it seems a bit late
<phoe>
compile-time type mismatches should be caught at compile time
<stylewarning>
i wish lisp had standard ways of hooking into the start & end of a compilation unit :)
<phoe>
it's called ASDF nowadays
<stylewarning>
i bet there's some crazy way to hook into the end w/o ASDF using the condition system
<stylewarning>
since warnings can be deferred to the end of a unit, or something like that. I bet that phoe guy knows about that
<phoe>
the thing is that one indeed needs to hook into the compilation unit - it's the implementation that actually gathers these warning into a bunch and then re-checks and prints them out together at the end of a compilation unit
<phoe>
I think you'd need a custom and/or hookable LOAD
<phoe>
and that's troublesome stuff
<phoe>
as for the condition system, huh; I can see no immediate way to create a set of conditions that will hook into the compilation unit (again, since it's not hookable by default) that will behave in the way we want
<phoe>
I think the easiest thing we can do is to create FASLs that also contain eval-when :load-toplevel logic for typechecking
<phoe>
this means that compile-and-loading code will execute validation twice in a row, but it should also be working
<phoe>
like, you will get compile-time errors if your code doesn't typecheck, and you'll get load-time safety in case a file is not loadable into the system
<phoe>
good design of macroexpansions will also mean that typechecks happen before any mutating of the Lisp image so you don't actually SETF FDEFINITION of anything that doesn't make sense to the typechecker
<phoe>
I actually think that you could want to add the actual SETF FDEFINITION calls to the finalizer list, so that load-time typechecks *all* have a chance to execute
<phoe>
this way you could typecheck whole modules before loading them
<phoe>
but that's a troublesome idea
wxie has quit [Ping timeout: 240 seconds]
<phoe>
and maybe unnecessary due to how ASDF recompiles stuff as required anyway
<lukego>
is there some obvious way e.g. slime option to have all def* forms highlighed? e.g. not just defvar but also e.g. defvar-unbound
<phoe>
if that is the same thing, that is; it might be too early for me to recognize
<fe[nl]ix>
stylewarning: well, it's a hack :)
karlosz has joined #lisp
notzmv has quit [Ping timeout: 246 seconds]
d4ryus has joined #lisp
ahungry has quit [Remote host closed the connection]
abhixec has joined #lisp
notzmv has joined #lisp
_jrjsmrtn has joined #lisp
__jrjsmrtn__ has quit [Ping timeout: 258 seconds]
gioyik has quit [Quit: WeeChat 2.8]
FennecCode has quit [Remote host closed the connection]
FennecCode has joined #lisp
Oladon has quit [Quit: Leaving.]
v88m has quit [Remote host closed the connection]
<jdz>
lukego: My slime does that bye default, probably a contrib.
<jdz>
s/bye/by
<lukego>
jdz: thanks for the tip, I'll grep the slime sources more thoroughly for where that happens
v88m has joined #lisp
<jdz>
lukego: Sorry, wrong information. All forms with define- are highlighted, not def-.
<Harag`>
morning
<Harag`>
beach: Could a naive implementation of sandboxed cl not just eclector and just override "interpret-symbol" to either block (raise an error) on cl functions you dont want to allow or just replace the cl function with a "safe" alternative.
<Harag`>
All you need then is a list of blocked and replaced functions. In theory that should work for macro's as well because at some time or another the "expanded code" will run into interpret-symbol as well.
<Harag`>
If you allow the "host" to amend the list of blocked or replaced functions, the host could customize behaviour to its specifications.
<lukego>
slime-fontifying-fu.el seems to be the relevant file, though as you say it doesn't catch this specific case
<beach>
Harag`: Sure, that might work. But you still have to write a complete evaluator.
FennecCode has quit [Remote host closed the connection]
FennecCode has joined #lisp
<Harag`>
could you not just do a run over the code to validate, and then pass validated code to eval? You could even compile and/or cache such validated code
<Harag`>
or am I missing the meaning of "evaluator" here
<beach>
It is not that easy.
<beach>
It is a code-walker problem.
jprajzne1 has joined #lisp
<beach>
You need to know what each macro does.
<beach>
That's why it is a complete evaluator.
<beach>
Suppose you want to make EVAL one of the unwanted functions. Now you see (let ((eval '(+ 3 4))) ...). Do you replace EVAL, or reject the code?
<Harag`>
so in essence you need to "safely" expand each macro to its fullest to be able to evaluate if its safe?
<beach>
Exactly.
<Harag`>
yeah that let one is a pain, I ran into it on my firts day of trying to write a parser for cl-isolated, ended up keeping context of "what" came before to decide
<beach>
The only solution to this problem is a "code walker" of which an evaluator is a special case.
Khisanth has quit [Ping timeout: 265 seconds]
fbmnds has joined #lisp
<Harag`>
ok thanx
john__ is now known as gaqwas
gaqwas has joined #lisp
gaqwas has quit [Changing host]
* Harag`
crawls of to go and bump his head some more ...
<fbmnds>
phoe: bike: your feedback helped me to track down the bug
<fbmnds>
is there something like lint for common lisp?
Lycurgus has quit [Remote host closed the connection]
ljavorsk has joined #lisp
orivej_ has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #lisp
wxie1 has joined #lisp
wxie has quit [Ping timeout: 256 seconds]
wxie1 is now known as wxie
iissaacc has joined #lisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #lisp
femi has joined #lisp
<fbmnds>
contrapunctus: thx - this looks interesting
gaqwas has quit [Remote host closed the connection]
kg_ has joined #lisp
orivej has quit [Ping timeout: 246 seconds]
orivej has joined #lisp
wxie has quit [Ping timeout: 240 seconds]
kg_ has quit [Client Quit]
mangul has joined #lisp
shangul has quit [Ping timeout: 246 seconds]
brown121407 has quit [Ping timeout: 260 seconds]
FennecCode has quit [Ping timeout: 240 seconds]
farooqkz__ has joined #lisp
aaaaaa has quit [Quit: leaving]
mangul has quit [Ping timeout: 256 seconds]
ayuce has joined #lisp
mangul has joined #lisp
farooqkz__ has quit [Ping timeout: 240 seconds]
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #lisp
random-nick has joined #lisp
orivej has quit [Quit: No Ping reply in 180 seconds.]
orivej has joined #lisp
wxie has joined #lisp
terpri has quit [Ping timeout: 260 seconds]
orivej has quit [Ping timeout: 240 seconds]
orivej has joined #lisp
ayuce has quit [Remote host closed the connection]
ayuce has joined #lisp
jesse1010 has joined #lisp
<fbmnds>
contrapunctus: sblint appears not generate warnings but one gets condensed compiler error messages at the exact line of code
terpri has joined #lisp
orivej has quit [Ping timeout: 240 seconds]
orivej has joined #lisp
wxie has quit [Ping timeout: 260 seconds]
orivej has quit [Ping timeout: 256 seconds]
orivej has joined #lisp
reggie__ has joined #lisp
reggie_ has quit [Ping timeout: 246 seconds]
bhartrihari has joined #lisp
micro has quit [Ping timeout: 265 seconds]
micro has joined #lisp
space_otter has quit [Remote host closed the connection]
toorevitimirp has quit [Remote host closed the connection]
cosimone has joined #lisp
pve has quit [Quit: leaving]
liberliver1 has joined #lisp
liberliver has quit [Ping timeout: 240 seconds]
liberliver1 is now known as liberliver
seok89 has joined #lisp
seok89 has quit [Remote host closed the connection]
seok82 has joined #lisp
<seok>
What is the right way to redefine a class?
nirved has joined #lisp
<Xach>
seok: re-evaluate the defclass form is one fine way
gravicappa has joined #lisp
<seok82>
would it cause problems with existing instances?
<Xach>
if you have live instances, and you care about how they adapt, write methods on update-instance-for-redefined-class or similar
Lord_of_Life_ has joined #lisp
<jdz>
seok: Everything will be fine if you're just adding slots, or changing slot options. Existing instances will preserve the slots they had before.
<seok82>
what about when deleting slots?
<jdz>
They will be deleted.
Lord_of_Life has quit [Ping timeout: 264 seconds]
Lord_of_Life_ is now known as Lord_of_Life
<jdz>
Poof, gone.
<Xach>
the grim GCer comes for all
<jdz>
The Grim Collector.
<Xach>
much better
kg_ has joined #lisp
<seok82>
Nice, that's straightforward
<seok82>
ty
<kg_>
Any Hunchentoot experts about the place? My server is returning the default 404 page, even though error-template-directory is pointing to the right place and I've created a 404.html file there.
<seok>
kabriel on windows? I'm getting error that looks like some linux-only dependency
wsinatra_ has joined #lisp
<Kabriel>
(test my-lst :limit nil)
<phoe>
therefore number-items is unreachable
rgherdt has left #lisp ["Leaving"]
<phoe>
therefore SBCL optimizes it away and issues a compiler note
<Kabriel>
seok: I didn't catch the windows part.
* Kabriel
using Linux
<seok>
I want to move to linux but I don't have money to build a new computer atm
<seok>
: (
<ghard>
Use a VM
theseb has joined #lisp
<seok>
Maybe
mankaev has quit [Ping timeout: 260 seconds]
<Kabriel>
phoe: ^^ it is an input, a user can always specify the keyword.
<phoe>
Kabriel: no, LIMIT is always non-null.
<phoe>
or rather, by the time it reaches the OR form, it is always non-null.
<phoe>
if you pass NIL as LIMIT, then > is going to signal an error.
<beach>
Good catch!
Necktwi has quit [Ping timeout: 256 seconds]
<phoe>
I mean, uhh, number-items, not limit
<phoe>
number-items is always an integer
<phoe>
since it's the result of LENGTH
<Kabriel>
humbling. Compiler (and phoe) +1, me 0
<phoe>
uh no, wait, I mean limit
<phoe>
blah
<phoe>
I was right, LIMIT is always non-null by the time OR is reached
<phoe>
so NUMBER-ITEMS is never reached
<phoe>
hence compiler note
<Kabriel>
makes sense; that is why when I remove the when clause, it leaves the or statement intact
<phoe>
yes, then such type inference is no longer made
karlosz has joined #lisp
<theseb>
minor issue but I'm trying to put in words why some people think python is easier.....lisp has a simpler implementation & structure...so if lisp is SIMPLER...how can people say python is EASIER....python has more (complex) syntax so i don't get it....only thing i can think of is the parens bother people...is that it?
<beach>
theseb: People say things that aren't true all the time.
<beach>
Noting to be surprised about.
<beach>
theseb: When people compare programming languages, they are not necessarily rational. Their minds are trying to justify their choice by inventing seemingly rational reasons.
Necktwi has joined #lisp
<phoe>
theseb: same reasons as when a lisp neophyte might try to convince the world that Java sucks
<beach>
Is someone trying to do that? It is a very bad idea.
<phoe>
I get a tiny vibe from things theseb posts that gives me such an impression; we've already had a similar conversation twice or thrice on the channel
<phoe>
"how can people say python is easier" - come on, that's literally what we've been talking about the previous time this subject was touched
<theseb>
beach: well i like both python AND lisp...i learned python years before lisp so it seems easier to me...I don't know if that's just because i've spent more time on it
<beach>
Yeah, rings a bell. But I never remember which participant it was. I think perhaps theseb is not alone?
<phoe>
because they find it simpler, that's it, that's the whole reason
lambda-smith has quit [Remote host closed the connection]
<theseb>
beach: i think *visually* there is less stuff to look at because python trades parens for spaces...so that MIGHT be something
<beach>
theseb: I guess I am not getting my message across here. People are not comparing things rationally. They speak out of emotional reactions.
lambda-smith has joined #lisp
jprajzne1 has quit [Quit: Leaving.]
<beach>
theseb: Here is the essence of what is going on:....
<phoe>
and "because they find it simpler" is a perfectly fine and perfectly normal and understandable thing given the way Lisp and Python have been evolving over the decades
<phoe>
which itself is a very complex issue that I don't claim to fully grasp myself
<beach>
You take a person who has a several-year investment in language A. Then that person is told that language B is simpler, faster, more powerful.
<beach>
theseb: Our minds can't cope with that situation. It is called cognitive dissonance.
<beach>
theseb: So their minds invent reasons for justifying their choice.
<beach>
theseb: One such possibility is to deny that it is simpler, faster, more powerful.
<beach>
That's an easy one because when can't measure simplicity or power.
<theseb>
beach: one measure of simplicity is size of grammar...lisp wins there
<beach>
theseb: Another popular thing their minds can do is to denigrate the messenger. That automatically puts the credibility of B in question.
<beach>
theseb: I give up. You are not listening. Good luck!
<phoe>
theseb: no it doesn't really win there
lambda-smith has quit [Client Quit]
<theseb>
beach: another measure is amount of characters in equivalent programs...python wins there (again...spaces instead of parens)
<beach>
theseb: I give up. You are not listening. Good luck!
<theseb>
beach: you think it is irrational
<theseb>
i get it
<beach>
I know. I have studied the subject.
<theseb>
or mental blocking
<theseb>
beach: really?
<beach>
The psychological phenomena, I mean.
<theseb>
oh
<phoe>
CL requires the programmers to memorize a ton of macros/special operators and their usage and arguments which are non-trivial *and* are a part of Common Lisp syntax because you cannot effectively write standard CL without them
<phoe>
while one can successfully argue that these are all symbols and therefore aren't a part of Common Lisp syntax the way the "while" keyword is a special symbol in Java syntax
<phoe>
but at that point the discussion becomes meaningless because Java syntax, defined in such a way, allows you to write basic programs, and Lisp syntax defined in such a way doesn't even allow you to call a single function, so obviously we are comparing apples and oranges here
<theseb>
phoe: i think you're saying the ramp up time for other langs is less
<theseb>
startup costs
<beach>
*sigh*
<phoe>
theseb: no, I'm saying that making the assumption "Lisp grammar === basic s-expressions" is false
<phoe>
I haven't even mentioned ramp up times anywhere, where did you pull that meaning from
<theseb>
phoe: it sounded like you were saying you had to do more initial work (" memorize a ton of macros/special operators and their usage and arguments which are non-trivial *and* are a part of Common Lisp syntax because you cannot effectively write...")
wsinatra has quit [Quit: WeeChat 2.8]
<theseb>
beach: on a different topic..i'm currently trying to write a simple lisp eval function in assembly
<beach>
theseb: What is the point of all this?
<phoe>
theseb: no, you learn them gradually
<phoe>
just like you learn the various parts of python/java/c/cpp/rust/whatever other language gradually
<theseb>
beach: ok..i'll tell you exactly what the point is...to be honest...
<beach>
theseb: Why on earth would you want to write something like that in assembly?
<theseb>
beach: you may recall i'm writing a curriculum..."Program To Processor"...i'm showing all steps from python to intermediate language (minimal lisp)..to assembly to machine code.....I was imagining the near future where I have to explain why some prefer python instead of lisp
mankaev has joined #lisp
<theseb>
beach: i was hoping to say something more scientific than just ..."different people just prefer different languages"..but i guess as you said....the issue is a complex one
<theseb>
beach: so the assembly is the last step of my compiler
<theseb>
that's why
<theseb>
comments welcomed....good or bad
<beach>
But you wouldn't write the assembly yourself.
narimiran_ has quit [Ping timeout: 256 seconds]
<theseb>
beach: i have a simple cpu i specified...so i need to write assembly for it
<theseb>
beach: there doesn't exist a compiler for the cpu i made up! ;)
RedMallet has joined #lisp
<beach>
No, you put the CPU specification in the code generator of a compiler, and then you write your code in Lisp.
<phoe>
theseb: time to make one
<phoe>
so, when you write an emulator for your CPU to verify it, you can ensure that it all works
<beach>
theseb: You want to write an evaluator? Well, one type of evaluator is a compiler. So you can write it in assembly or you can write it in Lisp.
<beach>
Why would you write it in assembly?
<phoe>
if someone works on a new CPU architecture then one of the first things I'd expect them to write is an emulator for it which can execute programs written for that CPU architecture
<beach>
theseb: It sounds like you need to follow my series of presentations for the online Lisp meetings.
<theseb>
I first convert a minimal python to minimal lisp code.....now i need to *run* that on my cpu....it doesn't know lisp so i must convert the lisp to assembly yes?
<phoe>
yes
<phoe>
how do you run that assembly then
<beach>
Yes, converting Lisp to assembly is the job of a compiler.
<beach>
So you write the compiler in Lisp.
<theseb>
beach: yes! that's what i'm writing now
<phoe>
so you're writing a compiler from X to assembly
<beach>
So why did you say you want to write an evaluator in assembly?
<theseb>
phoe: yes
<phoe>
so what beach said, why would you want to write anything high-level in raw asm
<theseb>
beach: give a sec to answer that...
kaftejiman has joined #lisp
<theseb>
beach: ok..what does it mean to "convert lisp to assembly"?...in my mind it means...you encode your program sexps into RAM....then, for assembly to "run" it...you need something in assembly that functions as the evaluator yes?
<phoe>
theseb: do you know what a compiler is?
<beach>
Yes, and you create that by cross compiling on a different Lisp system.
<theseb>
phoe: compiler essentially converts program in lang X to lang Y
<phoe>
theseb: correct
<beach>
theseb: Make the compiler compile from Lisp to assembly.
<beach>
Done!
cosimone has quit [Quit: Quit.]
<theseb>
beach: what do you mean by "cross compiling on a different Lisp system"
<phoe>
the compiler runs on x86 but generates ARM code
<phoe>
that's cross-compilation
<beach>
Write the compiler in Lisp. It generates assembly for your machine. Run the compiler on SBCL. It now generates assembly for your machine.
<phoe>
obviously ARM code won't run on the same machine that the compiler is running on - but it will run on a different machine, e.g. that does not yet have a compiler of its own
<beach>
Then, compile the compiler on itself.
<beach>
theseb: This is really basic stuff.
<phoe>
that's how GCC/GHC/SBCL/CPython/PyPy/literally any other compiler is ported to new architectures
<beach>
theseb: You need to do A LOT of rading.
<beach>
reading.
<beach>
Anyway, I need to go fix dinner for my (admittedly small) family. Good luck!
<theseb>
beach: thanks
<theseb>
phoe: yes....and if you make a new cpu.. you or someone needs to do the hard work of writing the cross compiler to target the new cpu right
<phoe>
theseb: no
<phoe>
the cross compiler already exists, it's the compiler
<phoe>
one only needs to write a new code generator for it that translates the compiler's intermediate representation into target assembly
<phoe>
that's why most compilers are modular.
<theseb>
phoe: yea...that's what i meant...we'd need to write the part to convert the intermediate code to our new cpu
<theseb>
that's what i'm on now..so i guess i'm current writing a "cross compiler"
<theseb>
phoe: do you agree when you "compile" you lisp down to asm that somewhere in that process it has to emit an evaluator in asm?
<phoe>
theseb: sure, EVAL needs to be translated into assembly
<scymtym>
beach: thanks
<phoe>
but EVAL is never *written* in assembly
<Bike>
if your lisp image does not have the eval function in it, like you can get with some comercial lisp implementations, there does not need to be any kind of evaluator in the object code.
<theseb>
phoe: oh i see what you mean....you want to first implement the LANGUAGE in assembly
<phoe>
theseb: no, not the language
<theseb>
in this case lisp
<phoe>
no!
<theseb>
or whatever the intermediate language is
<Bike>
writing anything in assembly is terrible and almost always unnecessary
<phoe>
you already have a piece of code that turns Lisp/Python/whatever into IR
<phoe>
where IR is some sorta intermediate representation
<theseb>
yes
<phoe>
you only need to write a piece of code that turns this IR into this new sort of native assembly.
<Bike>
and that code is probably itself written in lisp.
<phoe>
at this point, this IR has almost nothing in common with Lisp/Python/whatever
orivej has quit [Quit: No Ping reply in 180 seconds.]
<theseb>
phoe: wow...you opened my eyes...yea...i should do what you said
<theseb>
phoe: thanks for getting thru my thick skull
<phoe>
I say "almost" because the high-level structure is still there in some cases, but it's been dissolved into IR
<phoe>
theseb: no problem, please do the same to someone new to Lisp and/or compiler theory in a few years when you gain some more experience :D
<seok>
sorry to cut in guys, emacs/slime always hang when trying to print a long line of string, this kind of behaviour doesn't exist in cmd or IDE's for other languages. is anyone bothered by this?
<phoe>
also thank beach, he's been contributing to it
orivej has joined #lisp
<theseb>
phoe: yea...thanks again
<Bike>
seok: i haven't seen this. how long are we talking?
<seok>
atm I tried to print a text file 1800kb in size
<Bike>
you do that in IDEs a lot?
<phoe>
I think that's an emacs limitation, I don't know how well it handles buffers of such sizes
<seok>
sometimes when I want to know the string was received in memory
gaqwas has joined #lisp
gaqwas has quit [Changing host]
gaqwas has joined #lisp
<seok>
I just print it
<phoe>
received in memory? what do you mean
<seok>
just to see if I have the correct string in a symbol
<phoe>
in a symbol? what do you mean
<Bike>
if i wanted to do that i would probably take the length instead.
<phoe>
and then, when you've printed, you manually read all of these 1800000 characters to verify that it matches some other string?
<Bike>
that said, if you really do want to do this for some reason, it might be speedier if you avoided some complicated parts of the printer
<Bike>
like (progn (write-string huge-file) nil) kind of thing
<seok>
(setf x (uiop:read-file-string ...)) => (print x)
<Bike>
if the slowdown is on lisp's end. about emacs, i have no idea.
<phoe>
seok: and what do you do after you've printed it to actually verify that it contains the proper data
<seok>
It's a habit, but I've never experienced such slowdowns when writing in python or node
<phoe>
I assume it's slime presentations
<phoe>
try what Bike said - (progn (print x) nil)
<seok>
thats faster?
<phoe>
this way slime/emacs won't waste time computing the presentation for that object
<phoe>
yes
<phoe>
because it won't try to print the return value *again*
<seok>
oh
<seok>
yes it is
<Bike>
does that fix your problem?
<seok>
sort of, yes
<theseb>
phoe: "you only need to write a piece of code that turns this IR into this new sort of native assembly"...can i ask one more thing about that?
<seok>
I see, it hangs while trying to load the value of the return
<phoe>
theseb: sure
<phoe>
seok: yes, that part is called a presentation
<seok>
right
<seok>
I learn something new everyday here : D
<seok>
ty
kaftejiman_ has joined #lisp
<theseb>
phoe: suppose your IR is lisp-ish.....so it may define variables, add to an "environment", evaluate sexps...etc.......Now...when you convert that to assembly....you'll need to implement asm code to mimic an environment, define variables, evaluate sexps, etc right?
<phoe>
theseb: that's not IR
<Bike>
compiled lisp code does not need environments or variables or an evaluator.
<phoe>
or even if it is, it isn't the sort of IR that assembly is generated from.
<Bike>
it doesn't. i promise.
<phoe>
by the time you get to the lowest level of an IR, you no longer have environments or evaluator or whatever Bike said
<Bike>
just try it. get PAIP or something. compile to some virtual machine.
<phoe>
try reading some Python bytecode if you have a Python background
<Bike>
it's easy. you can write a compiler for a simple scheme dialect to a simple VM in like an hour.
kaftejiman has quit [Ping timeout: 246 seconds]
<theseb>
phoe: so my "IR" is too high level?
<phoe>
yes
orivej has quit [Ping timeout: 240 seconds]
<phoe>
it's literally Lisp
<Bike>
the IR is chosen for the compiler dev's convenience. it is inconvenient if it can't be translated into machine code fairly simply.
orivej has joined #lisp
<theseb>
Bike: i believe you ...i just can't see right now how you could make lisp code so low level it has no concept of an environment and eval
<theseb>
i'll need to read more about IRs
<phoe>
it's simple, it no longer looks like lisp code at that point
<Bike>
okay. imagine the function: (defun foo (x) x)
<Bike>
in machine code, this would move the argument, which is in some register, to the first return value register, and then return.
<theseb>
Bike: yes but your example also adds 'foo' to the environment
<theseb>
Bike: what if foo is called somewhere else in code
<Bike>
"environment" means a couple different things.
<theseb>
now your doing an asm lookup in asm
<theseb>
sorry.. i mean..now your doing an env search in asm
<phoe>
you no longer have variables, you have registers; you no longer have environments, you have memory locations; you no longer have functions, you have pointers in the global function hash table
<Bike>
in a running lisp image you probably do have some kind of global environment.
<Bike>
you don't really need LEXICAL environments, which is what i assumed you were talking about.
<theseb>
phoe: yes
<Bike>
depending on how your lisp implementation works, it might be possible to compile programs that don't have a persistent lisp image, and so have no global environment.
<phoe>
so, does that answer your question?
<theseb>
phoe: you're saying you have primtive equivalents in asm to all that lisp stuff
<Bike>
in that case defining functions with different names would work like it does in C or something.
<phoe>
theseb: "equivalents"?
<phoe>
hell no
<theseb>
phoe: registers instead of variables,....etc
<phoe>
they are THE LISP STUFF that has been compiled to assembly
<Bike>
i mean, the point of a compiler is to make machine code that has equivalent effects to some lisp code.
<phoe>
except for simple things like variables, yeah, the compiler usually gets rid of those as optimizations
<Bike>
that's what a lisp compiler is. that's the definition. you're making an "equivalent".
<Bike>
but there's no necessary correspondence whatsoever between particular structures in the lisp code and structures in the machine code.
<Bike>
this isn't unique to lisp. this is how compilers work.
<jackdaniel>
I want a mnemonic in my assembler for defclass
<jackdaniel>
dfc [magic] [more magic]
gaqwas has quit [Remote host closed the connection]
<phoe>
new lisp machine cpus when
<Bike>
the lack of correspondence may be more obvious in weird compilers. like you can compile lisp to FRACTRAN, in which programs are sequences of rational numbers. in the end it can all be arithmetic.
<phoe>
that's a sequence of parens that can be encoded into a bit vector
Lycurgus has joined #lisp
<phoe>
and computers like bits
<Bike>
or if you have (defun foo (x) (let ((y x)) y)) that'll probably compile identically to my previous example.
RedMallet has quit [Quit: WeeChat 2.6]
Blukunfando has joined #lisp
<theseb>
phoe: SICP in chapter 5 has a nice exposition where they specify a lisp evaluator in a assembly-like language...then that is meant to be converted to their random access machine
theBlackDragon has quit [Quit: Boom.]
<theseb>
they go thru each special form like lambdas and write them out in their low level spec
<phoe>
don't think there exists a machine that accepts such low level spec as direct input without compiling it further along the way
<theseb>
phoe: right.....i guess their low level spec is the lower level IR between lisp and asm you were talking about
<Bike>
which section are you talking about precisely? i'm looking at chapter five now.
<Bike>
do you mean like make-compiled-procedure?
Lycurgus has quit [Remote host closed the connection]
<Bike>
or like, in "compiling conditional expressions", there's a "false?" op but that's it. there's no "op" for if itself per se
theBlackDragon has joined #lisp
orivej has quit [Ping timeout: 258 seconds]
<theseb>
Bike: let me see...
orivej has joined #lisp
<theseb>
Bike: p513 has their slighly higher level pseudo-assembly that i guess you could call an IR
<Bike>
i'm reading online and don't have page numbers.