rumbler31 has quit [Remote host closed the connection]
<eschulte>
Does anyone know of a serialization library appropriate for deserializing from another programming language?
<eschulte>
in particular I'm thinking of something I could combine with snooze to provide access to CL objects from another language through a REST API (and a wrapper library in the other language)
<no-defun-allowed>
json's usually pretty common and cl-json handles it really well
anewuser has joined #lisp
<whartung>
Yea json is pretty lingua franca today
<eschulte>
oh, thanks!, that seems obvious. do you know off hand how it supports CLOS objects?
<eschulte>
never mind, reading the docs, this looks like the obvious choice, thanks!
Oladon has quit [Quit: Leaving.]
Josh_2 has joined #lisp
zmt00 has joined #lisp
beach has quit [Ping timeout: 264 seconds]
<Josh_2>
WHat's the best oath2 library?
<Josh_2>
oauth2*
nanoz has joined #lisp
terpri has quit [Read error: Connection reset by peer]
charh_ has joined #lisp
Fare has joined #lisp
charh has quit [Ping timeout: 245 seconds]
igemnace has quit [Quit: WeeChat 2.2]
<no-defun-allowed>
has anyone ever made a program which mutates some lisp code randomly until it runs without errors?
Pixel_Outlaw has joined #lisp
Josh_2 has quit [Remote host closed the connection]
<eschulte>
no-defun-allowed: close, see https://grammatech.github.io/sel/#Top but honestly it's support for C/C++ and assembler are much more mature than it's lisp support
zmt00 has quit [Read error: Connection reset by peer]
charh has joined #lisp
zmt00 has joined #lisp
<nydel>
hmm, i return to an old project depending on :USOCKET package and it seems the package function #'SOCKET-SERVER is not defined. is this my end? or has usocket changed?
charh_ has quit [Ping timeout: 245 seconds]
<nydel>
i wonder if usocket is even still the goto package. any help would be appreciated.
<Xach>
nydel: you have to depend on usocket-server i think.
khisanth_ has quit [Ping timeout: 244 seconds]
<Xach>
nydel: usocket is still Good
Fare has quit [Ping timeout: 240 seconds]
dented42 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
charh_ has joined #lisp
<nydel>
Xach: thank you - i was able to ql:quickload that right away. is this package connected to :USOCKET?
<Xach>
nydel: it is part of usocket
charh has quit [Ping timeout: 245 seconds]
charh has joined #lisp
charh_ has quit [Ping timeout: 252 seconds]
<nydel>
odd, as far as i can tell, what used to be :USOCKET is partially in :USOCKET-SERVER even though slime-mode autocompletes some functions from the latter with only the former loaded.
<nydel>
available documentation doesn't really explain what happened as far as i can see
pjb has quit [Read error: Connection reset by peer]
graphene has quit [Remote host closed the connection]
graphene has joined #lisp
nowhereman_ has joined #lisp
rumbler31 has joined #lisp
varjag has joined #lisp
rumbler31 has quit [Ping timeout: 240 seconds]
varjag has quit [Ping timeout: 260 seconds]
Kundry_Wag has quit [Remote host closed the connection]
Roy_Fokker has quit [Read error: Connection reset by peer]
pierpal has joined #lisp
razzy has quit [Ping timeout: 244 seconds]
sabrac has quit [Quit: Konversation terminated!]
sabrac has joined #lisp
Pixel_Outlaw has quit [Quit: Leaving]
dddddd has quit [Remote host closed the connection]
pierpal has quit [Read error: Connection reset by peer]
edgar-rft has joined #lisp
Fare has joined #lisp
anewuser has quit [Quit: anewuser]
Bike has quit [Quit: Lost terminal]
igemnace has joined #lisp
<aeth>
You know what's cool about CL? You write something and 19 times out of 20 it'll work in SBCL, CCL, and ECL without having to work around various quirks.
brettgilio has joined #lisp
<Blackbeard>
aeth: I like that you have so much power
<forumulator>
Hi guys. Need a little help. I'm trying to expand a lisp macro to product something like val(a)->val(b). However, `(,a -> ,b) doesn't give me the correct result.
<forumulator>
Because of the spaces. How can I put in a member access without spaces?
<no-defun-allowed>
You'd need to FORMAT and create a string probably.
<forumulator>
Can you give me an example, please?
<forumulator>
Little bit new to lisp
nanoz has quit [Ping timeout: 245 seconds]
<beach>
forumulator: Spaces?
<beach>
forumulator: Common Lisp macros don't work on surface syntax like that.
<beach>
forumulator: They work on S-expressions. Spaces are just related to how things are printed.
<forumulator>
Yeah I mean something like (defconstant a->b 5) is valid.
<beach>
It is.
<forumulator>
But (defconstant a -> b 5) is not
<beach>
Right.
<beach>
Because of the way the reader works.
<forumulator>
So how do I take care of that?
<forumulator>
I'm expanding the macro to a defconstant
<beach>
What macro?
<forumulator>
I have a macro, which has variables a and b
<beach>
Oh, that is very messy stuff.
<forumulator>
I finally need to expand to something like (defconstant val(a)->val(b))
housel has quit [Read error: Connection reset by peer]
<forumulator>
I could do a format, but it only outputs strings. How do I convert string to lisp code?
housel has joined #lisp
<beach>
If you are new to Common Lisp, that is definitely NOT something you want to do. You will get into all kinds of trouble with readtable-case and packages.
frgo has quit [Remote host closed the connection]
<forumulator>
It is something I have to do. Simple assignmetn.
<forumulator>
beach: I dont' care about messy at this point, I've spend hours breaking my head on this. Lisp is a major programming language, and there doesn't seem to be sufficient macro documentation.
<beach>
WHAT?
<forumulator>
Which part :)?
<beach>
"there doesn't seem to be sufficient macro documentation"
<forumulator>
There is some stuff, but I couldn't find advanced docs for the stuff I'm trying to do.
Inline has quit [Quit: Leaving]
<beach>
The fact that you can't find it is not the same thing as it isn't there.
<forumulator>
Right, sorry :)
<beach>
I take it this a university class you are taking? What university?
<forumulator>
I'd rather not say, if that's okay?
<beach>
Too bad. I would be interested in knowing where Common Lisp is still taught.
<forumulator>
beach: But yeah, university class. Everything's good except for this final part. Any idead?
<beach>
I told you.
<beach>
Use INTERN.
<beach>
And specbot gave you a link to it.
flip214 has joined #lisp
<forumulator>
So something like intern "a->b" ?
<beach>
I'd rather not say, if that's okay?
<beach>
Read the page of the link and try it.
<forumulator>
lol :). Thanks for the help, anyway.
rnmhdn has quit [Ping timeout: 252 seconds]
<forumulator>
Please don't mind, taking help on an assignment is a bit sketchy. Even if they are ones like this.
<beach>
It's a bad assignment for people who are new to Common Lisp.
<beach>
You will get into all kinds of trouble.
<forumulator>
Right. Whatever happened to the old, 'run a loop and print some stars' :)?
<forumulator>
Out of curiosity, does lisp provide enums in the language?
<beach>
What would be the use?
<beach>
From the assignment, I don't see the point.
<forumulator>
Ah. Cool, once again, thanks for your help :)
<beach>
Sure.
<beach>
In a language like C, "enums" are used to associate compile-time names with small integers at run-time.
<beach>
But Common Lisp has symbols at run-time, so there is no great use for such a facility in Common Lisp.
<beach>
The first example of the assignment seems to suggest that kind of use.
<beach>
But then, the example with the planets is totally different.
<beach>
Basically, constants with names like planet->mass->mercury are pretty useless in terms of programming.
<beach>
It would be preferable to use a hash table or something similar.
<forumulator>
I see.
<beach>
In fact, if you have to construct that kind of symbol at run-time, you would use a hash table (that of the package) implicitly anyway.
frodef has joined #lisp
<forumulator>
Instead of format and then intern, you mean? :D
_bryan has joined #lisp
jackdaniel has joined #lisp
<beach>
Yes, instead of FORMAT+INTERN it would be much better to use nested hash tables.
<forumulator>
Got it. Have to go to school now, thanks for your help! I'll ask my prof the point.
<flip214>
well, I don't see a package PLANET with a symbol MASS and a property MERCURY as pleasing.
<beach>
Right.
<beach>
The order seems weird as well.
<flip214>
although, for a single-level hash table, emulating multiple levels via a _single_-character (so it's "trivial" to split [ie. in C to change to a NUL ;]) isn't unheard of
forumulator has quit [Ping timeout: 256 seconds]
sauvin has joined #lisp
brettgilio has quit [Remote host closed the connection]
<no-defun-allowed>
morning beach
brettgilio has joined #lisp
<no-defun-allowed>
forumulator: my first piece of advice is to defenestrate your teacher
brettgilio has quit [Remote host closed the connection]
brettgilio has joined #lisp
<shka_>
good morning
<beach>
Hello shka_.
<no-defun-allowed>
hi shka
frgo has joined #lisp
<makomo>
@smorning
<makomo>
morning*
<makomo>
woops
<beach>
Hello makomo.
xrash has quit [Ping timeout: 244 seconds]
brettgilio has left #lisp ["ERC (IRC client for Emacs 27.0.50)"]
<no-defun-allowed>
hi makomo
<makomo>
\o
frgo has quit [Ping timeout: 245 seconds]
xrash has joined #lisp
scymtym has quit [Ping timeout: 252 seconds]
<no-defun-allowed>
o/
<phoe>
Are weak hash tables the only kind of weak storage available to me as the programmer?
trittweiler has joined #lisp
<jackdaniel>
phoe: common lisp doesn't give you weak data structures at all. as of compilers, usually you have hash tables with weak keys, values or both and for general structures you have a weak pointers
<phoe>
jackdaniel: I know, I'm not asking about standard functionality.
<phoe>
Oh - weak pointers is what I'm looking for.
frgo has joined #lisp
<jackdaniel>
that's why I've mentioned what some implementations provide. first part of the answer hints, that it was imprecise. if abcl provides weak heap, would that satisfy you if I'd said: weak heap is there too. ?
<jackdaniel>
trivial-garbage gives you a portability layer for these structures
<phoe>
yes, I see
<jackdaniel>
s/that it was/that the question was/
dented42 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<shka_>
phoe: please note that in practice, weak pointers in garbage collected language have a limited use
<jackdaniel>
I'm curious what use do they have in a language, which is not(!) garbage collected
<shka_>
jackdaniel: breaking circular dependencies in reference counting
<shka_>
obviously
<shka_>
which happens a lot!
<jackdaniel>
isn't reference counting a poor man's GC though?
<flip214>
it's a different model.
<beach>
shka_: Why do weak pointers have limited use when there is garbage collection?
<flip214>
more overhead all the time, but no GC pauses.
<beach>
flip214: Oh? No pauses? HAH!
<jackdaniel>
sure, but it's still in GC realm
frgo has quit [Ping timeout: 252 seconds]
<shka_>
jackdaniel: i would not say so, most people think about tracking GC when they are talking about GC
<beach>
flip214: Try having a huge tree and removing the last reference to the root.
<shka_>
beach: because the only thing that comes to my mind are caches
<jackdaniel>
shka_: I'm very sceptical about this claim (especially in context of discussing data structures with weak references)
<Shinmera>
Refcounting helps in tracking garbage and disposing of it. It is a gc technique, whether people like it or not.
<no-defun-allowed>
rc is cee user's attempt at GC
<no-defun-allowed>
*users'
<no-defun-allowed>
refcounting is gc like javascript is java
<shka_>
ok, let me rephrase it
<phoe>
welp, I've started an intense garbage collection discussion
* phoe
hides
dented42 has joined #lisp
<shka_>
in tracking GC language, reference counting have limited use when compared to reference counting
<flip214>
beach: special cases exempt, as always.
<no-defun-allowed>
if you have a doubly linked list RC is no good
<shka_>
flip214: not so special, really
<Shinmera>
shka_: a tracking gc still counts references and hence weak pointers have plenty of use.
frgo has joined #lisp
<shka_>
Shinmera: would you provide a handfull of those?
<shka_>
something that is not cache
<Shinmera>
anything where you aren't the one that is in charge of managing the object's lifetime
<flip214>
shka_: a big graph of structures with only a single entry point is less common than one with multiple entry points... n over k ;)
<Shinmera>
an easy example would be keeping track of instances of a class
<shka_>
Shinmera: yes, in cache
<Shinmera>
? that's not a cache
<no-defun-allowed>
"ooh! it's for realtime programs cause tracing isn't realtime!!! cause rc totally is deterministic!!1!!"
frgo has quit [Read error: Connection reset by peer]
frgo has joined #lisp
<shka_>
Shinmera: right, not a cache
<flip214>
shka_: keeping identity across "foreign" objects (like ones loaded from a database).
<shka_>
flip214: i don't get this one
<jackdaniel>
defending opinion just for a sake of defending it isn't a good way to manage ones time :) \o
<Shinmera>
jackdaniel: are you going to defend that opinion? :^)
<jackdaniel>
^_^ no, I've run out of coffee, need to go buy some
<no-defun-allowed>
i hope my abuse of exclamation marks indicated my sarcasm (^:
zooey has quit [Ping timeout: 256 seconds]
<shka_>
anyway, Shinmera is right with his example
<shka_>
i never did this myself, but this is legitimate example i guess
<shka_>
and may araise in practice
asarch has quit [Quit: Leaving]
<flip214>
shka_: imagine having a database table with records and a primary key
<flip214>
now, whenever a thread loads object #1 from there, you want to return the same (as in EQ) object in memory
<flip214>
so you'll need a hash table (database, table, record number) => WEAK object
<shka_>
flip214: so you are making a cache? :D
<flip214>
that's not a cache.
<shka_>
ok, continue
<flip214>
that's an identity-preserving database layer ;)
<shka_>
but it totally sounds like sort of cache to me
<phoe>
cache implies that data can be flushed at any time
<flip214>
a "cache" is to speed something up, IMO
<flip214>
this use is to get EQualness
<phoe>
if you flush what flip214 is speaking of, you no longer get EQ
<shka_>
well, it speeds things up proabbly anyway
<phoe>
sure, but speed isn't its main purpose
<phoe>
it's preserving identity
<shka_>
OR IS IT? :d
* phoe
facepalms
<shka_>
phoe: well, for me things are what they are because of how they behave, and not why they were created
<shka_>
so intermiediete layer that weakly stores objects and returns those when they are available is cache by definition
<phoe>
intermiediete layer that weakly stores objects and returns those when they are and is emptiable at any time*
zooey has joined #lisp
<phoe>
FTFY
<shka_>
well, still a cache
<flip214>
shka_: but with a weak pointer it doesn't _cache_, as in "keeps it alive when nobody else is using it to speed up next accesses"
<Shinmera>
arguing semantics is a worthless waste of everyone's time, please stop.
<flip214>
with a weak pointer it only "remembers that _this_ record is around to return identity"
<shka_>
anyway, Shinmera example is good enough for me, so i no longer support my claim
<flip214>
Shinmera: no it isn't! (weak pointer to mpfc)
vlatkoB has joined #lisp
<shka_>
still, in some tracked GC languages you don't have weak references
<shka_>
i wanted to say that C# does not have those but apparently it already have ^_^
<jackdaniel>
nobody claimed that weak references are a must in GCed languages. small remainder: original claim: "weak structures have limited use in GCed languages" followed by a question: "do they have *any* use in languages which are not GCed?". then discussion went south witch caches and refcounting classification
<shka_>
jackdaniel: you drink to much coffe
<jackdaniel>
no, you contradict yourself and this kind of ticks me off
<jackdaniel>
and it is a pattern
heisig has joined #lisp
<shka_>
well, if you say so
<jackdaniel>
regarding drinking too much coffee I'm not sure why you even said that
<|3b|>
weak pointers in non-gc language sounds useful to me. doesn't c++ have weak pointers now?
<shka_>
since C++11
<shka_>
and in boost for a long time
<shka_>
don't think to much about it ;-)
scymtym has joined #lisp
makomo has quit [Quit: WeeChat 2.2]
varjag has joined #lisp
<phoe>
Is CHANGE-CLASS thread-safe in general? I assume not.
<jackdaniel>
|3b|: but shared_ptr gives you GC by reference counting in C++, and weak_ptr is part of this machinery, no?
anunnaki has quit [Ping timeout: 246 seconds]
<beach>
phoe: It would depend on the implementation I would think.
<|3b|>
jackdaniel: yeah
<|3b|>
jackdaniel: though seems sort of useful to have it for same way we would use it in GC language, for stuff that wants to keep an eye on something they don't manage, without interfering with it
anunnaki has joined #lisp
* |3b|
wonders how c++ programmers would think of that sort of use :)
<jackdaniel>
fair enough
angavrilov has joined #lisp
<aeth>
shka_: usually when a language adds a feature it's because some niches really, really need it, even if it's of "limited use" elsewhere
<shka_>
phoe: i was searching manual of SBCL in order to find info on this, but i failed
<shka_>
few weeks ago
<flip214>
phoe: AFAIK it doesn't even keep history.
<shka_>
|3b|: honestly, having std::weak_ptr is a must
<oni-on-ion>
theres a saying "everything specified in the standard should be thread-safe" i just read
<shka_>
and it happens literally all the time in real life
<shka_>
it is a mess :(
<|3b|>
yeah, i'm just saying it could be interesting aside from that
<shka_>
ok, makes sense
pjb has quit [Ping timeout: 240 seconds]
<russellw>
it is said that ;; should be indented with surrounding code and ;;; should be indented at the left margin. But are there any actual usages where these give different results? In other words, is ;;; ever used other than between top-level forms which are at the left margin anyway?
<|3b|>
the language/implementation doesn't care, it is just a common style
<jackdaniel>
it sometimes is (I saw it used inside function body somewhere)
dented42 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<|3b|>
you can put them where ever you want and indent them oddly, it will just confuse people used to the 'common' style
<russellw>
|3b| right, I'm asking about usage in practice, because I am writing a code formatter
<flip214>
russellw: you are right.
<|3b|>
in practice for compiler or for human reader?
<russellw>
jackdaniel, so on that occasion, was ;;; indented to left margin?
<flip214>
in practice it comes up to the same thing. unless the human code writer accidentally put ";;;" within some s-expression.
<|3b|>
compiler sees first ; and ignores anything else including more ;
<jackdaniel>
russellw: yes
<russellw>
|3b| human reader. The compiler does not care whether the code is nicely formatted
<russellw>
jackdaniel, thanks!
<jackdaniel>
it was describing general flow of the algorithm I think (not code itself). I don't remember well
<|3b|>
ok, in that case, yeah, i'd expect ;;; at left even inside code
bigfondue has quit [Ping timeout: 240 seconds]
<russellw>
|3b|, cool, thanks
dented42 has joined #lisp
dented42 has quit [Client Quit]
<LdBeth>
;;; Should occur on toplevel only
pierpal has joined #lisp
* |3b|
also notes that "top-level" isn't always far left, for example eval-when and such, where you might wrap multiple definitions
kozy has quit [Ping timeout: 252 seconds]
kozy_ has joined #lisp
<russellw>
|3b| true. In that case, should ;;; between such top-level definitions, still be indented at the far left?
* |3b|
would still expect ;;; for the global definitions, even if inside eval-when or a closure, macrolet, etc
<|3b|>
i'd probably say yes
<russellw>
okay
<|3b|>
though probably wouldn't object either way
pierpal has quit [Ping timeout: 240 seconds]
<|3b|>
does your formatter add/remove newlines, or just adjust indentation?
<|3b|>
(assuming the input has newlines, and isn't just actual conses etc)
<russellw>
it adds/removes newlines
<|3b|>
cool
<|3b|>
code reformatter has been on my list of "things to maybe do one day" for a while, but never get to it
<russellw>
I can understand that :)
<beach>
russellw: What are you planning to use to read your code?
* |3b|
thinks a bit more and realizes caring about comments implies source probably has newlines too :)
vlatkoB has quit [Remote host closed the connection]
<russellw>
beach I was originally hoping to use the built-in reader, but eventually accepted this is not going to be possible and I have to write my own. So this is mostly done at this point https://github.com/russellw/lisp-format/blob/master/read.lisp
<beach>
Oh! I was going to recommend Eclector.
<beach>
It can handle stuff that is not read by an ordinary reader.
vlatkoB has joined #lisp
<beach>
Oh well, more duplicate work.
<russellw>
ah! checking it out, looks interesting. Oh well, mine is pretty much done now, so. More duplicate work indeed :)
shrdlu68 has joined #lisp
bigfondue has joined #lisp
<flip214>
|3b|: "probably"... #| this is a comment, too #|
<LdBeth>
Actually I'm working on my own copy of CLWEB these days
<flip214>
and even if it werent you could just write symbols as comments as well … you only need to be careful about punctuation ".,'" ;) oh no, now the line is wasted!
<LdBeth>
If you're interested in typesetting with plainTeX...
<|3b|>
flip214: i mean if you still see comments, you were starting from characters and doing something like READ rather than just getting a list
<|3b|>
(or are getting list input from someone who parsed text input and is reasonably likely to have preserved formatting along with comments)
<LdBeth>
What size is recommanded for the hash table of ## labels?
<beach>
russellw: That's very unfortunate.
<beach>
LdBeth: What do you mean? A hash table will grow as needed.
<LdBeth>
beach: I think I want it initialize as fast as possible
<beach>
LdBeth: To read code?
<LdBeth>
yes, beach. It is not frequently used but should not missing
<beach>
LdBeth: I think it won't matter much. My hunch is that the I/O is going to dominate anyway.
<LdBeth>
Ok, just use default size
<beach>
LdBeth: What is it that you are working on that requires this much attention to performance?
<beach>
Another reader?
<LdBeth>
it's a macro layer used to parse documents into lisp code and TeX, which usesthe standard reader macro.
<beach>
So you are implementing a superset of the Common Lisp reader?
<LdBeth>
Most control sequences are mature, I will mainly focus on improving data structures
<LdBeth>
For this one I think the standard reader is sufficient
earl-ducaine has joined #lisp
<beach>
But then you don't control the size of the hash table for labels.
<LdBeth>
I tested with several sizes, seems on CCL the default size works good enough
<LdBeth>
it involves memory alignment, i suppose
<beach>
How did you test those sizes? By using implementation-specific code?
<LdBeth>
by running the actual function with differenct size parameters
<LdBeth>
you mean the profiler?
<beach>
No, I mean, I don't see a way in the Common Lisp standard to control the size of that hash table, so if you used the standard Common Lisp reader, I don't understand how you can test different sizes unless you use implementation-specific code.
<LdBeth>
Ah, so I sets macro chars to my own function, with set-dispatch-macro-character
<beach>
I thought we were talking about the hash table for #= and ##. No?
<beach>
And you do this for the sole purpose of controlling the size, or do you have any other use for controlling this aspect of the reader?
makomo has joined #lisp
<no-defun-allowed>
i wonder if it's possible to compile a subset of CL to verilog
<LdBeth>
It reads lisp into tokens so something like read time eval, comments, optional evals can be preserved
<shka_>
well, + is a subset as well sooooooooo ;-)
<LdBeth>
#'no-defun-allowed: of course, and you can even do formal verification
<beach>
LdBeth: Got it.
fluxit has joined #lisp
<|3b|>
beach: what's the next part i need to specialize once i have HIR?
jasongrossman has joined #lisp
<|3b|>
looks like HIR-TO-MIR calls SPECIALIZE
<earl-ducaine>
Does there seem to be a generally prefered thead portability library? I was looking at Bordeaux Threads but I'm worried the API (at least the ql) seems to have changed quite a bit.
<|3b|>
bordeaux threads is the preferred thread lib
<earl-ducaine>
Maybe I just got unlucky and stumbled upon a prehistoric version in Trac?
<|3b|>
(or things that use it, if you want a higher-level API like lparallel)
<earl-ducaine>
This is the document I was looking at:
<beach>
|3b|: We don't do much beyond HIR at the moment.
<beach>
|3b|: The plan is for me to define a HIR->MIR translator that can be customized.
<beach>
|3b|: But I haven't done that yet.
<|3b|>
and MIR is still similar to HIR?
<beach>
Yes, except that address calculations are exposed and the objects manipulated may be machine data.
<|3b|>
ok, no address calculations since i'm running on a VM anyway
<|3b|>
so i can just start working on codegen from the HIR, since i probably wouldn't be changing much going to MIR anyway
<beach>
Yes, that sounds very reasonable.
<beach>
|3b|: Depending on the kind of HIR you generate, you will need to generate code for array referencing, CONS-cell referencing, standard-object referencing, perhaps fixnum arithmetic, etc.
<|3b|>
yeah, will need to figure out how to configure that stuff at some point
* |3b|
's current test function needs to pass function arguments and a literal int to 2 native functions
<|3b|>
then i need to figure out how to specify that i'm compiling the function for the native calling convention, and assemble it into something i can put in an apk file
<|3b|>
(specifying the calling convention is one of the things where dropping user/implementation declarations came up)
<|3b|>
actually, i guess i can walk up the tree of environments to find it, so maybe dropping it isn't a problem after all
<shrdlu68>
Is sb-kernel:hairy-data-vector-ref a potential optimization point?
<|3b|>
usually
<|3b|>
types for arrays and array indices can help
<|3b|>
*declaring types
razzy has joined #lisp
<|3b|>
and possibly using specialized arrays. i don't remember specifically which things that was for, but usually when i'm optimizing i want specialized arrays anyway :)
<no-defun-allowed>
shka_: (+ a b) -> `assign foo = a + b`
<beach>
|3b|: Interesting project I see.
<|3b|>
(be sure to validate the declarations first if running on other lisps or running at safety 0)
igemnace has joined #lisp
<|3b|>
hmm, i guess i have one more step at IR level, need to distinguish whether i'm compiling for native or lisp
xrash has quit [Ping timeout: 252 seconds]
<|3b|>
and propagate some of that env info into the IR while i still have env
<|3b|>
anyone have thoughts on how best to represent java's super keyword in a lisp style? (super (foo this ...))? ((super foo) this ...)? (foo (super this) ..)? (funcall-super 'foo this ...)?
<|3b|>
aside from the specific cases that can just be (call-next-method)
<|3b|>
(if i understand correctly, super is sort of generalized call-next-method, that lets you call any method or access a slot starting dispatch with superclass instead of actual class)
<TMA>
it makes no sense in multiple inheritance setting
* |3b|
currently is thinking of (foo (super this) ...) as best representing what it does
<|3b|>
TMA: yeah, purely for single dispatch
<|3b|>
though you could sort of generalize it to multiple, grab classes for current method, find applicable methods for requested function, remove first entry if exact match for current method
<TMA>
|3b|: is this special? [not in (declare (special this)) but extraordinary, separately handled etc.]
* |3b|
isn't sure that would actually be a useful thing to do though :)
<|3b|>
yeah, this is for compiling code to interact with java APIs, so i need to subclass java classes, and call methods from superclass, etc
<TMA>
|3b|: if so, just use SUPER as an extraordinary marker as well
<|3b|>
so first need to distinguish it from an intentional recursive call, and 2nd need to generate different bytecode
<|3b|>
TMA: right, but where do i put the marker? :)
<TMA>
instead of THIS you put there SUPER
<|3b|>
well, currently THIS is an explicit argument in my lispy code, so generating SUPER sounds a bit off
<TMA>
not (super this) ... the latter would seem to enable you to use (super foo) for any foo, which is not the case from your description anyway
<|3b|>
actually, maybe that would work for any foo, will have to try that on the VM
<|3b|>
(once i get to that point)
* |3b|
isn't sure if foo.super.bar() is valid in java
<TMA>
it is not
xificurC has quit []
<Shinmera>
super is private
* |3b|
isn't sure only working on some things is too much of a problem, same way you can't call call-next-method on aything that would change applicable methods
<|3b|>
also seems like it would be fairly rare, since most places could just use call-next-method
<TMA>
using super for other purposes than the equivalent of call-next-method is considered bad style even in java itself
<|3b|>
cool, i'll just leave it as (super this) for now and ignore it :)
<|3b|>
and leave it as an implementation detail with call-next-method as the intended API
frgo has quit [Remote host closed the connection]
<TMA>
[the usage super(argument1, argument2); in constructors is a (call-next-method) of sorts albeit you can choose which superclass constructor you use]
<|3b|>
right
pjb has joined #lisp
<shrdlu68>
Spec says "vector is analogous to list", so is (aref <vector> n) the same as (nth n <list>)?
<|3b|>
aside from runtime and improper lists, pretty much
<|3b|>
ah, i think it means the functions VECTOR and LIST rather than the types
<|3b|>
(though the types are also analogous to the extent both are sequences)
<beach>
|3b|: You know that the environment is gone by the time you have an AST, so you might need to define a new AST type as well.
<|3b|>
(elt <vector> n) is like (elt <list> n) :)
<|3b|>
beach: yeah, i think that's part of what i meant
<beach>
OK.
<|3b|>
or at least i would have noticed quickly :)
<beach>
Absolutely! :)
<|3b|>
hard to tell what needs stored where until i try to use it, so probably will be a few passes of this sort of thing
graphene has quit [Remote host closed the connection]
graphene has joined #lisp
housel has quit [Remote host closed the connection]
cl-arthur has quit [Quit: Lost terminal]
housel has joined #lisp
<pjb>
shrdlu68: aref is not the same as nth: (nth 4 '(1)) #| --> nil |# (aref #(1) 4) #| ERROR: Array index 4 out of bounds for #(1) . |#
<pjb>
shrdlu68: elt is like aref: (elt '(1) 4) #| ERROR: 4 is not a valid sequence index for (1) |#
<pjb>
this is why we have both nth and elt.
<pjb>
The older NTH is nice and gentle. It doesn't rely on a condition system, it just returns NIL.
dmiles has quit [Ping timeout: 245 seconds]
frgo has joined #lisp
X-Scale has quit [Ping timeout: 245 seconds]
<pjb>
The newer AREF and ELT are harsh and strict. They rely on the condition system, and like to dominate and control you.
<pjb>
You use whatever fits your personality.
<pjb>
There are even people who specify the type of array elements!
frgo has quit [Ping timeout: 240 seconds]
anewuser has joined #lisp
Balooga_ has joined #lisp
dmiles has joined #lisp
X-Scale has joined #lisp
<adlai>
no-defun-allowed: "a subset", sure, there are many trivial subsets (eg, bit vector operations, "logical" integer operations, etc). the interesting question is hunting for maximal feasible subsets.
<no-defun-allowed>
True
<no-defun-allowed>
There's a few simple parallels: = is let, <= is setf
frgo has joined #lisp
frgo has quit [Ping timeout: 252 seconds]
orivej has joined #lisp
cl-arthur has joined #lisp
<|3b|>
hmm, maybe it does eat the declarations in a way i can't get by walking up the env list
m00natic has joined #lisp
frgo has joined #lisp
frgo has quit [Ping timeout: 252 seconds]
igemnace has quit [Quit: WeeChat 2.2]
frgo has joined #lisp
SenasOzys has quit [Ping timeout: 260 seconds]
t0adst00l has joined #lisp
rumbler31 has joined #lisp
rumbler31 has quit [Ping timeout: 252 seconds]
igemnace has joined #lisp
Balooga_ has quit [Quit: Balooga_]
lagagain has joined #lisp
makomo has quit [Quit: WeeChat 2.2]
regreg has joined #lisp
arkaros has joined #lisp
Josh_2 has joined #lisp
<Josh_2>
Afternoon all
<russellw>
afternoon!
<shka_>
ok, so i was reading on SDRs used for HTM
<shka_>
and it is very much like data sketch
ggole has joined #lisp
<shka_>
but how they are using those is quite interesting!
xrash has joined #lisp
xrash has quit [Remote host closed the connection]
SilverSteeples has joined #lisp
SenasOzys has joined #lisp
DGASAU has quit [Ping timeout: 252 seconds]
eschulte has quit [Ping timeout: 252 seconds]
regreg_ has joined #lisp
regreg has quit [Read error: Connection reset by peer]
orivej has quit [Ping timeout: 252 seconds]
<hjudt>
is there any lisp project that has a docker file i can look into? i'd like to deploy a project to an openshift cluster.
jasongrossman has left #lisp ["ERC (IRC client for Emacs 26.1)"]
<dim>
it was contributed a nice spec to have a builder image different from the runtime image, wherein there's just the lisp image itself and its .so dependencies
Bike has joined #lisp
orivej has joined #lisp
rnmhdn has joined #lisp
doubledup has quit [Quit: Leaving]
Kundry_Wag has joined #lisp
graphene has quit [Remote host closed the connection]
graphene has joined #lisp
graphene has quit [Remote host closed the connection]
pierpal has joined #lisp
graphene has joined #lisp
orivej has quit [Ping timeout: 260 seconds]
Guest5800_ has joined #lisp
rnmhdn has quit [Ping timeout: 240 seconds]
arkaros has quit [Ping timeout: 272 seconds]
warweasle has joined #lisp
PuercoPop has joined #lisp
rumbler31 has joined #lisp
pierpal has quit [Read error: Connection reset by peer]
pierpal has joined #lisp
arkaros has joined #lisp
Josh_2 has quit [Read error: Connection reset by peer]
zfree has joined #lisp
Kundry_Wag has quit [Read error: Connection reset by peer]
rumbler31 has quit [Remote host closed the connection]
SenasOzys has quit [Remote host closed the connection]
rnmhdn has joined #lisp
SenasOzys has joined #lisp
Inline has joined #lisp
Inline has quit [Read error: Connection reset by peer]
Inline has joined #lisp
<phoe>
_death: I found the cause of the random socket errors I was getting
<phoe>
basically - socket operations are *not* thread-safe, and performing them at once from different threads causes things to explode.
<shka_>
phoe: you have multiple threads reading one socket?
rnmhdn has quit [Ping timeout: 252 seconds]
cage_ has joined #lisp
<phadthai>
hmm yes if they're thread-safe or not depends on OS, thread implemenation and polling method (signals may also be sent to the whole process, handled by whatever thread is next scheduled)
frgo has quit [Remote host closed the connection]
<phadthai>
s/implemenation/implementation/
<phoe>
shka_: I had
<phoe>
I don't have anymore
<shka_>
ok
<shka_>
btw, even if it would work, it would still be not the greatest idea
lavaflow_ has quit [Read error: Connection reset by peer]
frgo has joined #lisp
frgo has quit [Read error: Connection reset by peer]
frgo has joined #lisp
astronavt__ is now known as astronavt
lavaflow_ has joined #lisp
sleepnap has joined #lisp
zxcvz has joined #lisp
<phadthai>
so usually we can disable signals we don't want in threads we don't want them (usually using pthread_sigmask), configure sockets to not send signals like sigint and block/unblock status (via setsockopt), use custom locks/mutexes around syscalls like recv/accept, etc...
<phadthai>
(something your abstraction lib may or may not allow too)
ja-barr has quit [Ping timeout: 272 seconds]
ja-barr has joined #lisp
ealfonso has joined #lisp
frodef has quit [Ping timeout: 244 seconds]
pierpal has quit [Read error: Connection reset by peer]
Kundry_Wag has joined #lisp
pierpal has joined #lisp
pierpal has quit [Read error: Connection reset by peer]
shifty has quit [Ping timeout: 260 seconds]
frodef has joined #lisp
pierpal has joined #lisp
heisig has quit [Ping timeout: 252 seconds]
travv0 has joined #lisp
graphene has quit [Remote host closed the connection]
graphene has joined #lisp
edgar-rft has quit [Read error: Connection reset by peer]
SenasOzys has quit [Remote host closed the connection]
SenasOzys has joined #lisp
slyrus1 has joined #lisp
DataLinkDroid2 has joined #lisp
shka_ has quit [Quit: WeeChat 1.9.1]
DataLinkDroid has quit [Ping timeout: 256 seconds]
rnmhdn has joined #lisp
shrdlu68 has quit [Ping timeout: 252 seconds]
nowhereman_ has quit [Ping timeout: 250 seconds]
SenasOzys has quit [Ping timeout: 245 seconds]
astalla has joined #lisp
jeosol has joined #lisp
SenasOzys has joined #lisp
Kundry_Wag has quit [Remote host closed the connection]
travv0 has quit [Ping timeout: 260 seconds]
frgo has quit [Remote host closed the connection]
frgo has joined #lisp
igemnace has quit [Remote host closed the connection]
frgo has quit [Ping timeout: 244 seconds]
graphene has quit [Remote host closed the connection]
graphene has joined #lisp
igemnace has joined #lisp
frgo has joined #lisp
orivej has joined #lisp
arkaros has quit [Ping timeout: 250 seconds]
frgo has quit [Ping timeout: 240 seconds]
DataLinkDroid3 has joined #lisp
asarch has joined #lisp
DataLinkDroid2 has quit [Ping timeout: 256 seconds]
Kundry_Wag has joined #lisp
FreeBirdLjj has joined #lisp
rumbler31 has joined #lisp
rumbler31 has quit [Ping timeout: 252 seconds]
pierpal has quit [Read error: Connection reset by peer]
pierpal has joined #lisp
sjl has joined #lisp
frgo has joined #lisp
dale_ has joined #lisp
dale_ is now known as dale
arkaros has joined #lisp
frgo has quit [Ping timeout: 252 seconds]
arkaros has quit [Ping timeout: 252 seconds]
Kundry_Wag has quit [Read error: No route to host]
m00natic has quit [Remote host closed the connection]
arkaros has joined #lisp
frodef has quit [Ping timeout: 252 seconds]
<cgay>
I wonder where Land of Lisp's summary came from. "...but it's cryptic syntax...." Way to sell books, guys.
arkaros has quit [Ping timeout: 245 seconds]
<dlowe>
I'm not a big fan of Land of Lisp for a number of reasons.
<dlowe>
It seems to work for some people, though.
LiamH has joined #lisp
<aeth>
I recently looked through the book (it was part of an ebook bundle a long time ago) and it didn't look as bad as some people said. It moves pretty slowly, though. Takes a while to get to useful features of the language.
moei has quit [Quit: Leaving...]
<aeth>
It's dated, though. It recommends CLISP, whereas these days people generally recommend SBCL. And it mentions Clojure and Arc as up-and-coming Lisps.
<aeth>
(It's not just a recommendation for CLISP, the web chapter requires it.)
<Demosthenex>
hrm
<pjb>
Well, clisp has a good socket interface.
FreeBirdLjj has quit [Remote host closed the connection]
FreeBirdLjj has joined #lisp
FreeBirdLjj has quit [Ping timeout: 264 seconds]
<russellw>
What would be the most idiomatic way to order parts of a name? That is, in Scheme, there are lots of names like string->number. In my opinion, it would arguably make more sense to write such names like number-string or number-of-string when we are dealing with pure functions, because the function name is written before the argument, but string->number is the Scheme idiom. What is the idiom in
<russellw>
Common Lisp?
longshi has joined #lisp
<Bike>
i think i've used both foo->bar and foo-to-bar
<Bike>
in the actual language, usually those things take a variety of possible arguments
<Bike>
so e.g. the thing to get a string out of a symbol or character is just called 'string'
<russellw>
right, which is why the distinction does not show up in the standard library. But as far as you are concerned, the idiom is from->to, not to-of-from?
<Bike>
i'm mostly talking about my preference. i don't know that it's something there's really an idiom for.
<russellw>
right, but I need some tiebreaker
<Bike>
okay, then i would do from->to, yes.
<russellw>
cool, thanks
<russellw>
the specific use case that prompted the question is a pair of test functions, one of which tries reading and then writing, the other of which tries writing then reading
<pjb>
russellw: indeed, number<-string in scheme, number-from-string in CL would lead to more readable code. But still, left-to-right transformations are more common.
<pjb>
the -of- or -from- or <- connectors leads to more readable code than the -to- or -> or just - connectors.
<cgay>
a bold statement
<pjb>
Notice on the other hand, that in CL, you can specify the prefix in defstruct, and you can use foo. instead of foo-
<pjb>
So you write (human.head human)
<pjb>
But usually, and by default, it's human-head.
<russellw>
and I think given that the idiom is as it is, it is probably better if I follow it rather than doing things a different way
<pfdietz>
The alternate style is that of generic function accessors for standard classes. These typically omit the prefix entirely. arms, head, etc.
scymtym has quit [Ping timeout: 240 seconds]
orivej has joined #lisp
<pfdietz>
Because generic functions do not belong to any particular (base) class.
<pfdietz>
Are not members of, I should say.
Josh_2 has joined #lisp
shrdlu68 has joined #lisp
esrse has quit [Ping timeout: 244 seconds]
Jesin has quit [Quit: Leaving]
pierpal has quit [Ping timeout: 246 seconds]
<shrdlu68>
I have a binary tree data structure such that each node consists of 'left-child, 'right-child, and 'data (which is some arbitrary object stored at the node). I'm currently using a vector to hold the values, initialized as (vector nil nil nil). What's a better alternative, performace-wise?
<pjb>
none.
<shrdlu68>
I've tried a CLOS object, which was sub-par.
<pjb>
you can (defstruct (node (:type vector)) left right data) for ease of use.
<russellw>
I would expect on theoretical grounds a structure should be slightly faster because the bounds check can be omitted, but the actual performance difference too small to measure
terpri has joined #lisp
<pjb>
russellw: actually, I don't think structures can do without the bound check: (defstruct 2d-point x y) (defstruct (3d-point (:include 2d-point)) z) (3d-point-z (make-2d-point)) vs. (2d-point-y (make-3d-point))
<pjb>
But you should realize that it's silly to opimize the access to the node slots of a binary tree, when the access to the tree is a slow O(logn), when you could have O(1) with a hash-table!?!
<shrdlu68>
pjb: I know, trying to see how far I can take this.
<shrdlu68>
It's a space-time trade-off.
ggole has quit [Quit: ggole]
<pjb>
If you start using vectors for binary trees, why not using only 1 vector?
<shrdlu68>
The keys are bit vectors of length 1-240. I just walk the bit vector going left or right.
longshi has quit [Ping timeout: 250 seconds]
<LdBeth>
maybe for fast concatenation and substring
<russellw>
if all you want to do is lookup, I would expect a hash table to be both faster and smaller
<shrdlu68>
russellw: Faster, yes. Smaller, no.
<russellw>
why?
<russellw>
are bit vectors not stored very compactly if you just leave them in that form?
shka_ has joined #lisp
<shrdlu68>
I assume it's because it keeps a copy of each key for comparison.
<shka_>
good enving
<shrdlu68>
Hey shka_
<russellw>
... okay, I suppose it is still 32 bytes/key
<russellw>
evening!
<LdBeth>
Hello
<shrdlu68>
About 10 times faster, though.
<russellw>
yeah
<shka_>
shrdlu68: you figured something out?
<shrdlu68>
shka_: Nope.
Jesin has joined #lisp
<shka_>
oh, ok
<shrdlu68>
I tried using an octet vector rather than a bit vector, but octet vector was a bit slower than the bit vector.
<shka_>
as hash key?
<shka_>
shrdlu68: i had one idea
<shrdlu68>
Yes. I had to subseq and mask some of the bits of the last octet where length was not a multiple of 8.
zxcvz has quit [Ping timeout: 260 seconds]
<shka_>
well, i have something different you can try out
<russellw>
how many of these things are you storing, that you need to worry about 32 bytes per key?
<shka_>
basicly try something like this
<shka_>
still a tree
<shka_>
but composed of hashtables
<shka_>
so simply divide your key into fixnum chunks
<shka_>
it shouldn't be ultra hard to implement
<shrdlu68>
russellw: A lot, I'm keeping record of all 240-bit prefixes in a bit vector.
<russellw>
supposing you have an atom to convert to a (read)able string, is this the fastest/simplest way to do it? (format nil "~s" a)
<whartung>
uh.. “all 240-bit prefixes”, we only have 64 bit addressing…arent; you going to run out? “I know, I’ll use virtual memory! — Now you have 2 problems.”
<shrdlu68>
Actually all prefixes of length 1-240.
<whartung>
yea, you’re still going to run out of bits.
<shrdlu68>
whartung: I take a file, read it into an octet vector, convert that into a bit vector, and, for each bit, look back 240 bits, keeping track of the last (n=240) bits.
<whartung>
how big are these files?
warweasle_afk is now known as warweasle
<shrdlu68>
Around 200kb.
<russellw>
WTF
<shrdlu68>
Yep.
<russellw>
your entire data set will only be hundreds of kilobytes!
<whartung>
so you’re storing 200,030 bytes total.
<russellw>
you could run that on a 286
<russellw>
use the hash table and be happy
<whartung>
(be boiled in oil and fried in crisco if I ever call bytes octets)
<shrdlu68>
I think there's something I'm not explaining clearly.
<Blackbeard>
hello
arkaros has quit [Ping timeout: 272 seconds]
<Blackbeard>
in a .asd file
<Blackbeard>
should i put
<shrdlu68>
I've tried hashtables, takes around 5s to process a 36k file.
<Blackbeard>
:depends-on (#:stumpwm
<Blackbeard>
#:trivial-mimes)
<Blackbeard>
or
<Blackbeard>
:depends-on ("stumpwm"
<Blackbeard>
"trivial-mimes")
<shka_>
russellw: simply keys are long
<shka_>
and calculating hashes is slow
<Shinmera>
Blackbeard: both are ok.
<whartung>
the keys are 30 bytes long
<Blackbeard>
Shinmera (IRC): i see
<Blackbeard>
Shinmera (IRC): thanks :)
<russellw>
but he already said the hash table is 10 times as fast as the binary tree?
<shrdlu68>
Yes. But consumes much more memory.
<russellw>
Hundreds of kilobytes
<shrdlu68>
Nope, gigabytes.
<russellw>
you said the files are 200K?
<shrdlu68>
Actually my test file right now is /bin/cat, which is 36k
<shrdlu68>
I can barely process a 1MB file.
<russellw>
so where do you get gigabytes?
sauvin has quit [Read error: Connection reset by peer]
<shrdlu68>
russellw: 30Mb x 35000
<whartung>
where’s the 30MB coming from?
<shrdlu68>
Oh wait, 30b x 35000
<whartung>
that’s 1M
<whartung>
ish
<whartung>
add 8 bytes of overhead, and it’s 1.2M
<whartung>
1.3
<russellw>
also '5s to process a 36k file'? on a modern computer, it should be much faster than that. Calculating hashes is nowhere near that slow. By orders of magnitude. Have you profiled it? What compiler are you using?
<russellw>
so if I understand that correctly, you are using SBCL, which is generally reckoned the fastest available compiler, and it is indeed spending a large percentage of the total time calculating hashes. Okay, that is bizarre, and I have no explanation for it. Should be several orders of magnitude faster
<russellw>
afk
<shrdlu68>
russellw: Other implementations take so long I haven't really let them run to completion.
<pjb>
russellw: err, sbcl is one of the slowest compilers, if not the slowest.
<jasom>
it's entirely possible that the hash function for bitvectors is very bad.
<pjb>
Well, I believe. You would have to benchmark it.
<jasom>
and I see 3.7% of the time in sxhash, right?
<whartung>
3.7% shouldn’t dominate the discussion. It’s interesting, but obviously not “the reason” its slow
<whartung>
I assume this: (FLET "BODY-FUN-0" :IN SB-IMPL::GETHASH3) includes the body of the function?
<whartung>
in it’s times?
SenasOzys has joined #lisp
<shka_>
jasom: 3.7% self
<jasom>
shka_: 3.7% cumulative 0.8% self
graphene has quit [Remote host closed the connection]
<shka_>
no idea, but i would expect sbcl to have different logic for short bitvectors and long ones
<shrdlu68>
"enhancement: The value of SXHASH on bit-vectors of length equal to the word size now depends on the vector's contents instead of being constant; its value on bit-vectors of length divisible by the word size now depends also on the contents of the vector's last word." => http://www.sbcl.org/all-news.html
<shka_>
ouch
<shka_>
i think that in your case you will get plenty of collisions
<shka_>
anyway, it all makes sense
<jasom>
it uses a poor mixng function though; the longer the bit vector the more the difference, but even with 30 bytes the first 9 decimal digits are identical
<shka_>
jasom: question is: does it matter for hashtable implementation
vlatkoB has quit [Remote host closed the connection]
<shka_>
it may not need high avalanche factor
voidlily has quit [Remote host closed the connection]
<shka_>
good night everyone
<shrdlu68>
Goodnight shka_
<vtomole>
Good night
<jasom>
in any event 1M calculations of sxhash are nearly instantaneous
voidlily has joined #lisp
<shka_>
jasom: since they are literally just taking 64 last bits!
<shka_>
yeah, this is weird
<shrdlu68>
And here I thought I was pushing the limits of modern computers :P
<shka_>
(FLET "BODY-FUN-0" :IN SB-IMPL::GETHASH3) this stuff is weird
foom2 has joined #lisp
<shka_>
i wonder what it is doing…
<shka_>
anyway, off to bed
foom has quit [Ping timeout: 240 seconds]
<jasom>
however, the hash-table-size is only 4M when it hits 1GB of heap on my machine
tomaw- is now known as tomaw
rumbler31 has joined #lisp
regreg_ has quit [Ping timeout: 250 seconds]
shka_ has quit [Ping timeout: 252 seconds]
earl-ducaine has quit [Ping timeout: 252 seconds]
MichaelRaskin has joined #lisp
<russellw>
pjb, oh, which ones are faster? I thought nearly everyone was using SBCL
<oni-on-ion>
speed of compiling and speed of produced code are different metrics
<jasom>
space is almost entirely simple-array-unsigned-byte-64 objects
<russellw>
right, the conversation was about speed of the produced code
<shrdlu68>
Is that how bit vectors are implemented internally?
<Bike>
bitvectors are done as word vectors i think, yeah
<jasom>
I misspoke, it's only 20% byte-64 vectors, but it's the largest fraction
<jasom>
nope
<jasom>
because it's also about 17% simple-vit-vectors
<jasom>
maybe bit vectors that are a multiple of the word size are implemented that way?
<jasom>
no, that's wrong
<jasom>
Perhaps the hash tables use word vectors when there are collisions?
<jasom>
because there are ~3k of the word vectors but they are using over 100M of ram
<jasom>
also how many bit vectors are there supposed to be? I'm seeing 4M already and it hasn't yet run out of ram on my 1GB heap
<oni-on-ion>
heh @ last bullet item "(only with users permission and for maintenance reasons)" isnt that FB
<oni-on-ion>
wrong channel.
trafaret1 has joined #lisp
<trafaret1>
hi there
<oni-on-ion>
if you dont need to modify the data as they come from files, perhaps using a more efficient data structure than that ?
<oni-on-ion>
hi
slyrus1 has joined #lisp
<trafaret1>
need some help
<trafaret1>
I'm wondering about effective working wtih tables
<trafaret1>
does it solution to use lisp and mysql
slyrus has quit [Ping timeout: 252 seconds]
slyrus1 is now known as slyrus
warweasle has quit [Quit: rcirc on GNU Emacs 24.4.1]
<dim>
shrdlu68: try CCL, the GC is much better than SBCL's one in my playing around with pgloader
<dim>
+experience (maybe)
<Blackbeard>
Shinmera (IRC): oh thanks :)
<vtomole>
What's wrong with sblc's gc?
<vtomole>
*sbcl
rnmhdn has quit [Ping timeout: 252 seconds]
rumbler31 has quit [Remote host closed the connection]
<jasom>
nothing wrong with GC in this case though Shinmera
<jasom>
rather shrdlu68
<jasom>
shrdlu68: heap will almost always be exhasted during GC; its either during GC or during a point where GC is excluded.
Kevslinger has quit [Quit: Connection closed for inactivity]
<jasom>
it looks like the hash table is about to grow to ~600MB and we are using 500MB, so that checks out.
<dim>
vtomole: when using pgloader users often reach heap exhausted/game over message in situations where CCL pile through the work at a fraction of the memory usage; it might be my code though
<dim>
but when users report problems with SBCL and big/huge data sets being processed by pgloader, I know I can just recommend CCL and it's going to be ok.
<jasom>
sbcl is very conservative about *when* to invoke the GC, so I've seen heap exhausted when there would have been room had the GC been run sooner
<shrdlu68>
Me too.
<jasom>
i.e. I see heap exaustion errors fixed by just putting a (gc) somewhere in the loop
pierpal has joined #lisp
cage_ has quit [Quit: Leaving]
<jasom>
but in this case it's jsut that its growing the hash-table to be more than 1/2 the heap which just isn't going to work
<jasom>
but it looks like we have 4M objects in less than 2k buckets, which explains a lot. However ~200MB of simple-bit-vectors seems excessive if the original statement that it's only 1 vector per bit in the file.
<jasom>
4 million bit vectors would be 512k not the 32k that I'm testing on.
scymtym has joined #lisp
<shrdlu68>
30 vectors per bit,
lemonpepper24 has joined #lisp
<jasom>
oh, that makes sense then. Space usage isn't that high; looks like its 8bytes+size of bit-vector per bit vector and then thhere's another over 100M for the single-float arrays
graphene has quit [Remote host closed the connection]
<jasom>
so it's ~300M of data for 100M of hash-table overhead which isn't great but not terrible.
zxcvz has quit [Quit: zxcvz]
graphene has joined #lisp
arkaros has joined #lisp
<jasom>
however, it looks like it may be copying the data on a rehash, as that's the only thing that makes sense for a 600M allocation. I'll inspect the code
arkaros has quit [Ping timeout: 240 seconds]
<jasom>
also it looks like it always rounds the hash table size up to a power of two, which makes the default rehash-size of 1.5 rather stupid
<jasom>
total hash-table overhead seems to be 4 times the word size of the number of elements rounded up to a power of two
<jasom>
so a 64-bit target the overhead of storing 2^22+1 items would be ~256M.
<minion>
Remembered. I'll tell shrdlu68 when he/she/it next speaks.
jkordani_ has quit [Ping timeout: 252 seconds]
pierpal has quit [Read error: Connection reset by peer]
pierpal has joined #lisp
frodef has quit [Ping timeout: 244 seconds]
<razzy>
aaaah, i remembered shrdlu :]
<aeth>
pjb: whenever I have something where a compose macro/function/whatever might seem useful, I find that it's probably better just to combine two of the three compositions into one new, trivial, inline function
quipa has joined #lisp
<aeth>
i.e. instead of (another-function (some-function (symbol-to-keyword foo))) I can have (another-function (some-function* foo)) where the variation of some-function converts symbols to keywords
<aeth>
(I say i.e. instead of e.g. because it's pretty much always that, usually even symbol-to-keyword!)
<aeth>
This is common when I have a function with a case working on keywords but I take in arbitrary symbols from a macro.
frodef has joined #lisp
kushal has quit [Ping timeout: 256 seconds]
kushal has joined #lisp
comborico1611 has joined #lisp
dkmueller has joined #lisp
rumbler31 has joined #lisp
dkmueller has quit [Client Quit]
lumm has quit [Ping timeout: 245 seconds]
graphene has quit [Remote host closed the connection]
<LdBeth>
jasom (IRC): I think you would be interested to read Expansion Passing Style paper
rumbler31 has quit [Ping timeout: 252 seconds]
<LdBeth>
That was a macro system proposed for both CL and Scheme, but seems never been adopted
<housel>
Didn't scheme48 implement it? I can't recall precisely
longshi has joined #lisp
longshi has quit [Client Quit]
<LdBeth>
Scheme48 uses typical “hygienic” syntax
<housel>
Yes, but I thought under the hood it used expansion passing
Bike has joined #lisp
blt has quit [Ping timeout: 240 seconds]
varjagg has quit [Ping timeout: 240 seconds]
blt has joined #lisp
mutenewt has quit [Quit: Leaving]
<ober>
islisp macros are in the middle of the two extreemes right?
<aeth>
qapples: ##lisp is the channel for the Lisp family of languages and #lisp is the channel for Common Lisp
longshi has joined #lisp
<aeth>
qapples: Besides #racket you may also want to go to #scheme
<ober>
aeth: nice, and contrary to John McCarthy's wishes
ober has left #lisp ["ERC (IRC client for Emacs 26.1)"]
<aeth>
ober: If #lisp had the topic of ##lisp and we were in #cl or some other channel instead, then #lisp would be as dead as ##lisp is
<aeth>
Generally people want to talk about specifics.
asarch has quit [Quit: Leaving]
blt has quit [Ping timeout: 240 seconds]
<aeth>
(It's a coward move to make a claim and then leave before anyone can respond. So I posted my response even though I noticed the person I responded to left.)
Denommus has joined #lisp
<Bike>
mccarthy's wishes lol
<aeth>
I only do what our leader McCarthy would want us to do.
dto has joined #lisp
<Blackbeard>
aeth: don't worry you are right (:
<housel>
I've been in this channel off and on since 2003, I don't remember him ever coming here and opining about the channel topic
<dto>
any parenscript experts here? i'm a bit confused. the function APPEND is in the manual, but showing up as undefined.
<aeth>
There isn't even one definition of "Lisp". You could mean "traditional Lisp", in which case only elisp and CL count (for the living, modern Lisp languages). You could expand it a bit and include Scheme (and probably have to split this expansion in two, one excluding Racket because of its immutable conses and one including Racket). You could also be all-inclusive (e.g. Clojure, Hy, etc.). And there are probably other, messier ways that would
<aeth>
messier ways that wouldn't just make each definition a superset of the previous.
<aeth>
And an active channel devoted to the Lisp family would probably debate over definitions half of the time unless it had an official definition.
<aeth>
(Or, alternatively, it would be like /r/lisp and be 90% about Common Lisp)
nowhere_man has joined #lisp
nowhereman_ has quit [Read error: Connection reset by peer]
LiamH has quit [Quit: Leaving.]
Guest5800_ has quit [Quit: Connection closed for inactivity]
dale has quit [Quit: dale]
<aeth>
Personally, I would divide Lisps into traditional Lisps (CL is one), CL-likes (e.g. Parenscript, which calls itself "an extended subset of Common Lisp"), Schemes (e.g. Guile), Scheme-likes (e.g. Racket), and other.
rumbler31 has joined #lisp
<oni-on-ion>
why u so mean to parenscript, its use case is for outputting javascript, not being a CL implementation =)