<beneroth>
"They are problematic in compiler use (which is an example of an activity generally taking place to a great extent outside of compiler development)."
<beneroth>
wut?
<beneroth>
does this sentence make any sense, or is it just word picking?
<Regenaxer>
I don't understand it
<beneroth>
me neither
<Regenaxer>
Never mind :)
<beneroth>
maybe it means that the problem (FEXPRs cannot be compiled, generally) shows when USING a compiler, not when developing one? so stupid wordplay?
<beneroth>
the other sentence I first thought had something to it
<beneroth>
"The research literature is not very broad and often hints at semantic problems."
<Regenaxer>
I'm not sure if they grasp the power of FEXPRs
<Regenaxer>
they are ignored in CL
<beneroth>
I'm sure they don't :)
<beneroth>
I compare FEXPR to a cli tool, deciding itself what to do with the command line arguments it gets. or FEXPR basically being a whole VM/interpreter as a function.
<Regenaxer>
right
<beneroth>
thx
<beneroth>
as we said yesterday, software is dynamic re-arranging electronics.
<Regenaxer>
:)
<beneroth>
an interpreter is more dynamic than hardcoded compiled binary.
<Regenaxer>
In fact when *we* talk about FEXPRs, we think of running a 'prg'
<Regenaxer>
FEXPRs by themselves are more
<Regenaxer>
simply non-evaluating funs
<Regenaxer>
In Pil the common use is with prg
<Regenaxer>
but that makes no sense in CL
<Regenaxer>
So people are not aware of it
<beneroth>
oh
<Regenaxer>
With they "are more" I meant "are more general"
<beneroth>
"non-evaluating funs" or "funs not evaluating their arguments automatically when called" ?
<Regenaxer>
yes, both ok
<Regenaxer>
In case of pil non-evaluating the last arg
<beneroth>
so you say not only do "they" not know what they talk about when talking about FEXPRs, but neither do we (well, me) ?
<Regenaxer>
We know, but we think of the most common use case
<beneroth>
the last arg, which might be a list of args
<Regenaxer>
A single arg, which may be a list
<beneroth>
so I should find out what FEXPR meant historically in common lisp
<beneroth>
first sentence of wikipedia on FEXPR article: "In Lisp programming languages, a fexpr is a function whose operands are passed to it without being evaluated. "
<Regenaxer>
In other Lisps it took a single arg
<Regenaxer>
(defexpr foo (L) ...
<beneroth>
I have a hard time to imagine a less specific different meaning of the word...
<Regenaxer>
Not 'defexp' but something special
<beneroth>
ah
<beneroth>
already restricted
<Regenaxer>
Pil can mix it
<beneroth>
plus the additional restrictions of compiling
<beneroth>
ok
<Regenaxer>
(de foo (A B . C)
<beneroth>
so where we see an elephant they only see a single elephant foot?
<Regenaxer>
hehe
<beneroth>
(like the koan about those blind people)
<Regenaxer>
yeah
<Regenaxer>
So in CL a fexpr can be compiled fine
<Regenaxer>
but not the *call*
<beneroth>
well no yes lol
<beneroth>
bwahaha
<Regenaxer>
(<p> NIL (foo) (bar))
<beneroth>
so they offload all context to the caller
<beneroth>
our marketing should work more with specific code examples (rosetta style) maybe
<Regenaxer>
yes, in CL <p> would be a macro
<beneroth>
though people who are conditioned not to see the fnord, don't want to see the fnord, don't see the fnord even when you hold it to their face...
<Regenaxer>
I gave up on marketing. Thinking is too different
<beneroth>
T
<beneroth>
well, the filtering of the seekers ending up here from their own initiative works pretty well, quality-wise I would say
<beneroth>
most humans don't like to think
<Regenaxer>
ok, back home. I can't type well while walking ;)
<Regenaxer>
yesss
<beneroth>
welcome home :)
<Regenaxer>
:)
<Regenaxer>
Was just walking in the field
<beneroth>
Wirth "Reliable and transparent programs are usually not in the interest of the designer. "
<beneroth>
that's why we cannot have nice things
<Regenaxer>
looks like
<beneroth>
"In the practical world of computing, it is rather uncommon that a program, once it performs correctly and satisfactorily, remains unchanged forever." also Nik Wirth
<beneroth>
that is the practical benefit of picolisp, I find.
<Regenaxer>
Though it changes permanently ;)
<beneroth>
well, according to the quote this is the symptom of a satisfactorily program :)
<Regenaxer>
ok :)
<beneroth>
don't over-read the uncommon
<beneroth>
I understand it: successful programs which are fun and efficient to use change all the time.
<beneroth>
matches my experience
<beneroth>
only stuff which might be vital but is scary and full of problems gets frozen and not changed anymore
<Regenaxer>
very true!
<beneroth>
bwahaha
<beneroth>
"Don't blame me for the fact that competent programming, as I view it as an intellectual possibility, will be too difficult for "the average programmer" — you must not fall into the trap of rejecting a surgical technique because it is beyond the capabilities of the barber in his shop around the corner."
<beneroth>
Dijkstra (1975)
<Regenaxer>
yeah
<beneroth>
and of course "We must not forget that it is not our business to make programs, it is our business to design classes of computations that will display a desired behaviour."
<beneroth>
Regenaxer, one last question for today (if you dont mind): how do you do searches for duplicates (e.g. duplicate contacts in ERP) ?
<Regenaxer>
No problem!
<beneroth>
compare everything with everything using pilog?
<Regenaxer>
It is not clear what duplicates are
<beneroth>
T, it is a fuzzy thing
<Regenaxer>
yes
<beneroth>
I have an existing database (migrated), so their might be duplicates in it (and also false positives of course)
<Regenaxer>
every db has lots of duplicates after a while
<Regenaxer>
cause people do not search properly before they create
<beneroth>
I would want to make a search or so, showing a user a list of potential duplicates
<beneroth>
e.g. checking for similar company or person names
<Regenaxer>
Exact matches are difficult
<beneroth>
run on button push, or cronjob
<Regenaxer>
there are usually slight diffs in spelling etc.
<beneroth>
yeah, often one doesn't want exact match, I guess. maybe using tolr/3
<Regenaxer>
I have reports sometimes
<beneroth>
yeah, kind of a report
<beneroth>
well, exactly a report
<Regenaxer>
Now for BTG we do a different approach
<beneroth>
we have +Sn on most of the fields we want to use for duplicate checking
<beneroth>
ok?
<Regenaxer>
they must import manually from the old system what they need
<Regenaxer>
So for a while it will be kind of clean
<beneroth>
I see. and your system checks on creation and shows a list of potential duplicates?
<Regenaxer>
nope
<beneroth>
I thought this might be a handy feature
<Regenaxer>
When they need a contact, they search the old db
<beneroth>
but the current case here might already have such data, and too much data to go through manually
<Regenaxer>
I think there is no automatic way
<Regenaxer>
Some *are* really similar but still different
<beneroth>
well iterating over all contact names, searching for each similar contact names, mark as potential duplicate if differential is less than X points in custom pointing/duplicate-checker
<Regenaxer>
Same address, same name, but different companies
<beneroth>
yeah, but I would say they can show up in such a "duplicate report"
<Regenaxer>
They have about hundred thousand entries now
<beneroth>
maybe with the possible to mark them as "definitely not a duplicate, false positive" so it doesn't show up again on the report
<Regenaxer>
I think it does not work
<Regenaxer>
Duplicates often have some misspelling
<Regenaxer>
so they were not found and re-created
<Regenaxer>
So a checker will also not find them
<beneroth>
tolr/3 ?
<Regenaxer>
Or a company moved or changed some base data
<Regenaxer>
no
<Regenaxer>
Complete chaos usually :)
<beneroth>
your data is customer/vendors (creditors/debitors), I guess, right? our current project here is a database of donors/media contacts/expert knowledge contacts
<beneroth>
ok
<Regenaxer>
yeah, so similar problem
<beneroth>
so usage is not so much "find me person X" but more like "find me all contacts which this and that properties, so I can write them a message for whatever"
<Regenaxer>
this is already there, in the normal search dialogs
iuno has joined #picolisp
<Regenaxer>
But if fundamentally wrong data were entered, they are not found there
<beneroth>
but you did such reports in the past you said? you don't do them anymore because they're usually useless?
<beneroth>
how did you made them?
<beneroth>
yeah
<beneroth>
of course
<Regenaxer>
They did not use them I think
<beneroth>
I like to find probable duplicates, not all variants of wrong data
<beneroth>
e.g. they imported/incorporated contact data from multiple sources
<beneroth>
not necessarily doing checking for existing matches when doing that import/entered those new records
<Regenaxer>
yeah
<Regenaxer>
So similarity checks are a start
<beneroth>
so I would just iterate over e.g. company names, trying to find similar company names for every entry
<Regenaxer>
But the same company might exist correctly several times
<Regenaxer>
in different locations
<Regenaxer>
or branches
<beneroth>
maybe this could be speed up by not actually doing that, just by comparing all leafs on a joined branch in the +Sn index tree?
<beneroth>
other ideas?
beneroth has quit [Remote host closed the connection]
<Regenaxer>
Or two entries with different names are actually the same because the name or address changed
beneroth has joined #picolisp
<Regenaxer>
It is a big problem
<Regenaxer>
I said:
<Regenaxer>
Or two entries with different names are actually the same because the name or address changed
<beneroth>
yeah, I don't care about those :) I know.
<beneroth>
happy enough when we can detect the others :)
<Regenaxer>
yeah
<beneroth>
e.g. some press contact getting into the database via two separate sources, nobody searched first
<Regenaxer>
In ERP it is both companies and persons
<beneroth>
same here
<Regenaxer>
companies are more difficult
<Regenaxer>
they may have varying namings
<beneroth>
T
<beneroth>
or multiple names
<Regenaxer>
eg xxx GmbH and other additions
<Regenaxer>
yeah
<Regenaxer>
Especially big ones
<Regenaxer>
So many Siemens entries
<beneroth>
what about my idea of looking at the actual +Sn tree? might be an idea? worthless? or unnecessary, just do dumb search?
<beneroth>
haha, yeah
<Regenaxer>
Yes, +Sn +Idx is a start
<Regenaxer>
Especially if the original data were not checked with soundex and tolerant
<Regenaxer>
so simple mistakes will show up
<beneroth>
exactly
<Regenaxer>
Another way is check usage
<Regenaxer>
If never referred to, it may be garbage
<beneroth>
T
<beneroth>
I'm asking more about the implementation ideas, not conceptual :) I'm aware of the issues on that level :)
<Regenaxer>
yes, I thought so
<Regenaxer>
I think users must be educated to be paranoid
<beneroth>
yeah.. only solution, but not practical
<Regenaxer>
or only allow to create new with special permission
<beneroth>
use case here is more about data cleaning of existing corpus
<Regenaxer>
yeah
<Regenaxer>
I mean for the future ;)
<beneroth>
I just did a (scan (tree 'company '+Contact))
<beneroth>
I get many entries with exact same key
<beneroth>
(+Sn +IdxFold +String)
<Regenaxer>
T
<beneroth>
so instead of doing real search, maybe just scanning through the index and compare the contacts with same key would be good enough (and much smaller search space) ?
<Regenaxer>
and also addresses?
<Regenaxer>
or at least zip range
<beneroth>
yeah, something like that I have in mind
<Regenaxer>
ok, and then let the users decide?
<Regenaxer>
report or dialog
<beneroth>
yes
<beneroth>
final decision always manual
<Regenaxer>
Probably the only way, yes
<beneroth>
I've worked enough with data ;-)
<beneroth>
I just try to come up with meaningful tooling
<Regenaxer>
yep
<beneroth>
users will only do minimal data quality control if it's easy, low barrier
karswell has joined #picolisp
freemint has joined #picolisp
<beneroth>
have a nice evening, Regenaxer :)
<beneroth>
thanks for the talking, as always :)
beneroth has quit [Quit: Leaving]
iuno has quit [Quit: Leaving]
<Regenaxer>
oops, was on phone :)
ubLIX has joined #picolisp
xkapastel has quit [Quit: Connection closed for inactivity]
xkapastel has joined #picolisp
ubLIX has quit [Quit: ubLIX]
stultulo has joined #picolisp
f8l has quit [Ping timeout: 272 seconds]
stultulo is now known as f8l
stultulo has joined #picolisp
f8l has quit [Ping timeout: 258 seconds]
stultulo is now known as f8l
stultulo has joined #picolisp
f8l has quit [Ping timeout: 246 seconds]
stultulo is now known as f8l
orivej has quit [Ping timeout: 276 seconds]
stultulo has joined #picolisp
f8l has quit [Ping timeout: 258 seconds]
stultulo is now known as f8l
xkapastel has quit [Quit: Connection closed for inactivity]