devyn changed the topic of #elliottcable to: a super duper cult :)
<devyn>
I think we should just start trying to do real™ things in Paws and if we run into problems that require design changes to solve, we make those changes
<devyn>
we don't even necessarily need to replace cPaws first, since there's still quite a bit of abstraction you can do without a preprocessor
<devyn>
as a side-note, I had an interesting optimization idea last night
<devyn>
reactors have a local LRU cache of objects' members, and then just version the members with an atomic integer
<devyn>
not individual members but the Members data structure
<devyn>
that would significantly reduce contention on lookup-Things, which are very common
<ELLIOTTCABLE>
fucking fuck
<ELLIOTTCABLE>
"lookup-Things", and, LRU?
<devyn>
least recently used cache. it's a fixed size cache that throws out the least recently used things first
<devyn>
the most common kind of cache.
<ELLIOTTCABLE>
yeah, I understand; what about the version part?
<devyn>
basically whenever I *change* an object's members, increment the 'version'
<devyn>
and that version will be an atomic integer so I can just check whether my cached version is up to date
<devyn>
or whether I need to lock and fetch again
<devyn>
it only affects my impl, anyway :p
<devyn>
I also need a better queue strategy, for sure
<devyn>
but this will allow, for example, much less contention on say 'infrastructure' which is widely used
<ELLIOTTCABLE>
ahhhhhhh
<devyn>
thinking of ways to make parallel Paws actually work™ instead of just slowing it down massively
<ELLIOTTCABLE>
different versions of the same object-id object floating around in your system are going to be a necessity anyway
<devyn>
how so?
<ELLIOTTCABLE>
Spheres of Influence, as they used to be called. basically, one of the backing mechanisms for time-travelling error handling.
<ELLIOTTCABLE>
the partner to insanity.
<devyn>
ah, yeah, this isn't going to make that any easier
<ELLIOTTCABLE>
basically, there will be areas of a Paws system that "see" other areas at a past state, until that area applies
<ELLIOTTCABLE>
(as in transaction-applies)
<devyn>
this is a very low level detail that can't really be repurposed for that
<ELLIOTTCABLE>
nonono not saying your optimization will
<ELLIOTTCABLE>
but it seems to trigger some similarities in my head. maybe one cooptimization that handles both situations? idk.
<devyn>
object IDs as you do them aren't even a thing in Paws.rs at the moment; identity is just pointer-based
<devyn>
anyway I'll think about how to do that when we get to it
<ELLIOTTCABLE>
oh, they are, they're just not spec'd
<ELLIOTTCABLE>
part of Units, since they can share objects
<ELLIOTTCABLE>
omfg
<ELLIOTTCABLE>
does anybody here have experience with PEG?
<joelteon>
increases the value of 'someTime' by 3 weeks
<alexgordon>
joelteon: interesting
<alexgordon>
all white noise to me
<joelteon>
you're god damn right
<alexgordon>
joelteon: what I realized when I was designing furrow's datetime (lol I have to bring up furrow in every conversation, it's a rule)
<purr>
lol
<alexgordon>
is that you really only need three main types, and fuck the rest
<alexgordon>
1. Date (the number of nanoseconds since Jan 1st 1970, with no leap seconds included)
<alexgordon>
2. TimeDelta (in nanoseconds)
<alexgordon>
3. YMD (the year, the month, and the day)
<alexgordon>
representationally, that gets you pretty far
<joelteon>
yeah
<joelteon>
thyme is based on exactly one type: UTCView
<joelteon>
which is the number of days since epoch, and the time since midnight for that day
<alexgordon>
interesting
<joelteon>
i think it is anyway
<alexgordon>
how does it define "day"?
<joelteon>
int
<alexgordon>
but I mean abstractly
<joelteon>
well i think so
<joelteon>
hang on a sec
<alexgordon>
as 86400 seconds?
<joelteon>
newtype Day = ModifiedJulianDay { toModifiedJulianDay :: Int }
<joelteon>
no a Day is a 24(ish) hour period
<joelteon>
DiffTime is for time-of-day
<alexgordon>
joelteon: what's an hour then? :P
<joelteon>
3600 seconds?
<alexgordon>
so a day is 86400(ish) seconds. what's the ish?
<alexgordon>
leap seconds?
<joelteon>
yes
<joelteon>
Day knows nothing about leap seconds
<joelteon>
DiffTime is the bit that matters for leap seconds
<joelteon>
Days change halfway between sunset and sunrise
<joelteon>
you know, give or take
<alexgordon>
generally I think that leap seconds are a presentation issue only
<alexgordon>
because
<alexgordon>
leap seconds can change
<joelteon>
well, in a new york minute, everything can change
<alexgordon>
:P
oldskirt has joined #elliottcable
<alexgordon>
sooo
<alexgordon>
hi oldskirt whoever that is
<oldskirt>
I'M LORD OF THE OLD SKIRTS
<oldskirt>
of course
<oldskirt>
what else
<alexgordon>
joelteon: sooo I'm working on memory management for furrow
<alexgordon>
trying to solve it
<joelteon>
sooooo
<joelteon>
is it hard
<alexgordon>
oldskirt: should've called yourself oldtrousers, then you could be lord of the flies
<oldskirt>
hehe
<alexgordon>
joelteon: yes
<alexgordon>
but not as hard as I'd expected
<alexgordon>
so there are three kinds of types: copy-only, ref-only, and 'copy or ref'
<alexgordon>
copy only is something like Int, whereby there's no point taking pointers to it because it'll just be slower
<alexgordon>
but also small structs, small tuples, etc
<alexgordon>
the ref only types are really big types that you don't want to be implicitly copying over the place
<alexgordon>
like dictionaries
<alexgordon>
BSTs, etc
<alexgordon>
lastly there's copy or ref types, which can be used in either configuration. mainly the larger structs and tuples
<alexgordon>
as long as they don't contain a ref-only type
<alexgordon>
ANYWYA
<alexgordon>
*ANYWAY
<alexgordon>
the big impact this has is on variables and function calls
<alexgordon>
a "use" is when you either set a variable, or pass something into a function call
<alexgordon>
and on every use you need to consider whether it should be by-ref, by-copy or by-static-rename
<alexgordon>
which is a factor of 1) what kind of memory management the type supports, 2) whether the variable is used again, 3) whether the variable will be modified by the new owner
<alexgordon>
this is slightly complicated by the fact that types can have more lenient subtypes that convert into them
<alexgordon>
for example...
<alexgordon>
if you do [] then you get a value of type EmptyList, which is a unit type
<alexgordon>
or if you do 'x = [a, b, c]' that is a fixed-size list type (i.e. T[3])
<alexgordon>
T[3] may well support copying
<alexgordon>
T[3] coerces into [T], so any function that takes a [T] also needs to support T[3]
<Cheery>
alexgordon: are you familiar with that stuff?
<Cheery>
I wonder if I'd add generic functions into my language
<Cheery>
fixed arity generic functions
<Cheery>
read the julia paper and realised there's point in that.
<Cheery>
skipping the problem with single dispatch and methods isn't simple.
<joelteon>
perl is groce
<alexgordon>
Cheery: which stuff?
<Cheery>
alexgordon: type systems, type inference, that kind of stuff
<joelteon>
this mother fucker wrote a deployment system in perl
<joelteon>
er uh
<joelteon>
partially in perl
<joelteon>
brbeeeee
<joelteon>
ZNC is dying
joelteon has quit [Remote host closed the connection]
joelteon has joined #elliottcable
<joelteon>
ok i'm back
<devyn>
ELLIOTTCABLE: no, I mean a parser that can change at runtime
<devyn>
ELLIOTTCABLE: unless the goal is just to transform to cPaws *shrug*
<purr>
¯\(º_o)/¯
<devyn>
still, I don't know whether that's enough
<devyn>
ELLIOTTCABLE: oh, yeah, so, it turns out that that caching thing I talked about wouldn't have been a worthwhile optimization. I measured where the contention is and it is (thankfully) not really on the objects, but mostly on the queue
<alexgordon>
Cheery: more or less
<alexgordon>
devyn!
<devyn>
ELLIOTTCABLE: with 8 reactors on branch-a-lot, object contention wastes like 0.01s, but queue contention is wasting like 0.8s lol
<purr>
lol
<devyn>
ELLIOTTCABLE: so, to solve that, I'm going to ditch the single queue approach and just send stuff to each reactor in a round robin
<devyn>
alexgordon!
<alexgordon>
wow this milkshake is awesome
<devyn>
does it bring all the boys to your yard
<alexgordon>
*girls
<devyn>
*boys ;)
<alexgordon>
nah only girls get to enjoy my magnificent milkshakes
<devyn>
lol
<purr>
lol
<alexgordon>
devyn: oh I figured out how to store stack allocated dictionaries
<devyn>
alexgordon: what advantage does that have over {{h1, k1, v1}, {h2, k2, v2}, {h3, k3, v3}}
<alexgordon>
devyn: only have to fetch the list of hashes
<alexgordon>
then iterate over them until you find a match
<alexgordon>
then find the corresponding key and value
<alexgordon>
if you do {h1, k1, v1} ... and v1 is big-ish then it's less L1-cache friendly
<devyn>
well I mean it's better cache-wise, ah, yeah
<devyn>
I figured
<devyn>
ok
<alexgordon>
devyn: it's amazing how inefficient software is
<alexgordon>
all those dynamically allocated dictionaries
<alexgordon>
don't need to exist
<alexgordon>
could just be arrays!
<alexgordon>
stack allocated arrays
<devyn>
not all of them :p
<alexgordon>
80% of them :P
<devyn>
if you actually need to mutate them, then no
<alexgordon>
why not?
<alexgordon>
you can still mutate it
<alexgordon>
of course if you need to add a lot of values to something then yeah, you can't stack allocate it
<alexgordon>
but like, most dictionaries are maybe 5 pairs
<alexgordon>
so much waste
<devyn>
if you want to mutate it you probably want to do some preallocation since otherwise you'd need two memcpy()s and then zeroing out the newly created gaps…
<alexgordon>
devyn: deleting keys isn't really a common operation
<alexgordon>
but you can just zero out the row and ignore it
<devyn>
forget the zeroing part, nvm, but you still need to memcpy() twice to add keys
<alexgordon>
to add you just add onto the end if you allocated enough space
<alexgordon>
this is the kind of thing that a JIT would be good at though
<devyn>
no you can't, since the it's {hashes, keys, values}
<alexgordon>
devyn: zero out the hash then
<devyn>
you have to move the values one space over, then the keys+values one space over
<devyn>
huh?
<alexgordon>
say you have
<alexgordon>
{{123, 456, 789}, {"a", "b", "c"}, {x, y, z}}
<alexgordon>
if you want to delete "b" then you just do
<alexgordon>
devyn: you can't expand because it's stack allocated
<devyn>
sure you can
<alexgordon>
but you can preallocate sufficient space
<devyn>
it just requires moving the entire stack
<devyn>
:p
<alexgordon>
devyn: well yeah...
<alexgordon>
devyn: at that point you're better off doing a dynamic allocation
<devyn>
yeah, that's my point
<alexgordon>
devyn: but you can use PGO or runtime optimization to determine that a particular dict never uses more than 4 and then use that as an educated guess for preallocation
<alexgordon>
my point is that languages really aren't squeezing out the performance that they could be, especially with regard to high level data structures
<devyn>
it's hard to tell in a dynamic language whether you should be using a stack-allocated or dynamically allocated structure
<alexgordon>
well this would be a static language
<alexgordon>
if you know that a dict only ever uses a certain set of keys you can even turn it into a struct
<devyn>
but do note that Rust is moving toward DST (dynamically sized types) which will allow us to have stuff on the stack more easily
<alexgordon>
devyn: yeah my problem with that is that it's then all in the user's face
<alexgordon>
ever tried reading boost's code?
<alexgordon>
I'd rather hide this stuff away in a compiler optimization stage
<devyn>
Rust is pretty sensible about it
<alexgordon>
rust is going down the path of "give the programmer all the tools they need to make everything"
<alexgordon>
which is _fine_
<devyn>
bare types are always stack-allocated/in-struct, and if you want dyn-allocations there's Box<>
<alexgordon>
but it tends to result in mountains of complexity
<alexgordon>
which is the antithesis of C
<alexgordon>
devyn: right but to get what I'm proposing would require ridiculous amounts of complexity
<alexgordon>
devyn: so they'll be forced to balance performance against implementation complexity
<alexgordon>
whereas if it's an optimization stage the sky's the limit
<devyn>
it's so easy to overdo that though… just look at Haskell, wherein everything-is-an-optimization-stage, essentially
<alexgordon>
yeah but haskell is starting from a position of weakness
<alexgordon>
haskell's semantics don't reflect machine semantics AT ALL
<alexgordon>
so they compile the whole thing to Core then they do super-PhD optimizations, and somehow end up with LLVM IR
<alexgordon>
whereas Rust (and Furrow's) semantics are basically an extension of C's, so it follows the machine quite well
<devyn>
but when you're abstracting over things like this, you're deviating from the reality of the machine (obviously)
<alexgordon>
mmm I don't see it like that
<devyn>
so there's always the potential for bad optimizations
<alexgordon>
it's just getting the optimizer to choose storage representation and location
<devyn>
yes, and if it chooses stack when it really should have chosen dynamic, that's expensive
<alexgordon>
well be conservative then :P
<devyn>
:p
<alexgordon>
the key with optimization is to only ever optimize things that you know better than the programmer does
<devyn>
just saying, any abstraction like that is deviating from reality
<devyn>
not that that's a bad thing
<devyn>
it's just something to consider
<alexgordon>
if I can see that a dictionary is used locally, and it only has keys X, Y and Z, and I know that the value type is N bytes big, then I can be confident in storing it on the stack
<alexgordon>
most of the dictionaries I use in python are like that
<alexgordon>
if the compiler _can't_ be sure of that, then by all means store it on the heap
<devyn>
fair enough
<alexgordon>
same goes for lists, and strings
<alexgordon>
I'd love to see a language solve memory management for strings
<devyn>
idk, I really love the control Rust gives me over what's happening though… it has very high level abstractions but it's still fairly obvious to see what they're going to do
<alexgordon>
devyn: it gives you control, sure
<alexgordon>
but sometimes it's OK to relinquish some of that control
<alexgordon>
there's a continuum between "as fast as C" and "not as slow as java"
<devyn>
it is, yeah, though it can lead to random frustration when something gets "optimized" the wrong way :p
<devyn>
mm, yeah
<devyn>
then that's a worthwhile goal
<alexgordon>
don't optimize the wrong way then :P
<devyn>
it can be hard not to :p
<alexgordon>
eeeeeh
<alexgordon>
start with obvious semantics
<alexgordon>
make obvious optimizations
<alexgordon>
can't go wrong
<alexgordon>
devyn: really, storage optimizations are the most glaring hole in pretty much every language
<alexgordon>
things that make C++ programs slow: 1. allocation, 2. copying
<alexgordon>
what you _can_ do is what rust does which is make both explicit
<alexgordon>
want heap allocation? you've got to ask for it. want a big copy of an array? ditto.
<devyn>
and yet Rust still manages to be way simpler than C++ :p
<devyn>
but yeah, I guess you can try to do some storage optimization automatically instead
eligrey has quit [Read error: Connection reset by peer]
eligrey has joined #elliottcable
<alexgordon>
devyn: does it? I'm not sure...
<joelteon>
so guys, i found out something cool
<alexgordon>
Rust is more complex than C++ IMO
<joelteon>
godaddy uses puppet internally right
<joelteon>
makes sense
<devyn>
strongly disagree, if only because it's way more consistent and the compiler is very helpful
<joelteon>
we have no policy for managing modules; if you want to add a new module, put a submodule in your project
<joelteon>
1. the submodule URL must point to GD's github enterprise instance
<joelteon>
2. the submodule URL must be SSH not HTTPS
<joelteon>
if 1. or 2. isn't fulfilled no puppet masters will replicate until the offending module is removed
alexgord_ has quit [Quit: Computer has gone to sleep.]
Sgeo has joined #elliottcable
sharkbot has quit [Remote host closed the connection]