00:41 <cquinn> Hey all. Now that we have a nice cli library, I am thinking about getting back into working on the package manager we were discussing a while back. Having a real tool to flesh out the missing bits of the cli, plus adding some functionality to what we have with stable.

00:41 <cquinn> I have sketched out a rough design in a google doc here: https://docs.google.com/document/d/1c7puEQLks3X1wpabuXxox8Qi1HUhfSwhobUvmVE56Rw/edit#

00:50 <jemc> cquinn: speaking broadly, and at first glance, what you've laid out seems to make sense as a good next evolution for pony-stable

00:50 <cquinn> oh good, thanks for looking

00:50 <jemc> that last point in your document is definitely a sticky one, and I think it requires some more thought

00:51 <jemc> > Since Stable (and Corral) include all the package roots in a flat list in PONYPATH, it seems that the package name space is thus global for a project

00:52 <jemc> I feel like this is a root problem in the strategy of only setting `PONYPATH` to point to a package bundle

00:52 <jemc> in that I can't depend on two bundles that have the same package name inside them

00:52 <cquinn> yeah, I deduced that as I worked out all of the models of the different dep managers

00:53 <jemc> I think there's two overall paths that might make sense to mitigate that (without changing anything in ponyc to accomodate)

00:53 <cquinn> one option I thought of is the symlink farm to select the desired subset of packages

00:54 <cquinn> but the bundles might have trouble with that themselves

00:55 <jemc> yeah, I was about to say that both options boil down to the idea that the dep manager needs to "think" in terms of packages, not bundles

00:55 <cquinn> yes

00:55 <jemc> in that I need to specify which packages from which bundles I want to use, not just which bundles

00:56 <cquinn> exactly. that's why I added a package list to the project file. for that package->bundle mapping

00:56 <jemc> and possibly set up an aliasing system, so I have a way to include packages that have the same name?

00:56 <cquinn> interesting, yes. Go tools do that for forked or cached packages

00:57 <jemc> hm, maybe the "way out" is that the pony source needs to `use "bundlename/packagename"` instead of just `use "packagename"`

00:57 <cquinn> but I wonder if in Pony we might run into trouble if we start aliasing packages, how will packages within a bundle fine each other?

00:57 <jemc> within a bundle, you'd always want to use relative paths, I think

00:58 <cquinn> oh, that might work if we kept bundlename simple and not contain full location information

00:58 <jemc> hm, but we'd still have problems if I depend on two different bundles, who each depend on a bundle that they gave the same `bundlename`

00:59 <jemc> that is, if I depend on bundles A and B, and they each have a bundle that they call `c`, but they are different bundles, they all end up in the same global PONYPATH namespace...

00:59 <cquinn> we could allow aliasing of the bundlename part though

00:59 <cquinn> ah, ponyc would have to know about that

01:00 <cquinn> I think if we want to be completely flexible and handle all cases, we will need to make adjustments to ponyc

01:00 <jemc> right, I'm not sure we can totally avoid these problems without extra support from ponyc - which I was trying to avoid, but as long as we keep it very general and not tied up with the specifics of `stable` / `corral`, I think it would be okay

01:02 <cquinn> exactly. I'll think about that some more in my noodlings

01:05 <jemc> I wonder if PONYPATH could be made to accept a special path format such as `@/.corral/whatever`, where the `@/subdir` implies: "starting from the directory of the package being compiled, go up one dir at a time until you find the given subdir"

01:06 <jemc> then we give each bundle its own `.corral` subdir, and ponyc will use the right `.corral` for that particular bundle when compiling source files from a package in that bundle

01:07 <jemc> I think that would solve our problems of each bundle needing to have its own "namespace" for fetched packages, that doesn't pollute other bundles

01:07 <cquinn> I think I follow that

01:07 <cquinn> that could still be laid out with symlinks I think

01:08 <jemc> sorry, are you saying symlinks are compatible with that idea, or that symlinks make it so you don't need that idea?

01:08 <cquinn> I think they are compatible

01:08 <jemc> right, okay

01:09 <jemc> well, roll that around the old idea-mill, and we can discuss more later

01:09 <cquinn> ok, will do

01:09 <jemc> for now I've got carpentry to do :P

01:10 <jemc> (trying to build a bed frame for the new bed we bought before my inlaws arrive for a visit at the end of the week :yey:)

01:16 <cquinn> ha. good luck!

07:00 <cquinn> @jemc I think I just derived a solution that I think is the same as what you where saying above with the @ notation. Now I get it :)

16:54 <adam_> Performance question. I modified examples/httpget to connect to a server that delivers a stream data. Even if I don't do anything at all with the data, CPU usage jumps around in the 4% to 10%. What is going on here?

16:56 <adam_> Actual amount of data received is small, couple of hundred bytes a second

17:09 <adam_> Sorry, nothing to do with http. Running the example from https://stdlib.ponylang.org/time-Timer/#timer also uses the same amount of CPU

17:12 <adam_> Running same timer on a much less powerful (and single-core) cloud instance uses only 2-3% CPU.

17:12 <adam_> But in any-case, I would expect this to use a non-measurable amount of processing

17:17 <adam_> It used over 8 seconds of time to count to 42!

17:31 <adam_> One thread is sitting in epoll_wait, from ponyint_asio_backend_dispatch, presumably the timer. The others in nanosleep from ponyint_cpu_core_pause.

17:32 <jemc> adam_: you can reduce CPU usage for timers by increasing the "slop" of the timer

17:41 <jemc> https://github.com/ponylang/ponyc/blob/c2b44029687b7054b6d04636ac36fea2d0a273e4/packages/time/timers.pony#L24-L36

17:42 <jemc> adam_: also, in the past we have had some issues with "waiting" using more CPU than necessary on some platforms - what platform are you running on?

17:43 <jemc> it looks like there's an open issue on OSX - https://github.com/ponylang/ponyc/issues/1787

18:31 <adam_> jemc - ubuntu 16.04

18:33 <adam_> jemc - I could change slop for timer to fix this instance, but I also see it in the httpget example, I'd have to dig through the library code for that to ensloppen timers everywhere too

19:58 <jemc> in the issue ticket I linked to there was talk of a global runtime-level tweak option for this kind of thing

21:19 <adam_> jemc - I see that, but agree with SeanTAllen about it potentially appearing as black-magic, certainly it is to me. P

21:20 <adam_> Pony needs a better default here. I know nothing about scheduling, so I'm loath to modify the C code myself just to "fix" it for myself

21:21 <jemc> I think it's hard to know what the right default is - if we made it significantly "sloppier", we'd probably have people complaining about poor timer precision

21:26 <adam_> From the linked discussion, it seems the current version relied on nanosleep(0) taking longer than an instant. It seems this doesn't apply on OS X, or Ubuntu 16.04, so though it worked before, it was accidental. The sensible default would be however many micro-seconds it takes on those systems that don't display such high CPU usage.

21:27 <adam_> If people want nanosecond timers, then that could be a dial to twiddle, but I suspect that most don't, but will notice high CPU

21:28 <jemc> seems like a reasonable PR if you wanted to file it, though maybe check with SeanTAllen about it, because he's more plugged into this issue than I am

22:51 <SeanTAllen> I'm not in favor of that PR. I'd need convincing.

22:52 <SeanTAllen> "That PR" being a change to default slop.

22:59 <SeanTAllen> I'm not against it either. I am dubious.

23:03 <SeanTAllen> I'd like to have a better understanding of the reasoning.