02:36 <neynah> Happy holidays everyone! :)

07:51 <asheesh> Happy holidays to everyone, too!

20:07 <isd> Have to concur with maurer on being somewhat alergic to the vfs/syscall intercept thing

20:09 <isd> I feel like you could do basically everything you need without it. cpp's -M gives you the full dependency list, so if you just write a parser for a very small subset of make, you can lean on that. as for the merging tmp/ and src/... what's wrong with just passing an extra -I ?

20:11 <isd> Would also be a lot friendlier to using system packages.

20:12 <kentonv> isd: sure, specifically for C/C++ you can use the existing features of the compiler. But many tools haven't grown all the necessary flags.

20:12 <kentonv> and writing code to handle each one is kind of annoying

20:12 <isd> Is ekam actually usable/interesting for anything but C/C++

20:13 <isd> or do you mean stuff like yacc/capnpc?

20:13 <kentonv> code generators are one example. Neither protoc nor capnpc have a flag like -M.

20:14 <kentonv> also... what does cpp -M do when the header doesn't exist yet?

20:15 <kentonv> Ekam is able to understand when a C++ file includes a generated header that hasn't been generated yet, and re-run the compile later when that generated header shows up

20:15 <isd> Yeah, that would need to be handled separately.

20:16 <kentonv> yeah cpp -M doesn't even produce output if parsing failed (e.g. because a header wasn't found)

20:16 <isd> I guess I'm at the point where I feel like I'd rather have an 80% solution that resort to weird hacks like this. But then we're getting into the realm of philosophy.

20:17 <kentonv> sure, it's perfectly valid to argue that explicit build systems are better. The goal of Ekam was to try to create a system where you don't have to be explicit, and I think the syscall interception is necessary for that goal.

20:19 <isd> (I'm also not a big fan of vendoring)

20:21 <kentonv> I do often have to edit Cap'n Proto and Sandstorm together, which was a pretty big pain before we were able to build them together...

20:21 <TimMc> isd: What do you dislike about vendoring?

20:24 <isd> Also, sandstorm still has a bunch of custom build logic. I haven't looked closely at capnp's build system, but it seems like at best you're 1 for 2... I can't say that strikes me as wildly successful.

20:25 <kentonv> I don't understand what you mean by that.

20:27 <isd> I mean, it seems like the goal of this project is to have stuff "just work." And we know of two projects that are using it. And one of them still has a bunch of custom stuff in the build system. Not a lot of data, but this suggests that ekam isn't actually succeeding at the "just work" bit in practice, if half the projects that use it have to do custom stuff.

20:28 <isd> Re: vendoring... it has it's place. More what bugs me is having using system packages be difficult.

20:29 <isd> Granted, sandstorm does a bunch of custom stuff that would be really hard for any generic tool to pull off.

20:31 <kentonv> I mean, the goal of Ekam is not so much to be 100% automated as it is to remove the busy-work involved in adding each new source file and maintaining dependency relationships between them

20:31 <kentonv> I like that when I add a new .C++ file I don't have to go edit anything for it to start building.

20:32 <kentonv> I like that my binaries link in exactly the set of .o files they actually use rather than pulling in everything in all the libraries they declare dependencies on, without any need for me to declare this usage.

20:33 <isd> --as-needed?

20:33 <kentonv> no, look

20:33 <kentonv> that only helps if I first declare that I depend on the whole world

20:34 <kentonv> which isn't really a workable strategy

20:34 <isd> You mean having to declare the deps, not the just the fact that no extra symbols end up the binary. fair enough.

20:34 <kentonv> yes

20:35 <kentonv> I can just #include a header form a library I wasn't using before, start using it, and it works

20:37 <kentonv> currently Sandstorm delegates to Ekam purely for the C++ part of the build. Most of Sandstorm's code is Meteor-based Javascript. Meteor turns out to have a build system which basically accomplishes the same goals, for the narrow case of Meteor apps. So I didn't attempt to make Ekam handle that.

20:37 <isd> Yeah, just comes down the philosophy thing. Strikes me as a bad trade-off vs. having to add an #include *and* a -l<lib> in another file. I guess I'm just back to arguing that I'd rather have something explicit. But that's not really resolvable without better defined requirements.

20:38 <isd> It's also just hard to pull this off without digging into the language's tooling. #include is a nightmare of a "module system."

20:39 <isd> The plan9 c compilers have a #pragma lib foo that figures out the linking thing for you. And more modern languages generally are actually built with real build systems in mind.

20:39 <isd> If you want something that works with existing tools and does this there are going to be some hacks.

20:41 <isd> Pragmatically I'm more bothered by the difficulty of *not* vendoring. Less from a developer's standpoint, and more from the standpoint of wanting to be able to have stuff packaged for *nix in a way that's idiomatic with the local system

20:45 <isd> Like, I consider packageability a feature of the thing I'm shipping. And I want to use tools that facilitate that.

20:45 <isd> But, I'm about to head out the door. Later.

21:01 <kentonv> isd: I'm not totally sure what you mean about it being difficult to use system packages. You can setenv LIBS="whatever" and Ekam will add it at link time. Of course, what would be better is if Ekam had an index of symbols from system libraries and could link them automatically but I never got around to implementing that.

21:01 <kentonv> that's not the idea's fault -- it's just one of many ways that Ekam is incomplete

21:14 <isd> So could you theoretically rip out the vendoring of deps for existing projects this way? Maybe I was just under the impression that using system packages is harder than it really is.

21:15 <isd> And yeah, that's certainly not inherent to the approach.

22:21 <isd> In other news, there's a new IRC Idler release: https://groups.google.com/d/msg/sandstorm-dev/gCYNrdkmXVI/TZm2gT5jDwAJ

22:36 <kentonv> isd: yeah we could require all the dependencies to be system-installed and then just link them. But then there are problems like people want to build on distros that haven't updated that package in a while and it's too old, and they don't want to figure out how to update it from source, etc.

22:36 <kentonv> so, the usual arguments around vendoring

22:41 <isd> Yeah. This is also another thing that is "easy" if the language/it's tooling is designed around it -- it doesn't have to be hard to get a thing to build against libs in different locations.