kentonv changed the topic of #sandstorm to: Welcome to #sandstorm: home of all things sandstorm.io. Say hi! | Have a question but no one is here? Try asking in the discussion group: https://groups.google.com/group/sandstorm-dev
xet7 has joined #sandstorm
<abliss> so, random thought, but would anyone be interested in co-hacking by videochat?
<isd> Possibly.
<abliss> on a lark i'm streaming (and recording) my hack session for the next ~hour at https://meet.google.com/hau-adhc-xmp?authuser=1 , join if you feel like it :)
<JacobWeisz[m]> What are you hacking, abliss?
<abliss> tonight i'm trying to connect a remote python debugger into the grain so i can figure out what's going on with synapse (the matrix backend) inside its grain
nicoo has quit [Ping timeout: 240 seconds]
nicoo has joined #sandstorm
frigginglorious has quit [Ping timeout: 250 seconds]
indolering[m] has joined #sandstorm
<indolering[m]> Hola! I had an idea for a polygot Capn Proto implementation but (as I've never done low-level programming) I need some help hashing out the best architecture for it.
<indolering[m]> kentonv: you around?
<isd> indolering: channel is logged, shoot & maybe he'll see it.
<isd> (I also may have some thoughts)
<indolering[m]> Well, the first question I have is about using Capn Proto from garbage collected languages. The website states that dynamic languages should use a FFI but the JavaScript implementation has a big warning about being slow because it can't inline functions.
<isd> I think that depends highly on the particulars of the language.
<isd> fwiw, there is another js implementation that's not an FFI binding.
<isd> The trouble you run into with some languages is there's not necessarily a super efficient way to just access a byte buffer. But I think javascript's ArrayBuffers fit the bill
<isd> Also gc is distinct from "dynamic languages" (which people usually take to mean no types). There are a bunch of Gc'd implementations (including one I wrote)
<isd> But again, the relevant thing is "does this language have the low-level primitives we need?"
<indolering[m]> Derp!
<indolering[m]> Right. So, I've seen that there is a Python port that uses SWIG and there is some discussion on HN about refactoring CP to make it easier to use the C++ RPC backend.
<indolering[m]> I've never messed with SWIG, is there a reason Capn' Proto doesn't lean on it? Is it literally just that it doesn't produce the most idiomatic code?
<isd> I've never really used swig either. I remember seeing a comment of kenton's saying basically (and this matches my own intuition) yes, the FFI results in kinda awkward APIs.
<indolering[m]> FWIW, the other RPC libraries don't produce the most idiomatic APIs and aren't of the highest quality either.
<indolering[m]> the ports of*
<isd> I think (and again, I think kenton said something like this) RPC is probably a more fruitful thing to do this with than serialization, because the ratio of internal complexity to external API surface is much larger
<isd> You could build a design where you basically feed it messages you've constructed with a native serialization layer, and it just manages the internal state machine for the protocol.
<isd> There's still some fussing around with FFI awkwardness, but it buys you more than it would in the serialization case.
<isd> You can always wrap the FFI generated code in something nicer. But with the serialization layer you're getting a lot less actual functionality for that effort.
<isd> I've got to sign off for the night though. Best of luck.
<indolering[m]> Night.
_whitelogger has joined #sandstorm
frigginglorious has joined #sandstorm
frigginglorious has quit [Client Quit]
ecloud_wfh has quit [Ping timeout: 260 seconds]
xet7 has quit [Quit: Leaving]
xet7 has joined #sandstorm
ecloud has joined #sandstorm
robbt has joined #sandstorm
tian has quit [*.net *.split]
<kentonv> Ugh, when Meteor's incremental build can't parse my HTML template file, it doesn't seem to report any error on the terminal anymore... but the page fails to load
<xet7> Wekan works better in mobile browsers now that I deleted bundle/programs/web.browser.legacy https://github.com/wekan/wekan/commit/1235363465b824d26129d4aa74a4445f362c1a73
<xet7> before this Wekan did not load at all at Firefox for Android
<kentonv> no this is my own errors, like having mismatched tags or {{#with}}...{{/if}} or whatever. But for some reason Meteor isn't reporting the error on the console, it just says "Client modified -- refreshing (x16)" like any other change.
robbt has quit [Quit: Connection closed for inactivity]
<indolering[m]> Ian Denhardt: @ken
<indolering[m]> kentonv: My last set of questions revolves around WASI.
<indolering[m]> AFAICT WASI should handle primitive type conversion and Wasmer makes it very easy to import WASM into any major language.
<indolering[m]> Has anyone dug into that route?
<indolering[m]> Sorry for all the ELI5, I promise I'll get to adding value in a minute.
<kentonv> sorry, I don't understand what you're getting at
<indolering[m]> I was talking to ian earlier about building wrappers for Capn Proto.
<indolering[m]> I built a polygot DNS record validation library for EasyDNS a few years back using Haxe.
<indolering[m]> I've been wanting to build a more robust workflow for polygot libraries and I thought Capn Proto might be a good choice.
<indolering[m]> However, I don't have any experience with C++ and low level programming, so I've got a few questions about integration.
<kentonv> ok
<indolering[m]> Have you looked into using WASM for Capn Proto?
<kentonv> like compiling Cap'n Proto to Wasm? I haven't tried it but I'd like to at some point.
<indolering[m]> Have you heard of Haxe?
<kentonv> hmm, I don't think so
<indolering[m]> Haxe is a language which transpiles into JavaScript, C++, C#, Java, JVM, Python, Lua, PHP, and Flash.
<indolering[m]> Each additional supported language does not come for free, you have to put some amount of work into them.
<indolering[m]> (Haxe provides conveniences for interacting with the target language, including embedding the target language in the code and externs. But still not zero effort.)
<indolering[m]> What does come for free is unit testing. You write one set of unit tests for your Haxe code base and it is tested using the native language of each target code base.
<indolering[m]> From the maintainer
<indolering[m]> 's perspective, you don't really need to know the target language that well.
<indolering[m]> Combine that with Bazel and you have a fairly robust system for maintaining and distributing polygot libraries.
<kentonv> ok... so are you trying to suggest that Cap'n Proto should be rewritten in Haxe, or...?
<indolering[m]> No, I'm aggressively uninterested in writing Cap'n Proto in Haxe.
<indolering[m]> But I would be interested in doing the front-end work if others are willing to help me out with connecting it to the backend.
<indolering[m]> There are some TypeScript/JavaScript/C#/PHP -> Haxe converters ... if anyone wants to build a full-blown implementation in Haxe.
<simpson> indolering[m]: Having done the implementation in Monte, I think that the first question to answer is whether you want to write the capnpc-haxe plugin in Haxe itself, or in another language (OCaml, presumably?)
* abliss will be videohacking again at https://meet.google.com/hau-adhc-xmp for the next ~hour if anyone wants to drop by
<indolering[m]> simpson: So I can't just use an FFI to forward everything to a native or WASM library?
<indolering[m]> Haxe has nice parsing facilities, but each language still needs some massaging to make it feel native.
<indolering[m]> Or, rather, you could leverage Haxe's OCaml backend to produce the Haxe code but since you need to get nitty gritty with each language it would be better to use Haxe directly.
<simpson> indolering[m]: capnpc constructs a certain kind of request, which is itself encoded as a Capn Proto buffer, and sent to the plugin. So your plugin will have to be able to decode some Capn Proto bytes, as a bootstrapping exercise.
<simpson> You *could* use FFI to C++ to wrap the native libraries. I don't have much experience with that. The C++ library has its own event loop, which may be a boon or an obstacle.
<simpson> indolering[m]: If nobody's shown you yet, there is a page that guides this decision-making process: https://capnproto.org/otherlang.html
<indolering[m]> I read that.
<indolering[m]> I would need someone to help me out with wiring up to the C++ FFI. I could start by porting the Lua or JavaScript implementations to Haxe.
<indolering[m]> wrappers*
<indolering[m]> I was asking about WASM because I was hoping that would handle converting primitive types and naming conventions.
<indolering[m]> Then I would just need to wrap everything in the target language's version of a promise.
<indolering[m]> That's what I was hoping to have to deal with. Then I can just use macros, reflection, or string templates to customize it for each language.
<indolering[m]> But I guess one can't get around parsing Capn Proto because it includes things like structs and inheritance.
<indolering[m]> Then the first step is building a parser in Haxe for the compiler.
<simpson> Okay, having read the rest of the conversations over the past few days, I think I have a better idea of what you want to do. So, for starters, I think that there's no real reason that one cannot build Capn Proto modules on-the-fly. Like, if staged programming is available, then we should be able to dynamically compile.
<simpson> Perhaps, to build a capnpc-haxe, you might only build the bootstrap portion using FFI. Once bootstrapped, you would process bytes only in Haxe.
* indolering[m] reads through the JS and Lua implementations once more.
<kentonv> ugghh my UI code is so bad. It's so weird to write code and _know_ it sucks but not know what to do about it.
<JacobWeisz[m]> I mean, if it works, technically, publish and then file an issue that it sucks and that someone should make it better?
<kentonv> that's pretty much my plan
<kentonv> maybe zarvox can bail me out........
<abliss> Ian Denhardt: Punchline from yesterday's debug flailing: I had omitted a magic decorator (`@defer.inline_callbacks`) on my function. which meant that its code was never getting called; the parent code was calling `is_valid = module.check_password` and then `if is_valid: ...`, which passed because `is_valid` held the uncalled promise rather than the expected boolean result. Yay for untyped languages.
<abliss> I still don't know why attaching the debugger in some places seems to fail and not other places. My suspicion is still that some magic inside asyncio is converting the debugger's blocking writes into nonblocking ones somehow.
<kentonv> indolering[m], I feel like in a language like Haxe that targets transpiling to many other languages, it's ironically harder to rely on FFI to another language's implementation of Cap'n Proto... because you have to make that FFI-to-C++ work in every target language...
<indolering[m]> The real time savings of Haxe come from having a single testing and build system for all languages.
<indolering[m]> You get some time savings from having the same core business logic.
<indolering[m]> But you are correct in that when it comes to interacting with an FFI, it is only helpful in that there is a single codebase for handling each compilation target.
<indolering[m]> Although I am sure that the macro facilities are stronger than those of many of its targets.