<mnutt__>
Jan\: do you see any errors in the grain log when I hit it?
<mnutt__>
ohh, I think that error may be coming from sandstorm itself. and it may be an access control thing because you shared a grain with me and I then generated a tracker code from it. I’m not quite sure, though.
<zarvox>
that sounds like it could be a Sandstorm bug
<zarvox>
mnutt__: did you log in before generating a new tracker code?
<mnutt__>
possibly just a weird edge case where he granted me permissions and I’m trying to grant myself more permissions
<mnutt__>
zarvox: no, I didn’t
<zarvox>
thanks
<zarvox>
when you generate a roleassignment, you pass on only the permissions you have been granted, even if your sharing action says "grant all permissions"
<mnutt__>
I just did, let me try again
<zarvox>
that said, I'm not sure what happens when we have a forSharing: true token that chains to another forSharing: true token
<Jan\>
mnutt__: no errors in log
<zarvox>
which is what would happen in the view-as-anonymous-and-generate-new-apitoken flow
<dwrensha>
zarvox: you should be able to have arbitrarily long chains
<zarvox>
that's what I'd expect too, but I'm not sure we've ever tested that setup
<mnutt__>
(I realize now that I should really have a second permission, “view the dashboard”…though you can’t even do anything else anyway)
<dwrensha>
on master it apparently fails to generate a child token at all
<mnutt__>
and also that I should very plainly state that hummingbird fills a very specific niche: you have a lot of traffic, and you don’t care about any sort of persistence. basically you want to put it up on your wall to look cool. (that’s what we do with it, anyway)
<mnutt__>
ok, I’m off for a while, later
mnutt__ has quit [Quit: mnutt__]
<zarvox>
all righty then! sounds like we have a bug or two to file
jadewang has quit [Remote host closed the connection]
jadewang has joined #sandstorm
<Jan\>
zarvox: yeah the sahere button works for me
<Jan\>
*share
funwhilelost has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<zarvox>
yeah, the share button is fine as far as I know; it's the webkey generation and use of tokens created by sessions using that sharing token that seem to have issues
<dwrensha>
but I can repoduce it on Oasis with Gitweb
<dwrensha>
so I conclude that some of the changes on master fixed that problem since the last release
<zarvox>
hmmm, on Oasis, I found myself able to generate child tokens with offer templates and with the webkey dialog from a session using a webkey, but unable to use them on the API endpoint. Is that what you observe as well?
<asheesh>
This is a case where it was possible to have a provider with an accountId but no identityId which if we were validating these as Capn Proto data structures, wouldn't be possible, right?
<asheesh>
I'm happy to wait on that question until later, but if it's true, it's interesting, I think.
<asheesh>
I guess I'll go read the capnp.
<dwrensha>
zarvox: yeah, that's consistent with my observations
<dwrensha>
zarvox: and after applying #1214 the problem goes away
<zarvox>
asheesh: I observe that probably like >50% of our bugs live in the JS and could be caught if we had full-on typechecking :)
<zarvox>
dwrensha: yaaaaay thanks for investigating/fixing!
<asheesh>
zarvox: If we can migrate bug-sensitive code to TypeScript or something, I could deal with that.
<asheesh>
Either way dwrensha++ for finding this very subtle bug.
<asheesh>
+ fixing
<asheesh>
Props to Jan\ for finding it in the wild.
<dwrensha>
the webkey popup case is still broken
<dwrensha>
might also be an easy fix....
<Jan\>
my sandstorm will update automatically, right?
<dwrensha>
Jan\: yes, if your sandstorm.conf has "UPDATE_CHANNEL=dev"
<Jan\>
dwrensha: I'm using stable I think
<dwrensha>
I believe we only have the one channel for now
<dwrensha>
I think any value other than "UPDATE_CHANELL=none" will enable autoupdates
<dwrensha>
* "UPDATE_CHANNEL=none"
<Jan\>
dwrensha: ok so only one channel called dev ?
<dwrensha>
yeah, I think "dev" is the only channel right now
<Jan\>
dwrensha: yeah my conf has it
jadewang has quit [Remote host closed the connection]
<Jan\>
TypeError: Cannot call method 'split' of undefined
<Jan\>
at dispatchToMeteorOrStaticPublishing (app/server/pre-meteor.js:424:37)
<Jan\>
at Server.redirectToMeteorOrServeStaticPublishing (app/server/pre-meteor.js:414:12)
<Jan\>
at Server.emit (events.js:98:17)
<Jan\>
at HTTPParser.parser.onIncoming (http.js:2113:12)
<Jan\>
at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:122:23)
<Jan\>
at Socket.socket.ondata (http.js:1971:22)
<Jan\>
at TCP.onread (net.js:528:27)
<asheesh>
Hi Jan\
<asheesh>
That sounds like my code.
<asheesh>
Let me go read it.
<Jan\>
ok I do see a bunch of error: Error: Invalid authorization token [403] now that I look at the admin logs. I thought you guys were asking about the app log
<asheesh>
OK Jan\ I'll file a bug about the split of undefined thing!
<Jan\>
asheesh: there was 3 lines before it
<Jan\>
app/server/pre-meteor.js:424
<Jan\>
var hostname = req.headers.host.split(":")[0];
<Jan\>
^
<asheesh>
Yup, already found that in the source (-:
<zarvox>
looks like req.headers.host is undefined
<zarvox>
rather few clients will omit the Host: header these days though :/
<zarvox>
that might have been my curl-ing.
<asheesh>
Hah, hilarious.
<zarvox>
Hmmm, no. Even curl will send the Host: header by default, if you give it a URL including a hostname.
<zarvox>
could be a crawler or pen-test script though, bots will send all sorts of garbage
<Jan\>
Using LOAD DATA INFILE will greatly speed Piwik's archiving process up. To make it available to Piwik, try updating your PHP & MySQL software and make sure your database user has the FILE privilege.
<Jan\>
If your Piwik server tracks high traffic websites (eg. > 100,000 pages per month), we recommend to try fix this problem.
<Jan\>
Error: LOAD DATA INFILE failed... Error was:
<Jan\>
Try #1: LOAD DATA INFILE : SQLSTATE[HY000]: General error: 29 File '/var/piwik/tmp/assets/option-46b355af2ae1c784956e9322f5d6fc5e.csv' not found (Errcode: 13),
<Jan\>
Try #2: LOAD DATA LOCAL INFILE : SQLSTATE[42000]: Syntax error or access violation: 1148 The used command is not allowed with this MySQL version[42000]
<Jan\>
zarvox: ^^
<zarvox>
Jan\: Hmmm, looks like MySQL requires you to specifically enable FILE?
<Jan\>
zarvox: this message from your package of piwik
<Jan\>
btw IU'm Jan02 on github
<zarvox>
Yeah. I'm trying to understand it. I've never used LOAD DATA INFILE
<asheesh>
I'm more concerned by: '/var/piwik/tmp/assets/option-46b355af2ae1c784956e9322f5d6fc5e.csv' not found (Errcode: 13) (which is Permission denied)
<asheesh>
FWIW.
jacksingleton has quit [Ping timeout: 250 seconds]
eternaleye is now known as M-eternaleye
M-eternaleye is now known as eternaleye
<Jan\>
dwrensha: i think you should use stable wordpress, not alpha versions!
<Jan\>
dwrensha: i'm not sure if its the alpha or sandstorm but you can't install plugins
<asheesh>
I'm +0 on vagrant-spk mysql growing this configuration fwiw
jadewang has joined #sandstorm
paroneayea has quit [Read error: Connection reset by peer]
paroneayea has joined #sandstorm
<zarvox>
Jan\: I think I have a patch for you, but I'm not sure how to test this
<zarvox>
are you importing/exporting data or something?
bb010g has joined #sandstorm
<zarvox>
(also dang, Piwik takes 3 minutes to pack)
mnutt__ has joined #sandstorm
<zarvox>
also also the Piwik self-check appears to report even the currently-released Piwik package supporting LOAD DATA INFILE (Administration -> System Check)
<zarvox>
Jan\: I need more information on what you were trying to do to reproduce your issue
jacksingleton has joined #sandstorm
mnutt__ has quit [Quit: mnutt__]
simonv3 has quit [Quit: Connection closed for inactivity]
funwhilelost has joined #sandstorm
kecolus has quit [Quit: Leaving]
funwhilelost has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
ocdtrekkie has quit [Remote host closed the connection]
aldeka has quit [Remote host closed the connection]
aldeka has joined #sandstorm
ocdtrekkie has joined #sandstorm
|jemc| has quit [Remote host closed the connection]
<zarvox>
asheesh: is there a way to generate a Debian chroot without needing root? I'm trying to debootstrap, but something is going awry
<zarvox>
I run "fakeroot fakechroot debootstrap --arch=amd64 --variant=fakechroot jessie debian-fakechroot"
jadewang has quit [Remote host closed the connection]
jadewang has joined #sandstorm
jadewang has quit [Remote host closed the connection]
jadewang has joined #sandstorm
jadewang has quit [Remote host closed the connection]
jadewang has joined #sandstorm
jadewang has quit [Remote host closed the connection]
jacksingleton has quit [Ping timeout: 260 seconds]
mnutt__ has quit [Quit: mnutt__]
<ocdtrekkie>
Just unboxed and racked a new PowerEdge R730 server. God (and Comcast) willing, it's going to run a Sandstorm server on it.
jadewang has joined #sandstorm
jacksingleton has joined #sandstorm
jadewang has quit [Ping timeout: 260 seconds]
jacksingleton has quit [Ping timeout: 260 seconds]
<zarvox>
ocdtrekkie: :D
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 260 seconds]
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 245 seconds]
ocdtrekkie has quit [Remote host closed the connection]
aldeka has quit [Remote host closed the connection]
aldeka has joined #sandstorm
ocdtrekkie has joined #sandstorm
<Jan\>
zarvox: the only thing I did wa going to Administration -> System Check and pasted you the errors from there
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 260 seconds]
<Jan\>
more than 250 open issues guys
M-hrjet has quit [Ping timeout: 245 seconds]
M-hrjet has joined #sandstorm
kevix has quit [Quit: Leaving.]
kevix has joined #sandstorm
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 245 seconds]
kevix has quit [Ping timeout: 250 seconds]
kevix has joined #sandstorm
jinnko` has quit [Quit: Coyote finally caught me]
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 260 seconds]
KCinJP has quit [Quit: Leaving]
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 250 seconds]
itscassa has quit [Quit: Enough, I'm out.]
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 260 seconds]
[d__d] has quit [Remote host closed the connection]
[d__d] has joined #sandstorm
<mortehu>
In src/sandstorm/supervisor.c++ line 2010, `send()` is called on an expectSize request, but the promise is only kept in an automatic variable and destroyed soon after. Isn't there a risk that a future version of Cap'n Proto will cancel that request before it's sent?
jadewang has joined #sandstorm
<dwrensha>
mortehu: why would the request be cancelled?
jadewang has quit [Ping timeout: 260 seconds]
<dwrensha>
mortehu: oh, I think i understand your point
<dwrensha>
yeah, I think `expectSizeTask` should be stored somewhere
<dwrensha>
probably `expectSizeTask.detach()` would be the right thing
<dwrensha>
... because the caller is supposed to ignore exceptions thrown by expectSize()
<dwrensha>
mortehu: would you be interested in submitting a pull request to fix this?
simonv3 has joined #sandstorm
mnutt__ has joined #sandstorm
mnutt__ has quit [Client Quit]
[d__d] has quit [Ping timeout: 250 seconds]
<mortehu>
dwrensha: Yes.
<mortehu>
I just wanted to confirm that .detach() would be the right approach.
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 250 seconds]
mnutt__ has joined #sandstorm
<dwrensha>
mortehu: hm. I think the most correct thing would be to cancel the pump() if expectSize() throws an exception that's not of type UNIMPLEMENTED
<dwrensha>
I think the docs to util.capnp were written before Exception.Type.Unimplemented existed
<dwrensha>
it's a bit roundabout, but you could arrange for cancellation of pump() by using Promise.exclusiveJoin()
<dwrensha>
transform the expectSize result promise into a NEVER_DONE promise on success (including Exception.Type.Unimplemented), and then exclusiveJoin that promise with the pump() promise
<mortehu>
I have to shave the meteor yak first.
<mortehu>
Hadn't actually tried building it before.
<mortehu>
sandstorm, that is.
<dwrensha>
perhaps it would make sense for there to be an optional argument `expectedSize` to pump()
<mortehu>
It calls itself recursively, so it'd also need some way to signal that it's already been called.
mnutt__ has quit [Quit: mnutt__]
[d__d] has joined #sandstorm
mnutt__ has joined #sandstorm
<dwrensha>
Anyways, I do think the detach() approach is probably adequate. If something goes wrong, the pump() loop should notice soon enough.
[d__d] has quit [Ping timeout: 250 seconds]
jacksingleton has joined #sandstorm
jadewang has joined #sandstorm
jadewang has quit [Ping timeout: 245 seconds]
ocdtrekkie has quit [Remote host closed the connection]
aldeka has quit [Remote host closed the connection]
aldeka has joined #sandstorm
ocdtrekkie has joined #sandstorm
<mortehu>
It seems like the project doesn't compile with the version of v8 (4.6) included in node.js 5.1.0 (latest stable release). Getting «no member named 'New' in 'v8::String'», which seems to be present in the most recent v8 (4.9). I would expect node-capnp to use the version from node.js, though.
<dwrensha>
it should be using the node that's bundled with meteor
<dwrensha>
which is 0.10.x
<dwrensha>
are you building with `make continuous`?
<mortehu>
No, "m -j1". It rebuilds everything every time. :)
<mortehu>
I mean make
<mortehu>
I forgot not everyone has 'alias m=make'
funwhilelost has joined #sandstorm
[d__d] has joined #sandstorm
jacksingleton has quit [Ping timeout: 245 seconds]
<mortehu>
Ok, so it's using a much older V8 too. 3.28
mnutt__ has quit [Quit: mnutt__]
[d__d] has quit [Ping timeout: 260 seconds]
<dwrensha>
so `make -j1` tried to link with the wrong verson of v8?
<dwrensha>
if so, that's a bug
mnutt__ has joined #sandstorm
<mortehu>
make -j1 works as desired. I had to edit the Makefile manually to provide the include paths for where nodejs-dev ends up on Debian. I'm not using Meteor. I also had to edit some code to work with the more recent version of libuv present in Debian, but that was just a few lines.
NOTevil has joined #sandstorm
<dwrensha>
oh, so you're trying to build the c++ stuff without install Meteor
<dwrensha>
*installing
<mortehu>
Yes
<dwrensha>
Did installing Meteor fail for you somehow?
<mortehu>
I just want to compile some code. I don't want a second package manager with an apparently out-of-date repository. :P
<dwrensha>
you might enjoy commiserating with maurer, who has been trying to get Sandstorm to build on nixos
<mortehu>
If sandstorm is to ever be included in Debian, in needs to be able to compile against the versions that are current at that time.
<mortehu>
Aha, thanks
<mortehu>
For now, I give up compiling. Will you take care of the expectedSize thing?
<maurer>
They're building against meteor, which intentionally uses the older libuv
<maurer>
to prevent people from having to change their code
<maurer>
They may do it eventually, I can't speak for them, but they won't "have to"
<zarvox>
Jan\: which version of Piwik are you running? 2.15.0-sandstorm-2?
<mortehu>
Hm, you might be talking about the incompatibility between node.js and wrong versions of v8. I think when you use the versions that are distributed together, you don't see as many errors?
<Jan\>
zarvox: yes
<zarvox>
:/ that's super weird
<zarvox>
On both my local VM and on Oasis, there's a green checkmark next to Database Abilities: LOAD DATA INFILE
<zarvox>
with that same version
funwhilelost has joined #sandstorm
<Jan\>
zarvox: would sharing the grain help you debug?
<mortehu>
maurer: The only file that includes <uv.h> is ./deps/node-capnp/src/node-capnp/capnp.cc, and I don't get any UV related errors after making just a small number of changes there.
<zarvox>
possibly - what distro+filesystem are you running Sandstorm on?
<zarvox>
and do you have mandatory access controls enabled, like selinux or apparmor?
<maurer>
mortehu: Is your end goal "have node-capnp" or "run sandstorm" ?
<maurer>
mortehu: if it's "run sandstorm", then doing it without meteor is something I'd love to know how to do, but couldn't figure out last time I tried
<maurer>
If it's "have node-capnp", perhaps build that project directly
<zarvox>
Jan\: I'd need the full admin access to the grain, though; self-check requires admin perms
<Jan\>
zarvox: ok sent you PM
<mortehu>
maurer: My end goal is to be able to build sandstorm, to be able to write patches. I want to do that on Debian unstable, without installing another package manager. Ultimately I'd like to see sandstorm in Debian, and for that, it can't depend on outside or old packages.
<zarvox>
Jan\: d'oh I need the 'superuser' permission which is not granted by any of the provided roles
<maurer>
mortehu: Ah, you've got the same problem I have
<maurer>
mortehu: I have not solved this problem, I got partway and stopped
<zarvox>
Jan\: try making a webkey and pasting that to me, maybe?
<maurer>
mortehu: Even after you solve the building problems, you're going to hit problems with ekam, since it doesn't want to link against system copies of capnproto/node-capnp/libseccomp etc
<maurer>
I wish you good luck, and can show you what I've got, but what I've got is basically "I can do a hermetic build of the C++ portions"
<mortehu>
ekam seems like a good way to scare away contributors.
<Jan\>
zarvox: done
<zarvox>
Jan\: thanks, I can see it now!
<zarvox>
and sure enough, your instance complains.
<XgF>
mortehu: Meteor is a bunch of libraries not just a package manager as my limited understanding of JavaScript tells me
<zarvox>
Jan\: do you see anything suspicious in your audit log (/var/log/audit.log)? which distro is this on?
<maurer>
XgF: Meteor is a js framework, a particular version of node, a patched mongo, a particular libuv, and a packaged npm
<Jan\>
zarvox: Ubuntu Linux 15.04
<maurer>
XgF: along with some tooling over the top to help people deploy sites
<mortehu>
Does Sandstorm depend on those other parts to build?
<zarvox>
Jan\: sorry, that'd be an "ssh into the machine" sort of thing to debug
<Jan\>
zarvox: its a different log ?
<XgF>
Meteor just reminds me of my last webdev escapade, where I needed to use my package manager to install a package manager to install a package manager to get some JavaScript
<zarvox>
Yeah, it's one from the OS, that isn't under Sandstorm's control
<Jan\>
zarvox: oh!
<Jan\>
sure hold on
<zarvox>
My guess is that a Mandatory Access Control system like apparmor is restricting anything named /usr/bin/mysqld from accessing files outside of certain paths
<zarvox>
and that you'd see that denial in your audit logs
<Jan\>
zarvox: there is no audit.log in /var/log
<zarvox>
oh, hmm. /var/log/messages might have lines prefixed "audit:"
<zarvox>
?
<zarvox>
All the distros send logs to slightly different places, I forget which go where
<zarvox>
or maybe it shows up in the output of dmesg?
<zarvox>
or if this box has systemd-journald, then you can run "journalctl --since today" and look for audit in there?
<zarvox>
Jan\: as far as I can tell, the LOAD DATA INFILE support is not essential for Piwik to function, it just makes the archival process faster for sites with large amounts of data
|jemc| has joined #sandstorm
mnutt__ has quit [Quit: mnutt__]
funwhilelost has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
mnutt__ has joined #sandstorm
jadewang has joined #sandstorm
funwhilelost has joined #sandstorm
mnutt__ has quit [Quit: mnutt__]
<sprin>
asheesh: VideoPair is really neat. such a small amount of code, too!
<sprin>
very nice example of letting sandstorm handle the "hard stuff" (auth, sharing)
<sprin>
I had an occasion to use it, but I was on another machine, and couldn't package it for myself. Will you be publishing the package soon?
mnutt__ has joined #sandstorm
mnutt__ has quit [Quit: mnutt__]
mnutt__ has joined #sandstorm
<asheesh>
sprin: I was going to, but user testing discovered bugs!
<aldeka>
:D
<simonv3>
Hmm, I didn’t screenshot before I refreshed (sorry) but I just got an error that said basically “Error: Remote exception remote exception remote exception remote exception remote exception (etc)”
<simonv3>
Not quite sure what it meant, but refresh made it go away
<simonv3>
If I see it again, I’ll screencapture, but just thought I’d mention it in case that meant something to anyone.
<simonv3>
And the context of this (sorry, should have made that clearer above) is that I accessed a grain of an app that had been open for a while
<dwrensha>
simonv3: yeah, I've seen similar errors
aldeka has quit [Remote host closed the connection]
ocdtrekkie has quit [Remote host closed the connection]
bb010g has joined #sandstorm
aldeka has joined #sandstorm
ocdtrekkie has joined #sandstorm
mnutt__ has quit [Quit: mnutt__]
mnutt__ has joined #sandstorm
home has quit [Quit: Leaving]
notevil has quit [Quit: Zooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo!]
<mnutt__>
I don’t know how big most peoples’ grains tend get, but I’m happy to report that I now have a Davros grain with 167GB of photos running with no issues
<mnutt__>
(not that I’d expect any issues, just thought it was cool)
<asheesh>
That is pretty cool!
severine_ has joined #sandstorm
kentonv has joined #sandstorm
<asheesh>
sprin: I think that maybe the fact that Static Publishing on Sandstorm doesn't support HTTP range requests is breaking moving left and right, resulting in a "strange white box where the video should be" explained the other person.
<dwrensha>
asheesh: it doesn't support range requests?
<asheesh>
That's what kentonv & jparyani seem to think but I haven't verified.
<asheesh>
All I really know is "sometimes Chrome on OS X gets a white box where the video should be and it seems to relate to jumping around in the video's location".
<dwrensha>
I think it used to. maybe it no longer does with the new backend setup
<asheesh>
Interesting.
<kentonv>
dwrensha: correct
<dwrensha>
that is sad
<asheesh>
Oh, then I should self-host for more features!!
<kentonv>
I agree. The current hack is meant to be temporary.
<kentonv>
asheesh: this applies to self-hosting too. We don't have two codepaths.
<dwrensha>
I don't think self-hosting makes a difference
<asheesh>
Oh, okay.
<asheesh>
nm then.
<kentonv>
when we do the web publishing driver, it should be better.
<asheesh>
Maybe I can find a way to work around this then in my app for now. I'll see what is possible.
<asheesh>
It's weird that it seems to work OK in Chromium on Debian but not in Chrome on OS X.
<dwrensha>
I noticed sometime in the past year that Chrome started degrading more gracefully on Browserquest-in-Sandstorm
<dwrensha>
the audio used have annoying problems related to the lack of range requests, but nowadays it's basically adequate
<dwrensha>
and I didn't change anything
<sprin>
kentonv: thanks for the feedback on 1202!
jadewang has quit [Read error: Connection reset by peer]
jadewang has joined #sandstorm
<sprin>
one question I have: the reason for restricting response headers is to disallow the app from adding headers that sandstorm has responsibility for, right?
<sprin>
it's not quite the same security/privacy motivations for restricting *request* headers
<sprin>
so I have to wonder: why restrict response headers at all? as long as sandstorm removes/overrides any of the headers it owns, I can't think of a good reason not to let all other headers pass.
<sprin>
One reason I can think of is that it's "secure by default" to use a whitelist rather than a blacklist.
mnutt__ has quit [Ping timeout: 260 seconds]
<sprin>
So presuming we want to implement the response header whitelist to protect against the app sending common security-related headers,
<sprin>
we need to implement the whitelisting outside of the grain.
<zarvox>
If we have a whitelist, then when browser vendors add a new header with a security impact, then we do nothing, and apps work as before. If we do a blacklist, then we are changing behavior, and that could potentially break apps *after people are using them on Sandstorm*, which we really want to avoid
<sprin>
If an app really wanted to, they could bundle their own version of sandstorm-http-bridge that allowed arbitrary headers.
<sprin>
zarvox: yes, I agree with that benefit.
<dwrensha>
sprin: how so?
larjona has quit [Remote host closed the connection]
larjona has joined #sandstorm
<sprin>
zarvox: I'd like to add that reasoning in the docs, b/c I think it's not quite clear to app authors why request/response headers are so locked down
<zarvox>
it was proposed that additional headers matching an X-Sandstorm-* be whitelisted, so apps can add context in headers, rather than having to modify their POST data and rewrite a bunch of code that works elsewhere
<sprin>
dwrensha: yes, but I am assuming that any whitelist implementation needs an "additionalHeaders" structure in the WebSession
<zarvox>
but then the question is "who enforces constraints on the additionalHeaders"
<sprin>
zarvox: yes, that's the meat of my question
kecolus has quit [Quit: Leaving]
<sprin>
If we did not have to deal with the wildcard, this could be implemented as an enum with capnp, as I understand it
larjona has quit [Remote host closed the connection]
<sprin>
well, even with an enum, it is up to the consumer to interpret the enum to mean a list of valid keys in "additionalHeaders"
larjona has joined #sandstorm
<sprin>
ignore that last statement: it does appear to be possible to define the name attribute of a Header as an enum, fwict
<sprin>
which raises the question: why is headerWhitelist a :List(Text) inside of Context and not an enum inside of Header?
<dwrensha>
sprin: I guess you would translate between camelCase in Cap'n Proto and dash-case in HTTP?
<dwrensha>
xMyHeader and x-my-header?
<sprin>
hmm, good point. That doesn't seem like a great idea, and explains the list instead of enum, then.
<sprin>
Since there are headers in the wild that mix camelCase and dash-case
<sprin>
I believe both XML schema (XSD) and JSON schema allow for restricting string values to arbitrary lists, which can include regexes
<sprin>
So the restriction is handled by validating deserializers, meaning clients are guaranteed conforming values
<sprin>
What's the stance taken by capnp on such restrictions?
<sprin>
It appears enum is the only mechanism to restrict a field's values
<sprin>
I'm no fan of either XSD or JSON schema, but it's interesting that they both have similar mechanisms for expressing constraints
<sprin>
I will address the question of constraints on field value's to the capnp list.
<sprin>
With the current set of options for constraints, it seems like a `const` list of valid values/regexes is the best option
<sprin>
The drawback is that the client must always know how to interpret this list, and apply custom validation logic after deserializing.
<zarvox>
another option might be to have a "extraHeaders" field, with an arbitrary set of nested key/value pairs, which get serialized to/from X-Sandstorm-<key>: <value>
<sprin>
zarvox: That's interesting!
<zarvox>
it doesn't solve the "whitelist other reasonable headers" problem, though it does solve the "I want to attach some more data to this request outside the body" problem
<sprin>
It means that we can't implement a general response header whitelist, but it handles the case of the wildcard very nicely (and safely)
<sprin>
yeah
<zarvox>
possibly someone could produce a better name than "extraHeaders"
<sprin>
how about sandstormAppHeaders? since the prefix is X-Sandstorm-App-