mnutt_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
isd has quit [Ping timeout: 240 seconds]
mnutt_ has joined #sandstorm
aeviator has joined #sandstorm
aeviator has quit [Ping timeout: 264 seconds]
isd has joined #sandstorm
<mnutt_>
everything is working great using libreoffice to display doc/xls/ppt previews
<mnutt_>
if for whatever reason I wanted to sandbox libreoffice further, is it possible / worth it to do inside of sandstorm?
<mnutt_>
also...libreoffice requires its config files to be writeable, and it touches them when it starts. I set its config directory inside /var, but it means that if a user previews anything, they automatically get added 1.5MB of dead weight to their grain
<zarvox>
basically no - we disable most of the syscalls that you'd want to use for sandboxing (unshare, chroot, setuid, seccomp)
<zarvox>
you could maybe put the config directory inside /tmp, which is tmpfs (capped at 16MB)?
<mnutt_>
ah, no problem. it seemed like more trouble than it was worth, I just have a couple of apps that I'd prefer not to have access (libreoffice, imagemagick) but the potential damage is limited to the grain at least.
<mnutt_>
libreoffice also has a thing where if you try to start it and your config files aren't there, it'll write them then die
<zarvox>
Yeah. Longer-term, it'd be nice to have apps be able to spawn sub-grains, so they can opt into further isolation like this
<mnutt_>
so I guess it's a tradeoff between either writing to /tmp and having to manage this thing dying the first time it runs, or take the 1.5MB hit
<zarvox>
(there are other reasons we'll need to make that work eventually, like having static publishing be implemented as a driver that spawns grains accountable to the user rather than to the admin)
<zarvox>
can you build the tmpdir contents and ship those in the package, and copy them to /tmp before the first launch?
<mnutt_>
ah, cool. sub-grain feels a lot cleaner long-term
<mnutt_>
oh, that's a good idea
<zarvox>
also hi from Minneapolis everyone! the Minnesota state fair is great!
<kentonv>
some day we'll let you launch a separate subordinate grain to run libreoffice, and delete it when it's done, or something
<kentonv>
oh zarvox said that already
<kentonv>
I should read all the things before I reply
<isd>
So I'm fiddling with wrapping more of the sandstorm apis in idiomatic-go interfaces, and as part of testing changed some logic so the grain waits for the call to done() on the bytestream for the body of an http reply.
<isd>
result: browser never sees data already written, grain never hears back from done()
<isd>
kentonv: I'm wondering if the senderPromise hack that you proposed/I implemented has anything to do with this, but I would expect the browser to still see the data if that was all that was going on
<isd>
Unless sandstorm is waiting on websession.get() and friends to return
<isd>
...in which case I've just triggered a deadlock
<isd>
but I don't know enough about how sandstorm treats this or have enough of a handle on capnproto's rpc protocol to be sure of what's going on.
wolcen has quit [Ping timeout: 250 seconds]
<isd>
Yeah, it's not just that that's never resolving, I think it's the senderPromise hack. I don't really understand the browser never sees anything though.
aeviator has joined #sandstorm
aeviator has quit [Ping timeout: 276 seconds]
_whitelogger has joined #sandstorm
_whitelogger_ has joined #sandstorm
_whitelogger_ has joined #sandstorm
_whitelogger has joined #sandstorm
crw has joined #sandstorm
<mnutt_>
libreoffice adds about 46MB overhead. I guess it's not that bad in the scheme of things? Anybody know the average package size?
<mnutt_>
spk size, I mean
_whitelogger has joined #sandstorm
_whitelogger has joined #sandstorm
<ocdtrekkie__>
That feels like a lot of overhead for just one filetype preview.
<ocdtrekkie__>
Most packages seem to range in the 20-40 MB area.
<ocdtrekkie__>
A few particularly bulk outliers are in the 80-120 MB area.
frankier has joined #sandstorm
<isd>
libreoffice is an enormous application
<asheesh>
I think if one day, the core Sandstorm team (myself included) implements "app addons" or some other way for apps to have features that are optional, it makes sense to expect SPKs to remain small.
<isd>
You don't get desktop software that's much bigger unless you start looking at things like games that have all of these assets
<asheesh>
For as long as we don't, I expect SPKs to grow larger and larger while remaining useful, and I think that's just life.
<asheesh>
I have a script that loops over all apps in the app market, though. Let me dig it up.
<asheesh>
Let me write that loop to find out this answer.
<isd>
I mean, you can always shell stuff out to another grain via the powerbox. You could always add an app that just exposes a capnp interface to interesting bits of libreoffice functionality. The harder questions are about how to communicate to the user about what's going on there.
<asheesh>
I'm 100% A-OK with distributing 47MB more, personally. kentonv can say what he thinks though. I think that in the long run, "small app packages" is either incompatible with featureful apps, or incompatible with a lack of easy separation of functionality into separate grains supported by the platform.
<asheesh>
So I'm excited to see apps grow in functionality and think that the loss of some disk space to that cause is A-OK.
<mnutt_>
and Word's bullets come out as two unknown unicode characters
<mnutt_>
which I went to some effort to find and replace, but there are a whole range of them I haven't tracked down
<asheesh>
Fascinating.
<asheesh>
That sounds like utf8 vs latin1 fighting fwiw
<asheesh>
Is the first unknown Unicode character Â? Oh I guess you said unknown Unicode character.
<mnutt_>
sometimes they come out as \uF0B7\uF0B7 and other times \uF077\uF077
<asheesh>
Wow, well, OK!
<mnutt_>
I'm just assuming that it's supposed to be two characters because that's how it renders. but there should only be one bullet
<mnutt_>
maybe some proprietary word thing that is bullet + spacing
<mnutt_>
(I've been mainly testing on resumes because I have a lot laying around, there are lots of bullets...)
<asheesh>
It doesn't seem to successfully index pubform.doc, which is the sample Word doc I linked to in the PHP app.
<asheesh>
s/index/preview/
<asheesh>
It's spinner-ing at "Loading preview" for "select_license_programguide.docx" which is googlable
<asheesh>
PROPFIND /remote.php/webdav/select_license_programguide.docx 207 2.067 ms - -
<asheesh>
{ Error: read ECONNRESET at exports._errnoException (util.js:1026:11) at Pipe.onread (net.js:563:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
<asheesh>
mnutt_: Well there's my demanding bug-report-of-the-night for you. : )
<mnutt_>
yep, definitely does not work for me either. that's what I get for posting a spk I've only tried with vagrant-spk dev
<asheesh>
: )
<asheesh>
Needs more alwaysInclude in sandstorm-pkgdef.capnp.
<mnutt_>
weird, I would expect adding more docs with vagrant-spk dev to pick up on more files
<asheesh>
Yeah, I'm honestly not sure. I didn't try my sample app packed either.
<asheesh>
I had better fall asleep for now though
<asheesh>
.
isd has quit [Quit: Leaving.]
<mnutt_>
I'm calling it a night as well. Please disregard the spk above, I'll have a working one out tomorrow
jemc has joined #sandstorm
mnutt_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<asheesh>
BTW the grain can navigate out of the Sandstorm frame in my Chromium, presumably because the Chromium PDF renderer ignores Content-Security-Policy restrictions on navigation.
<asheesh>
(requires me to click a link, but still)
<asheesh>
I found that fascinating.
xet7 has quit [Quit: Leaving]
aeviator has joined #sandstorm
aeviator has quit [Ping timeout: 250 seconds]
niek has joined #sandstorm
ocdtrekkie_ has joined #sandstorm
sydney_u1tangle has joined #sandstorm
decause_ has joined #sandstorm
iangreenleaf has quit [Ping timeout: 240 seconds]
sydney_untangle has quit [Ping timeout: 240 seconds]
ocdtrekkie has quit [Ping timeout: 240 seconds]
ecloud has quit [Ping timeout: 240 seconds]
XgF has quit [Ping timeout: 240 seconds]
niekie has quit [Ping timeout: 240 seconds]
decause has quit [Remote host closed the connection]
XgF has joined #sandstorm
iangreenleaf has joined #sandstorm
ecloud has joined #sandstorm
_whitelogger_ has joined #sandstorm
_whitelogger_ has joined #sandstorm
jemc has quit [Ping timeout: 240 seconds]
jemc has quit [Ping timeout: 240 seconds]
digitalcircuit has quit [Read error: Connection reset by peer]
digitalcircuit has quit [Read error: Connection reset by peer]
digitalcircuit_ has joined #sandstorm
digitalcircuit_ has joined #sandstorm
digitalcircuit_ is now known as digitalcircuit
digitalcircuit_ is now known as digitalcircuit
xet7_ has joined #sandstorm
xet7_ has joined #sandstorm
_whitelogger_ has quit [Remote host closed the connection]
_whitelogger_ has joined #sandstorm
_whitelogger_ has joined #sandstorm
_whitelogger_ has quit [Remote host closed the connection]
_whitelogger_ has joined #sandstorm
_whitelogger_ has joined #sandstorm
_whitelogger has quit [Remote host closed the connection]
_whitelogger has joined #sandstorm
_whitelogger has joined #sandstorm
_whitelogger has joined #sandstorm
nwf has quit [Ping timeout: 240 seconds]
nwf has joined #sandstorm
dwrensha has joined #sandstorm
mnutt_ has joined #sandstorm
mnutt_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
Telesight has joined #sandstorm
mnutt_ has joined #sandstorm
mnutt_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
mnutt_ has joined #sandstorm
aeviator has joined #sandstorm
<aeviator>
Hi everybody…just for the log: I was asking for making Gitlab Continious Integration working with the Sandstorm grain. As it is not supported, one solution I found (and I'm gonna use) is to set up a separate Gitlab instance on the server, and include a push to the gitlab grain in sandstorm as the last step of the CI pipeline. That way one can use the gitlab grain like one would push directly to the grain and have a fully working CI environ
<aeviator>
@dwrensha thank's again for your support
aeviator has quit [Ping timeout: 252 seconds]
aeviator has joined #sandstorm
aeviator has quit [Ping timeout: 252 seconds]
nwf has quit [Ping timeout: 255 seconds]
aeviator has joined #sandstorm
nwf has joined #sandstorm
aeviator has quit [Ping timeout: 244 seconds]
mnutt_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
decause_ is now known as decause
decause has quit [Changing host]
decause has joined #sandstorm
aeviator has joined #sandstorm
ThePurgingPanda has quit []
ThePurgingPanda has joined #sandstorm
aeviator has quit [Ping timeout: 264 seconds]
mnutt_ has joined #sandstorm
<mnutt_>
ok, I'm back to stumped again. I have the app working in `spk dev` mode, but my packed app doesn't work. I've tried strace-ing it to find stat()s, anything else obvious I should try? Is there a way to enter a grain not running in dev mode?
mnutt_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
mnutt_ has joined #sandstorm
<kentonv>
mnutt_: you could try adding various things to alwaysInclude and see if it makes a difference
mnutt_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
ThePurgingPanda_ has joined #sandstorm
ThePurgingPanda has quit [Ping timeout: 264 seconds]
ThePurgingPanda has joined #sandstorm
ThePurgingPanda_ has quit [Ping timeout: 250 seconds]
mnutt_ has joined #sandstorm
ThePurgingPanda_ has joined #sandstorm
ThePurgingPanda has quit [Ping timeout: 244 seconds]
ThePurgingPanda has joined #sandstorm
ThePurgingPanda_ has quit [Ping timeout: 276 seconds]
<mnutt_>
rocketchat gets a nice mention in the comments
ThePurgingPanda_ has joined #sandstorm
ThePurgingPanda has quit [Ping timeout: 265 seconds]
ThePurgingPanda has joined #sandstorm
ThePurgingPanda_ has quit [Ping timeout: 244 seconds]
ThePurgingPanda_ has joined #sandstorm
ThePurgingPanda has quit [Ping timeout: 244 seconds]
ThePurgingPanda has joined #sandstorm
ThePurgingPanda_ has quit [Ping timeout: 265 seconds]
aeviator has joined #sandstorm
isd has joined #sandstorm
isd has quit [Ping timeout: 244 seconds]
<DanC_>
"My grains' total size: -2.72e+7B"
isd has joined #sandstorm
<pod>
in the admin/personalization settings page the link to "Hacker CMS" sets the destination sandstorm URL to be local.sandstorm.io. should I open an issue on GH?
<dwrensha>
DanC: !
<dwrensha>
DanC: can you find any particular grain that reports a wildly inaccurate size?
<DanC_>
the total size is now 366MB (which still seems high, but...)
<DanC_>
why the heck does tinytinyrss take 27.5MB?
<isd>
Also, with openWebSocket, how/when does the websocket get closed? there doesn't seem to be an API for this.
<dwrensha>
isd: one end closes the stream by dropping its WebSocketStream handle (which points to the other end)
<isd>
dwrensha: Ok, thanks. Any idea about the API thing?
<dwrensha>
I'm trying to understand what you're doing
<dwrensha>
you're wrapping the Cap'n Proto interfaces in HTTP interfaces?
<dwrensha>
or is the problem that RPC in go-capnprotos is not very idiomatic Go?
<dwrensha>
"So I'm fiddling with wrapping more of the sandstorm apis in idiomatic-go interfaces, and as part of testing changed some logic so the grain waits for the call to done() on the bytestream for the body of an http reply."
<dwrensha>
which APIs? which bytestream?
<isd>
dwrensha: I have an app that I want to listen for a websocket via an HTTP API (i.e. not in an iframe), and I'm talking to sandstorm directly via capnproto,
<isd>
The docs say that for these requests to get through, you have to specify apiPath in sandstorm-pkgdef.capnp
<isd>
but that variable is inside bridgeConfig, and I'm not using sandstorm-http-bridge
<isd>
so I'm trying to understand whether the docs are assuming sandstorm-http-bridge, and if so, how that changes what I need to do.
<isd>
I think I figured out the stuff that I was messing with last night.
<dwrensha>
oh, so there is the WebSession vs ApiSession question
<dwrensha>
when you grain gets a UiView.newSession() call, the `sessionType` argument indicates which kind of session is being requested
<isd>
Ah, that makes sense.
<dwrensha>
sorry, going AFK now
<dwrensha>
ApiSession extends WebSession
<dwrensha>
I think it works to just return a WebSession every time
<isd>
dwrensha: I will try that, thanks for your help.
<mnutt_>
it's unfortunately 100MB, because there are tons of libreoffice libs and they're lazily loaded depending on which files are being converted, so I took the conservative approach and added them all
<isd>
dwrensha: will keep that in mind
mnutt_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
Telesight has quit [Quit: Leaving.]
<digitalcircuit>
Does Sandstorm do delta app updates? That might at least limit the impact of large apps on limited bandwidth/data capped connections.
<pod>
dwrensha: that PR LGTM. merge at will :) thanks for the quick turnaround!
ThePurgingPanda_ has joined #sandstorm
ThePurgingPanda has quit [Ping timeout: 240 seconds]
aeviator has quit [Ping timeout: 276 seconds]
<dwrensha>
isd: actually, I think always returning a WebSession is fine. What doesn't work is "preventing" an app from being able to use an HTTP API by throwing an error when an ApiSession is requested. In that case, Sandstorm falls back to requesting a plain WebSession anyway.