kentonv changed the topic of #sandstorm to: Welcome to #sandstorm: home of all things sandstorm.io. Say hi! | Have a question but no one is here? Try asking in the discussion group: https://groups.google.com/group/sandstorm-dev
<ocdtrekkie>
WordPress is kinda a dystopian nightmare level Sandstorm package. Like people expect to have it, but trying to shoehorn such a large and comprehensive platform into a Sandstorm grain is... eh.
<ocdtrekkie>
Sandstorm's magic is in that single document granularity and ideally relatively stupid/simple apps, trying to strip bigger platforms to fit is hard.
<ocdtrekkie>
Like ownCloud/nextCloud has always been a major request but it's probably insanely impractical to try to do.
<crab>
yes. and i found the changes that were used to make the earlier package hard to follow (not at all meant as a criticism of the original changer, but he was obviously figuring out what to do/how to do it as he went along, so there were some missteps and so on).
<isd>
Yeah. Long term ideally we'd have smaller focused apps, rather than awkward ports of big things like wordpress. But bootstrapping...
<crab>
if anyone has suggestions for other things i can try to update, i'm at least 50% ears.
<crab>
(the other 50% is all left feet)
<isd>
Also, right now "public" grains are a bit of an open question on sandstorm, which is conceptually what you want a wordpress site to be. So instead it does static publishing (which is also desirable since it means not worrying about wordpress's lousy security track record), but that's awkward because no comments, etc.
<crab>
i'd be delighted if i could offer some people (e.g., non-technical friends and family) wordpress with static publishing via my sandstorm server, comments notwithstanding.
<ocdtrekkie>
isd: The ability to put a dynamic Sandstorm grain at a predefined friendly subdomain is one of those things that we probably need a story for at some point.
<ocdtrekkie>
Number one thing people seem to want to do.
<ocdtrekkie>
I'd love to be able to say bountyboard.ocdhost.sandcats.io would be a thing or the like.
<isd>
Yeah, definitely on my mental list of "mandatory eventually"
<isd>
crab: if you're looking for something simple, the gitweb app should be easy to update.
<crab>
i feel dumb. i can't even figure out how this installs gitweb
<crab>
unless it was written so long ago that there was no separate gitweb package in debian, and apt-get install git would also install gitweb
<crab>
but i guess it doesn't need to change gitweb at all, because gitweb leaves all authentication to a frontend web server
<isd>
Yeah, that's kinda why I recommended it -- there's not much to it.
<isd>
The only think interesting about it is the offer template.
<crab>
is dwrensha still around? i.e., should i try to contact him, or just do $whatever?
<isd>
I haven't heard much out of him since Sandstorm Inc. stopped paying him. You could ping him and see where he stands on things, if he's not interested in jumping back in maybe ask kentonv to transfer the package.
<crab>
no email address for him on github
<ocdtrekkie>
Kenton can reach any Sandstorm core dev if needbe, but for now I would re-key the app.
<ocdtrekkie>
If you get a working package, we can either reach out to David about a key transfer, or we can use a replacement key.
<ocdtrekkie>
(In the latter, we add an entry into Sandstorm's code which specifies the two appIds should be handled as one, essentially.)
<crab>
ok. so as far as an update goes, all i really need to do is figure out how to rebuild the package.
<crab>
i shall attempt to do so.
<crab>
it doesn't seem to be documented what sort of vm to setup. given that it installs nginx and doesn't need much else, would "diy" be the right thing?
<crab>
i'll try it.
<crab>
i see that it's trying to build on top of the debian/contrib-stretch64 box
<crab>
should i try to use a recent debian?
<crab>
it's a real pity that libvirt provider is unmaintained and doesn't work with recent vagrant and nobody cares.
<ocdtrekkie>
crab: GitWeb uses the diy stack.
<ocdtrekkie>
The .sandstorm folder has a stack file that so-specifies.
<crab>
aha. so i see now, thanks.
<crab>
can i think of what vagrant-spk is doing as being something like building a docker image?
<ocdtrekkie>
It is somewhat similar, yes, in that you are scripting the assembly of a VM here.
<ocdtrekkie>
Whereas traditional spk development would be that you install the actual program and then pack it.
<ocdtrekkie>
Some Sandstorm apps have separate commands for launching a new grain than reopening an established one defined in their pkgdef, I believe.
<ocdtrekkie>
Very few apps actually use this, I think preferring to detect on run what state the grain data is in.
<crab>
wait, spk is a thing by itself, separate from vagrant-spk?
<ocdtrekkie>
Yeah.
<crab>
somehow that never occurred to me.
<ocdtrekkie>
spk is the original Sandstorm packaging tool, and what vagrant-spk is actually calling to build the package.
<ocdtrekkie>
vagrant-spk is a way of building a VM to install Sandstorm, install a package to dev on, and then package the app.
<ocdtrekkie>
Generally speaking, if an app's repo has a .sandstorm folder, it was probably built with vagrant-spk, and if it's pkgdef file lives in the root of the repo, it probably was packaged with spk.
<isd>
worth noting, spk itself does somewhat less that folks sometimes expect. It just bundles up the packages and signs it, but there's no build infra included like with vagrant-spk
<isd>
*than
<ocdtrekkie>
Oh, also, there's meteor-spk too, which only works for Meteor apps.
<crab>
i read the discussions on the -dev list about meteor etc., but i haven't much looked into what meteor is/does yet.
<crab>
(i have a general idea, i haven't dug into the details)
<isd>
It's a javascript web framework.
<isd>
Sandstorm is written using it, and there are a few apps that use it as well.
<crab>
docker-spk README says "the filesystem must be constructed to behave correctly inside Sandstorm's sandbox environment"
<crab>
i have a general understanding of what that means, but where should i look for documentation about it? would that be in spk, or does spk itself have the same expectation, that it's given a working layout to package up?
<crab>
(i know i don't need to know any of this to build gitweb-sandstorm again, but the ETA for the debian/contrib-stretch64 box is 1h+ here, so i'm just poking around)
<crab>
i'll get it working on stretch, and then look at updating the base image to buster.
<ocdtrekkie>
Well, some things like /proc aren't available inside a Sandstorm grain. The biggest thing for Sandstorm grains is that most of the file system is not writeable. /var is the grain's persistent storage, and I think /tmp is available but discarded on grain close.
<ocdtrekkie>
Essentially, just because it works in Docker doesn't mean docker-spk can turn it into a working Sandstorm app.
<isd>
Yeah, that comment isn't really more or less relevant to docker-spk, it's just worth stressing that you can't just convert arbitrary docker images and have them work.
<ocdtrekkie>
We upgraded vagrant-spk from Jessie to Stretch and it was a major undertaking that required updating each and every vagrant-spk stack.
<ocdtrekkie>
For diy, I will not really matter probably if you change the box it's using to build the app. But moving vagrant-spk in general up from Stretch is a decently hefty undertaking.
<ocdtrekkie>
One we should probably tackle, but probably after someone other than kentonv has merge rights to it.
<isd>
Also, fwiw, the "stacks" are just scaffolding, so after init "diy" isn't really any different than any other stack, and you can hack on and tweak all of them as needed for your app.
<isd>
vagrant-spk doesn't do anything special depending on what stack you picked, it just dumps some starter scripts in a directory for you.
<ocdtrekkie>
Indeed.
<ocdtrekkie>
The starter scripts built for different stacks though usually do hardcode installing the correct version of a package for the flavor of Linux used.
<ocdtrekkie>
Also sometimes distros conduct horrible crimes that break good functionality.
<ocdtrekkie>
Like Debian's extremely vexing choice to install MariaDB when you "apt-get install mysql".
<ocdtrekkie>
It's been two years since I found that out and I'm still kinda angry about it.
<crab>
i take it mariadb is incompatible with mysql somehow?
<ocdtrekkie>
In subtle ways, yes.
<ocdtrekkie>
It broke the vagrant-spk stack, for example. And I spent a lot of time trying to figure out why, pouring over mysql docs.
<ocdtrekkie>
Just to discover that, rather than throwing an error for not having a package, like any competent group of package library maintainers would do, they decided to install a competitor which responds to most of the same commands... more or less.
<ocdtrekkie>
I still kinda think Oracle should C&D them for installing a competitor using their trademark.
<ocdtrekkie>
It's bad to root for Oracle against open source software most of the time, but I find Debian's behavior here so wrongheaded, I want someone to make them fix it.
<crab>
that sucks.
<crab>
but it's not the only such decision in debian (e.g., switching back and fortch between ffmpeg vs libav)
<isd>
Honestly it's probably in Oracle's best interest for folks not to ask why MySQL isn't in Debian. They might get an answer.
<ocdtrekkie>
I mean, fundamentally, I think a package manager is untrustworthy if you say "install X" and it decides "install Y" is an acceptable behavior. Unpopular opinion, but I don't really think anyone should trust Debian's package manager until that policy is revised.
<ocdtrekkie>
isd: I have no problem with them not including MySQL. I have a problem with them believing it's okay to install something that's not MySQL when I send the command to install mysql.
<crab>
do i understand correctly that by the time the spk package is built, it no longer matters which os you started to build on, just that the right files are in the right places?
<ocdtrekkie>
Pretty much, yeah.
<crab>
i.e., if i wanted to make vagrant-spk use alpine or something, spk/sandstorm wouldn't care?
<crab>
ok. that's helpful. thanks.
<ocdtrekkie>
Sandstorm uses the kernel of whatever server Sandstorm is installed on, but basically all of the dependent files otherwise are included in the package.
<isd>
Yeah. Sandstorm apps are basically containers, just a bit more heavily sandboxed and with a different file format for images/apps vs. docker.
<crab>
ok, so i have a question about static files.
<crab>
when an app, say radicale, has a giant blob of javascript that's served to the client, does the request for that go through all of the same steps that a request to the app itself go through? the http bridge, and so on?
<crab>
or does it depend on how you configure, say, the nginx that you choose to run in your package?
<crab>
so where does nginx fit, inside the grain? between sandstorm-http-bridge and the http app?
<crab>
so in this case, i'm running nginx in sandstorm-gitweb because gitweb is a fastcgi app, and i need something that speaks http to the bridge and fastcgi to gitweb?
<isd>
Exactly.
ocdtrekkie has quit [Ping timeout: 258 seconds]
<isd>
Communication pattern is: raw sandstorm api <---> bridge <---> nginx <---> fcgi/gitweb
<crab>
ok. so i can put try_static etc. in my nginx, but as far as sandstorm itself is concerned, there's no difference between static assets and app requests
<dckc>
re spk and vagrant-spk and such... my favorite hack was figuring out how to turn a nixpkg into a spk
<crab>
wonder what's special about the sandstorm boxes.
<crab>
oh. maybe they just have sandstorm installed?
<crab>
can anyone else download the sandstorm/debian-jessie64 box?
<crab>
it's the only box listed on the vagrant box search page
patrickod has quit [Ping timeout: 250 seconds]
patrickod has joined #sandstorm
_whitelogger has joined #sandstorm
_whitelogger has joined #sandstorm
_whitelogger has joined #sandstorm
ocdtrekkie has joined #sandstorm
<ocdtrekkie>
crab: The Sandstorm-specific Jessie box is deprecated.
<ocdtrekkie>
Are you sure you aren't using an old version of vagrant-spk?
<ocdtrekkie>
vagrant-spk should by default be using contrib-stretch64
<crab>
vagrant-spk v0.236
<crab>
isn't stretch older than jessie?
<crab>
no, it isn't.
<ocdtrekkie>
No
<ocdtrekkie>
Oh, are you using GitWeb's original folder here?
<crab>
yes, i am
<crab>
that's where the jessie is coming from
<ocdtrekkie>
Because you are probably using the old Vagrantfile in the GitWeb repo then.
<crab>
should i try to setupvm and then make the gitweb changes from scratch against the new vagrantfile/other generated files?
<ocdtrekkie>
The sandstorm/jessie box is long gone from the cloud, it's part of why we did the switch to stretch to begin with
<ocdtrekkie>
You honestly could probably replace the Vagrantfile and global-setup.sh entirely with the versions found in the vagrant-spk repo.
<crab>
when i do setupvm, i do get a stretch box
<ocdtrekkie>
Yeah. Grab the Vagrantfile and global-setup.sh from a new project.
<ocdtrekkie>
And paste them over the ones in GitWeb's directory.
<ocdtrekkie>
So that you're using a current/working version.
<crab>
done. rerunning.
<ocdtrekkie>
Another handy tool is vagrant-spk vm destroy if you need to clear out anything vagrant-spk built with old files. But since you failed to even download the VM before, my guess is you shouldn't need to do that here.
strugee has quit [Ping timeout: 252 seconds]
strugee has joined #sandstorm
<xet7>
crab: What do you mean with "it's a real pity that libvirt provider is unmaintained and doesn't work with recent vagrant and nobody cares" ? I use vagrant-libvirt about daily with my laptop having remote desktop with virt-manager to remote server VMs. What exactly does not work?
<xet7>
hmm, not laptop, I mean desktop
<crab>
which version of vagrant are you using?
<xet7>
On my desktop Debia Unstable, Vagrant 2.2.6+dfsg-2 and vagrant-libvirt 0.0.45-2. On server Ubuntu 18.04.3 LTS, vagrant 2.2.6
<crab>
that is very nice to hear. how did you install vagrant-libvirt?
<crab>
the vagrant-libvirt docs claim only 1.8 is supported, and though i had managed to install it somewhere around 2.1.x, it stopped working at some point.
<crab>
i don't remember any more what the problem was, but it was something not trivial.
<crab>
but if you're using it with 2.2.6, i should definitely try it again.
<xet7>
Hmm, this seems to be version 0.0.45-2 package at unstable debian
<crab>
thanks. i'll try it from the package (i used to do plugin install before).
<xet7>
Usually I just install virt-manager to both client and server, and maybe also libvirt-bin, and then connect to server with QEMU/KVM over SSH root user and machine name, having machine name ssh details at ~/.ssh/config
<xet7>
At server start-libvirt.sh I have:
<xet7>
systemctl start libvirt-bin
<xet7>
systemctl start vagrant-libvirt
<xet7>
virsh net-start vagrant-libvirt
<crab>
BTW: once i replaced the Vagrantfile/global-setup.sh from the gitweb-sandstorm project with newer versions, and tweaked setup.sh to install the gitweb package, i have a working gitweb grain on my local sandstorm
<crab>
\o/
<xet7>
Nice :)
<crab>
a small step for man, many small sideways steps for a crustacean.
<crab>
it looks to be working fine.
<crab>
Error message given during initialization: Unable to resolve dependency: user requested 'mutate (= 1.0.0)'
<crab>
xet7: 👆 is what i get with vagrant 2.2.6 and vagrant-libvirt 0.0.45, does it ring any bells?
<crab>
i tried to install vagrant-mutate, and there's a 1.2.0 package.
<crab>
(this is, btw, not the error i had earlier)
<crab>
let's see what vagrant plugin repair does…
<crab>
ok, it doesn't break any more.
<crab>
xet7: thanks for the tip. maybe whatever the problem was before, it's fixed now. i'll download a libvirt box and bring up some libvirt hosts later.
<ocdtrekkie>
Any idea what the version difference is between the GitWeb we have and the GitWeb you've packaged? Any significant stuff?
<ocdtrekkie>
Presumably this is why we don't worry too much about out of date Sandstorm apps. :P
<ocdtrekkie>
Though updating to the latest plus the PR isd wanted is probably a worthy release.
<crab>
so we're about 20 changes behind, none especially earth-shattering. it turns a few more things into links, and that's about it.
<crab>
(i'm looking at the upstream git.git repo for the changes to the gitweb/ subdir)
<crab>
well, whether worthy of release or not, i'm glad i did this. it let me understand the basics with a trivial example. much easier than trying to update wordpress.
<crab>
having already done it, if you're interested in helping me to make a new release, that's cool.
<crab>
but i have to admit it's not terribly compelling.
<ocdtrekkie>
When Kenton's got some time, we should ask him about getting the key from David.
<ocdtrekkie>
Even if it's not super compelling, enough people dislike "old apps" that pushing a release is kinda nice to do.
<isd>
Yeah, I suggested it partially because there should be fewer surprises than other things. I'm more interested in that PR landing than the version bump honestly.
<isd>
It's probably a bigger deal than anything that's happened upstream
<isd>
You could add the clipboard button on top of that if you want.