16:48 <TMM> Hi all, I have a question: My sandcats.io certificate expired today I read a blogpost that the certificates are supposed to be migrating to letsencrypt

16:48 <TMM> but that appears to not have happened on my install

16:54 <TMM> Can I do this from the command line? Due to HTST I can't get at the web interface at all anymore

17:09 <kentonv> TMM, check /opt/sandstorm/sandstorm.log to see if there's any hints what went wrong. If you restart sandstorm, it should immediately retry getting a cert...

17:10 <kentonv> err

17:10 <kentonv> /opt/sandstorm/var/log/sandstorm.log

17:43 <TMM> It doesn't, it says it'll retry the renewal, but it's not

17:43 <kentonv> I think it retries something like every 6 hours, but should retry immediately if you do `sudo sandstorm restart`

17:43 <kentonv> I'd try that and then watch the log to see what happens

17:45 <kentonv> if it fails to get a cert and the reason why isn't obvious from the log, then it would help if you could e-mail the log to kenton@sandstorm.io so I can try to decipher it

17:46 <TMM> Give me a sec

17:46 <TMM> Are the certs still on the filesystem?

17:47 <kentonv> new certs will only be stored in mongo, not placed on the filesystem

18:05 <TMM> Ah, that may be my problem then

18:07 <kentonv> oh... uh oh, are you using some separate server to terminate TLS, pulling the certs from the sandcats directory? I guess I didn't consider that case when implementing ACME support.

18:22 <TMM> yeah

18:22 <TMM> that's exactly what I do

18:23 <TMM> Because otherwise sandstorm would require its own IP address on my box

18:23 <TMM> Can I move an existing sandstorm install to a different domain?

18:24 <TMM> I can just disable https on sandstorm itself and get a wildcard cert from letsencrypt for my own subdomain

18:25 <kentonv> doh, ok... so as a quick-but-temporary fix, you can revert to globalsign

18:25 <kentonv> do `sudo sandstorm mongo` to get to the mongo prompt

18:26 <kentonv> then run the mongo command: db.settings.remove({_id: "acmeAccount"})

18:26 <kentonv> hmmm

18:26 <kentonv> actually, I guess it's going to try to create a new account basically right away, ugh

18:27 <kentonv> ok, never mind that

18:27 <kentonv> you can extract the tls keys from mongo

18:27 <kentonv> db.settings.findOne({_id: "tlsKeys"})

18:27 <TMM> no, that did work :)

18:27 <TMM> it said 'planning to migrate to acme' but didn't on the first run

18:28 <kentonv> ah, well... ok, then you have a certificate good for a few days

18:28 <kentonv> I probably should have suggested pulling out the let's encrypt certificate in the first place, since that'd be good for three months. Oops.

18:28 <TMM> I like the idea of just extracting the keys from mongo

18:29 <TMM> I didn't see your suggestion for going into mongo until after I had alreadt deleted the acmeaccount

18:29 <TMM> sorry :-/

18:29 <kentonv> my fault, but it's fine

18:29 <TMM> I could just do a new install of sandstorm I guess?

18:30 <TMM> Can I just save all my grains and import them in a new install?

18:30 <kentonv> nah just go with the globalsign cert for now and you'll get a new LE cert again in a week

18:30 <kentonv> this hacky terminal command should work for extracting the key: echo 'db.settings.findOne({_id: "tlsKeys"})' | sudo sandstorm mongo

18:31 <kentonv> so if you had a cron job that does that once a day and saves the output to the right places you should be good

18:31 <TMM> OK yeah, that would work just fine

18:31 <TMM> Maybe worth documenting, I can't imagine I'm the only person running sandstorm like this

18:32 <kentonv> and yes, you can move sandstorm to a new domain without reinstalling. You'd need to edit /opt/sandstorm/sandstorm.conf to change the config there, and then you'll need to use an admin token to reconfigure your login mechanisms (since OAuth login is tied to the domain)

18:32 <TMM> Yeah, this together with some jq will work fine

18:33 <kentonv> oh if you change the `findOne` to just `find` it might be easier to parse the output

18:33 <TMM> My setup is otherwise prooobably pretty odd, but I don't think that matters. I run sandstorm in a systemd nspawn container

18:34 <TMM> But that shouldn't matter for this I think

18:34 <digitalcircuit> kentonv: Low priority update from a week ago or so, ACME account auto-created successfully here. I'll find out tomorrow after renew if the automatic certificate switchover was successful :)

18:34 <TMM> Hmm, it doesn't seem to include the intermediate certs there or maybe I'm just not looking at it right

18:37 <kentonv> TMM, `certChain` should include intermediates

18:37 <kentonv> you should see multiple BEGIN CERTIFICATE / END CERTIFICATE blocks all concatenated together under `certChain`

18:49 <TMM> kentonv: thank you! I'll write a script to extract these into a format nginx can read then

18:50 <TMM> kentonv: and I'm sorry the whole sandstorm thing didn't work out financially :( My company could probably pay for some kind of support contract on a self hosted version if that's something that exists?

18:53 <kentonv> not really. I certainly wouldn't have time to offer reliable support, so can't in good conscience sell support contracts. If you're looking for somewhere to throw money, though, https://github.com/zenhack does a lot of Sandstorm work and accepts sponsorship via github sponsors. :)

18:59 <TMM> Alright! I'll see how I can have my company do sponsorship on GitHub then

18:59 <TMM> My accountant always has... Questions. :P

20:54 <TimMc> TMM: It's good for them. Builds character. :-)

21:00 <isd> I think I'm going to send a pr for the cgroups stuff without actually adding any policy. It will make the backup thing easy to fix.