00:26 <isd> I think if we want the wiki to be navigatable we need to write some overview/indexing pages; the "pages" dropdown is a lost cause

00:26 <isd> Copying & pasting the notes from each meeting into their own wiki page makes sense to me.

00:28 <isd> (I can't even find the main office hours page via "pages")

00:46 <CaptainCalliope> Cools. I'll try and work on it this week. Probably do some quick and dirty organizing and then a second pass a few weeks from now after some stakeholder analysis.

15:09 <ocdtrekkie> I was accompanying someone to the ER most of yesterday, so everything I planned to do this weekend didn't get done.

15:09 <ocdtrekkie> I also have comments on some chat logs here, but I will get to that eventually as well.

17:17 <ocdtr_web> Add-ons would be very interesting. Though I suspect it will be difficult to figure out how to let apps integrate with them? WordPress, Jupyter, etc. all have their own way of handling plugins and add-ons. How much work would be needed to teach them to work with Sandstorm app add-ons?

17:18 <ocdtr_web> michaeln3: I am not sure how bad Davros being 125 MB is today, since a Sandstorm server is unlikely to have more than one or two Davros package versions on hand at a time. Wekan is a 90 MB SPK.

17:18 <ocdtr_web> It is kinda unfortunate that much size comes from just a preview feature for a handful of file formats, though I am guessing there are more formats that could be added using the same libreoffice code?

17:19 <michaeln3> yeah the goal would be to support previewing anything openoffice supports

17:19 <michaeln3> er, libreoffice

17:20 <michaeln3> I don't know if my use-case is a one-off though, in which case maybe it's not worth going too far down the addons path

17:21 <michaeln3> the other motivation (besides size) was that libreoffice presents quite a bit larger attack surface area

17:22 <ocdtr_web> I think unintentionally the first document I tried to test with the new Davros was an RTF file. Which I'm sure libreoffice understands but wasn't specifically allowed with Davros. :P

17:23 <ocdtr_web> It was just something nonsensitive I had on my desktop at the time.

17:23 <michaeln3> ah, oops. yeah I'll work on a comprehensive list of supported types

17:24 <michaeln3> also going to see if converting to pdf and previewing that works better than html, as libreoffice's html output is pretty rough. enough to see what the document says, but the formatting is pretty bad

17:24 <ocdtr_web> Probably the biggest reason I like separating it into another app if possible, is that it'd make it easy for other apps to implement support for previews, presumably.

17:24 <ocdtr_web> Which might be a case for Powerbox over an Add-on.

17:39 <ocdtr_web> Whew, how to debug something that returns 207 as it's HTTP status...

17:41 <ocdtr_web> michaeln3: Curious by the way, have you looked at isd's Sandstorm filesystem demo apps? Ideally if Davros supported that Powerbox interface, everyone else could build apps that used files from Davros grains, rather than building their own file upload, which is only sometimes worthwhile.

17:44 <michaeln3> I saw it existed but haven't gotten a chance to really look into it. I'm on board with the idea that any app that needed/produced files could pull/push them to davros

17:45 <michaeln3> (or anything else that implemented it)

17:47 <ocdtr_web> I think there's probably a lot of cases where it'd still make sense for apps to have their own uploaders/file storage, but it would also be super handy to pick files out of a Davros grain as an option instead of uploading from your own file system, and perhaps in some cases have an app that just depends on Davros as it's file storage.

18:50 <isd> I still need to finish updating that demo. The zip uploader is broken right now.

18:51 <isd> But yes, I think davros should support something like that.

19:04 <ocdtr_web> isd: Arguably the question, I assume, is if there's anything that should be added/removed to that particular capnp definition prior to Davros implementing it. Because once a production app supports it, it's going to be much more difficult to amend.

19:04 <ocdtr_web> But if we're ever going to have a Sandstorm filesystem specification, someone needs to take the leap on defining it, and Davros needs to support it.

19:17 <isd> I think we should kill the symlink stuff; at the time I was thinking of some use cases where it would be handy, but I think kentonv was right that it's a really thorny thing, and maybe not worth the trouble.

19:17 <isd> adding stuff is relatively easy actually. capnp is good for that.

19:19 <DynamicMetaFlow> Greetings, I hope everyone is well, I'm looking into Sandstorm again and trying to access the most recent meeting notes and I'm unable to.

19:19 <DynamicMetaFlow> I'm curious about what's the onboarding process for new contributors, and what type of support is needed.

19:20 <ocdtr_web> isd: Yeah, I know adding new things isn't hard, but ideally we want to make sure everything in there is something we want to support in the manner that it is defined.

19:20 <ocdtr_web> DynamicMetaFlow: Welcome! What issue are you having accessing the notes?

19:21 <DynamicMetaFlow> Thank you, I'm trying to access to Feb 23 shared notes from the following link and I'm unable to https://github.com/sandstorm-io/sandstorm/wiki/Office-Hours

19:21 <ocdtr_web> Hmm, I am wondering if something is wrong with his server, I am getting the same.

19:22 <isd> captaincalliope: want to have a look?

19:22 <ocdtr_web> Just that grain. I wonder if it got corrupted/damaged.

19:22 <DynamicMetaFlow> I dedicated some time reading over the shared notes from past meetings and reading posts on the Google Groups to get an understand of where the project is and how I should direct my learning as it relates to Sandstorm

19:22 <ocdtr_web> Heh, I stopped mirroring the notes on sandstormcommunity.org because I didn't think anyone cared, but now I guess I wish I'd kept up on it.

19:23 <ocdtr_web> CaptainCalliope had talked about mirroring them directly into the GitHub wiki in the future.

19:23 <ocdtr_web> We will recover the notes somehow or another, probably if you check back after tomorrow's meeting.

19:24 <isd> I'm going to add migrating the notes as an agenda item.

19:24 <ocdtr_web> Do you have any particular areas of interest with the project, DynamicMetaFlow? Are you using Sandstorm currently?

19:27 <DynamicMetaFlow> Not at the moment, I've used it in the past. I'm in a fortunate position of where I can allocate time during the day to my learning and contribute to FOSS. I'm interested in exploring that with Sandstorm.

19:29 <ocdtr_web> We have a lot of various areas that need love and affection, so to some degree it is ideal to get an idea where you feel your contribution will be useful. People have had issues building Sandstorm for dev purposes in the past, so I know isd is actively interested in people building Sandstorm and reporting bugs/issues/challenges doing so.

19:30 <isd> It would also be useful to know what your skill set is.

19:31 <DynamicMetaFlow> Thank you, I will spend some time reviewing the open issues and seeing where I can support.

19:32 <ocdtr_web> We have front end work, back end work, packaging new apps or updating old apps is always valuable. And we're happy to point you in the direction of the best issues to look at if we know where you are comfortable working. :)

19:34 <ocdtr_web> Feel free to ask us any questions you have at all.

19:34 <DynamicMetaFlow> Will do, thank you!

19:48 <ocdtr_web> Side note that now that https://help.github.com/en/github/administering-a-repository/managing-teams-and-people-with-access-to-your-repository means you can give people triaging access only, I should see if some folks want me to clean up GitHub issues for some Sandstorm apps.

19:48 <ocdtr_web> It is kind of amazing that it took until 2020 for GitHub to have the ability to do this.

19:50 <ocdtr_web> I kinda wonder if Microsoft had some impact on that, because being able to delegate limited access to groups within an organization is a pretty big part of Microsoft's bread and butter. And it's hard to imagine any Microsoft product being dependent on an all or nothing access system.

19:52 <isd> yeah, they may be pushing things like this more.

19:52 <isd> although there was a long period where GitHub's issue tracker and review tools were frustratingly primitive, and they did start to work on that a while before microsoft bought them.

19:53 <isd> I pushed a commit to the sandstorm-filesystem repo removing the symlink stuff. I'm mostly happy with the rest of the interface, pending feedback from others.

19:54 <isd> This might be the kind of thing where the first app that uses it (probably davros) will shake out some issues and we can address those, maybe making the interface stable once we've got that little bit of experience.

19:57 <ocdtr_web> Above comment about triaging access is because I found an issue I wanted to close on Davros. And I'm aware that michaeln3's time is very limited and probably shouldn't be spent triaging three year old issues.

19:57 <michaeln3> it was on my todo list :)

20:00 <michaeln3> I am curious what people think is the most important priorities for davros. my guess would be client support, but I need to go through the issues and see if there are any other themes

20:02 <ocdtr_web> Some manner of sync support is definitely a big one, though with the issues supporting ownCloud/Nextcloud being what they are, something other than WebDAV might be appealing.

20:02 <ocdtr_web> rsync? https://github.com/mnutt/davros/issues/24

20:05 <isd> At some point I'd like to write a fuse filesystem for the fs protocol I specced out. That would let users (at least on Linux) 'mount' a davros grain, which would be neat. But probably better to focus on more broadly applicable sync mechanisms first.

20:05 <isd> honestly the biggest pain point for me right now is startup times.

20:05 <isd> This is a problem that a ton of apps have, not just davros. and it's not that bad in comparison, but it's long enough to be a bit annoying.

20:06 <CcxWrk> rsync supports arbitrary "shells" which can be anything that spawns remote rsyncd and provides bidirectional reliable channel. That could totally be something on top of websocket or similar.

20:09 <CcxWrk> There is also http://zsync.moria.org.uk/ which is actually designed for dumb file servers.

20:11 <ocdtr_web> Ultimately, the criteria for what Davros supports sync-wise should be pretty straightforward: The protocol should change very rarely or not at all, it should work over HTTPS, and clients should ideally exist for the major platforms.

20:12 <CcxWrk> The last one might be tough if you count mobile platforms.

20:14 <ocdtr_web> https://acrosync.com/ios.html "the only rsync client for iOS"

20:15 <ocdtr_web> So maybe plausible? But I was mostly thinking about desktop OSes at the time.

20:16 <CcxWrk> iOS is rather painful to deal with since Apple store is GPL incompatible. Have fun reimplementing everything.

20:17 <ocdtr_web> Yeah, I don't think we want to get involved in any app store clients where possible.

20:17 <ocdtr_web> Davros used to work with ownCloud clients as a way around that question, TTRSS still works with many existing TTRSS clients on mobile stores, etc.

20:18 <ocdtr_web> Presumably we want to support protocols which already have client software available.

20:18 <CcxWrk> I think zsync is probably the closest to what you want in principle. It requires just two things of the server: run zsyncmake (plain C program) upon files changing and support range HTTP requests.

20:19 <ocdtr_web> Range request support is on the Sandstorm todo list.

20:19 <CcxWrk> The actual diffing is done on the clients. This works well if you have unidirectional sync where you have someone publishing one true version of things and arbitrarily many clients pulling it.

20:20 <CcxWrk> If you are considering data push or bidirectional sync then you need something significantly more complex.

20:21 <CcxWrk> Bidirectional sync is complicated in principle, by the nature of a distributed system.

20:21 <CcxWrk> I think SyncThing would you be your best bet for that currently. But that doesn't run over HTTP.

20:27 <CcxWrk> ocdtr_web: Do you at least have websocket (or some persistent bidirectional stream) support in Sandstorm?

20:28 <michaeln3> isd: yeah, I'd like to figure out what can be sped up

20:29 <michaeln3> davros takes ~250ms to wake up, in sandstorm, up from about 150ms locally

20:31 <isd> CcxWrk: yes, we support websockets.

20:31 <michaeln3> the HTML fetch only spends 5-10ms on the backend, then the rest is mostly in frontend js land

20:33 <michaeln3> unfortunately on my server sandstorm often takes 2-3s before it even starts making davros requests, even when the grain is warm

20:33 <CcxWrk> OK then, writing a rsync shim shell should be feasible. The rrysnc wrapper is already in perl so slapping a WS library in front should not be hard.

20:35 <CcxWrk> But if this is supposed to be about bidi sync they you probably should think through consistency issues first.

20:37 <michaeln3> consistency issues were one of the reasons I got out of the remote file sync business 4 years ago :)

20:37 <CcxWrk> Well, it's not something you can easily patch once the design is out. :]

20:38 <isd> I wonder if we could get unison to talk to davros. I'm not sure what transports it supports (I used to use it over SSH), but if we can solve that: good consistency model, cross platform...

20:38 <CcxWrk> There's also Git-Annex FWIW.

20:38 <CcxWrk> (Which has a bunch of varied backends)

20:42 <michaeln3> yeah I built davros on webdav, ran into consistency issues, wished I had started with syncthing or something, but it would have meant pretty much starting from scratch

20:42 <michaeln3> I like git-annex, not sure if there are mobile clients or how important that is

20:42 <CcxWrk> Kinda https://git-annex.branchable.com/install/Android/

20:44 <CcxWrk> But dealing with Haskell packaging is too much of a "fun" experience for me.

20:45 <michaeln3> it seems like the landscape is sort of split into 1) power user tools that require intentional changes (git-annex, rsync, etc) and 2) consumer tools that can suffer from consistency issues (owncloud, dropbox, etc)

20:47 <CcxWrk> You can have reasonably predictable automatic conflict resolution I believe. As a matter of fact I have drafts of one for Tahoe-LAFS, whenever I get working on that.

20:49 <CcxWrk> But of course it can't magically merge data, you will always end up with multiple versions on a file level. Directories though can be implemented using some sensible CRDT.

20:52 <michaeln3> yeah, versioning was also something I was going to work on at some point but it required some big architectural changes and moving away from a plain file tree backend

20:55 <CcxWrk> BTW your list missed some of the distributed filesystems of old, like AFS/CODA. But those aren't really relevant.

20:56 <ocdtr_web> And I really like the plain file tree backend.

20:56 <ocdtr_web> It's really nice when you need to extract crud from a grain, and "download it and unzip it" is a reasonably trivial way to do it.

20:56 <ocdtr_web> It's why DokuWiki and TextEditor and the like are such beautiful apps.

21:00 <isd> I really think unison is a good option if we can figure out the transport. I might even be up for pushing patches up stream to get it to support websockets or such if need be.

21:00 <isd> Good consistency model, cross platform, GUI...

21:06 <CcxWrk> Last time I checked the only conflict resolution Unison really supported was last write wins. Been years though. Also note that each version is protocol-incompatible. Just so you know.

21:07 <isd> If both copies are modified it gives you a choice of which to keep (or you can not sync that file).

21:07 <isd> But yeah, I'm aware of the protocol issues.

21:11 <CcxWrk> So they finally implemented it? I remember it being documented so but with some note somewhere else saying it's TBD.

21:15 <isd> This what I remember from using it years and years ago.

21:15 <isd> Maybe the docs are (or were) out of date, or maybe you looked a long time ago.

21:16 <CcxWrk> OK then. I did try it probably several years before you. :] Has been long time.

22:23 <CaptainCalliope> <DynamicMetaFlow "Thank you, I'm trying to access "> Huh. I'm having no issue accessing these notes. I wonder what the issue was/is for you.

22:23 <CaptainCalliope> Anyways, I have a little bit of time now. Gonna move the notes over to the wiki.

22:43 <DynamicMetaFlow> CaptainCalliope, Thank you for moving the notes over to the wiki

23:20 <CaptainCalliope> Done, I'm also revising the sidebar a little.

23:22 <CaptainCalliope> kentonv: Did this ever happen? [Spring 2015 Sandstorm Usability Research](https://github.com/sandstorm-io/sandstorm/wiki/Spring-2015-Sandstorm-Usability-Research)