<TimMc>
I tried to send some mildly sensitive documents to my credit union by using Davros, and they said it didn't load. (They got the "stay anonymous" prompt, so I suspect the grain was still warming up and they had a white screen under the shell.)
<TimMc>
But then they tried to get me to use their "secure mail" web service, and they're not receiving the docs that way, which I guess makes me feel a little better. :-P
<TimMc>
Hiccups aside, it has been really nice to be able to email people links to Davros grains rather than sticking the files in the (really nonsecure) email.
<isd>
I'm glad it's been useful at least.
<isd>
I really wish more of our core apps were faster to start.
<isd>
mnutt: I feel like way back there was some discusion of davros boot times on the mailing list and you did some investigation, but I forget where that went. Do you know where it's spending its time?
<xet7>
isd: "more of our core apps were faster to start" :D You mean Wekan? :D
<mnutt>
some of it came down to node app initialization in the fs; I did some work to reduce it via caching require paths but there may be more there to do
<mnutt>
I'm curious what sorts of davros start times you're seeing as logged to stdout ("Davros started in XXXms") vs observed time, if that's a lot longer
<mnutt>
I haven't followed sandstorm internals recently, is there any penalty at all for large spk sizes? davros's is particularly large due to the openoffice dependency
<xet7>
I have made some tests with alpha-minimal version of Wekan webpages, they do load instantly. But that's just with very minimal amount of features. I presume I need to continue developing minimal version, because many run into speed issues with Wekan.
<isd>
spk size shouldn't matter; it just gets extracted to the local file system and then bind mounted when the app starts, so the actual contents of the filesystem are irrelevant.
<xet7>
But full Wekan versions if about 60k files of HTML/CSS/JS/images etc, it's kind of heavy currently
<isd>
(obviously actually needing to hit the disk a bunch during startup would matter though. I wonder if using webpack or rollup on the server would speed anything up by reducing the number of files that need to be read...)
<xet7>
But full Wekan version of about 60k files of HTML/CSS/JS/images etc, it's kind of heavy currently
<xet7>
mnutt: Thanks, I'll try does it work with Wekan
<mnutt>
I found that starting the application under strace and watching the syscalls (particularly the filesystem ones) was enlightening; there were a number of things getting loaded I was able to defer or lazily load
<xet7>
Wekan could really benefit from optimization
<xet7>
If I knew how to do it
<xet7>
And very likely in Wekan's 60k files there is a lot that is not really needed
<isd>
Time to research profilers?
<xet7>
Yes
<xet7>
I'll research
<isd>
This is the other thing -- the bottlenecks are almost never where you'd expect them to be.
<TimMc>
Ooooh, I got a screenshot of the loading failure from the person at the credit union... and they're definitely using some version of Internet Explorer. :-/
<isd>
Tangentially related, I did a bunch of tuning for haskell-capnp in the past couple days, got a 6x speedup on basic file upload for https://github.com/zenhack/ocap-merkledag
<xet7>
BTW, that minimal Wekan webpage version does how Wekan all boards page in Netsurf and Internet Explorer
<isd>
That could be the issue. I don't know exactly whether we're doing stuff that specifically doesn't work on old ie, but we're definitely not testing with it.
<xet7>
Maybe with Netsurf and Internet Explorer support I could recommend Wekan to old organizations, and Amiga/RISC OS etc users
<xet7>
Maybe support links/elinks also?
<isd>
links would mean no client-side js at all.
<xet7>
Yes, that Wekan minimal webpage version does not have any Javascript or SVG.
<xet7>
It's just Wekan webpage saved to disk, Javascript removed, only HTML+CSS, with a little modifications.
<xet7>
For translation, PHP loads translation JSON file from disk, and replaces text on that webpage, to make translated webpage
<xet7>
I have not added database support yet
<xet7>
It's just a test
<xet7>
Also, in Netsurf drag-drop is not possible, I think. So instead, it could be having form with checkboxes, and then move all checkboxed cards to other list, etc
<xet7>
for multiselect
<xet7>
old school style
<xet7>
For CSS, for each page I could add only that CSS that is used at that page, not needing all the combined CSS
<xet7>
But, updating webpage would need reload webpage, at least in Netsurf. I could look what minimal Netsurf Javascript support has, can that be used for something.
<xet7>
$ ./main-linux
<xet7>
pkg/prelude/bootstrap.js:1244
<xet7>
throw error;
<xet7>
^
<xet7>
RangeError: Maximum call stack size exceeded
<xet7>
at Object.join (path.js:1033:7)
<xet7>
What does that pkg error mean?
<xet7>
That Wekan is too big?
<xet7>
Hmm, I try with start script
<isd>
Looks like you're getting a stack overflow (recursion is too deep)
<xet7>
Is it possible to increase recursion depth?
<isd>
Don't know. But, depending on the issue, that may not help -- usually a stack overflow means you've got infinite recursion somewhere.
<xet7>
I have also hit limits of MongoDB: restoring my 2.5 GB mongdump with mongorestore goes very slowly and timeouts, on Qubes OS VM. So only restoring by copying raw MongoDB database files is possible.
<xet7>
I exported my MongoDB database to JSON. 2 biggest files: attachments.chunks.json 1.9 GB, tempstore.chunks.json 1.6 GB. Next biggest is 22 MB.
<JacobWeisz[m]>
I feel like storing attachments in the DB is a bad idea.
<xet7>
Well, some years ago I did see some other app using about 10 000 files on Windows. That's kind of too much for Windows filesystem.
<isd>
Eh. unless it's actually causing problems, not having to ad-hoc your storage solution is a win from a software engineering perspective.
<xet7>
It was hard to list files in directory
<xet7>
I'm currently trying to export files from GridFS to SQLite
<isd>
...the ui code for the grain list and the way it interacts with the bulk action buttons is so convoluted. I'm trying to figure out a reaosonable way to put a status indicator for backups in there somewhere without just making it even messier, and it's... hard.
<xet7>
I feel stupid trying to list files in MongoDB with code. Most of examples just do not work. I'm much more used to VBA and SQL.
<xet7>
Compiling MongoDB C driver...
<isd>
I have this fantasy of switching sandstorm over to postgres or something someday.