Guest16621 has quit [Remote host closed the connection]
afuentes has quit [Ping timeout: 240 seconds]
harish has quit [Ping timeout: 268 seconds]
<ill_logic>
I'm now uploading (attempting to, anyway) a 25 gigabyte file though a vagrant-hosted dev sandstorm instance
<ill_logic>
Using a "range request" hackaround.
<ill_logic>
It worked with a 900MB file.
<ill_logic>
But it's huffing and puffing.
<ill_logic>
It shouldn't be taking up a lot of memory, it's chunking it into 5mb parts.
<ill_logic>
I'm hoping it's just because it's a VM. But maybe not? Is this something I shuold be cautious about trying on oasis? (assuming I can even pay for a plan that gives me that much space :-P)
<ill_logic>
I mean it's not so important. My target for this sort of thing is a computer that you have locally.
<ill_logic>
And that's what I have.
<ill_logic>
(localhost)
<ill_logic>
In practice, something on your home network.
<isd>
I vaguely remember there being something about sandstorm not dealing with range requests well.
<ill_logic>
But, I'm curious why it's taking up these resources. I just wonder if it'd make a non-virtualbox version of this also huff and puff the sam eway.
<ill_logic>
Well, the requests themselves are 5mb file chunks.
<ill_logic>
I'm sending thousands of requests.
<isd>
Are you doing them serially?
<ill_logic>
Should be.
<isd>
if each one is on it's own Tcp/TLS connection that's 3 round trips per request. the latency is going to *really* add up
<ill_logic>
Yeah, I'm explicitly failing if the file size doesn't match what the content range says it's starting at.
<isd>
even if you're reusing the tcp/tls connection, you'd be likely to hit tcp stalls.
<ill_logic>
So it must be in serial.
<isd>
Which is going to destroy performance
<ill_logic>
I think jquery-file-uploader sends the chunks in order.
<ill_logic>
and says so
<isd>
I think there was some recent talk on the mailing list re: range-requests.
<ill_logic>
Yeah. When they happen I'll switch to using them for real.
<isd>
Yeah, chunking it that much is going to introduce a lot of overhead.
<ill_logic>
I know it's in the works. I just want to get this project out the door.
<ill_logic>
I think that the issue I'm having is unrelated though. I could be reading the headers. Instead I'm reading post vars.
<isd>
I'd suggest bigger chunks if you can manage them, and parallelize if possible, which will help with the latency. But that's assuming my intuition is correct, which it's possible it isn't.
<ill_logic>
I can probably crank it to about 10. After that it starts failing I think.
<isd>
Why are you chunking it at all, vs. just uploading the whole thing?
<ill_logic>
As you said, Sandstorm doesn't like big uploads.
<isd>
How does it fail exactly?
<ill_logic>
I don't recall.
<ill_logic>
Doing it in parallel sounds like it would be an even bigger load. And I don't mind the latency. This isn't something that the user would do often.
<isd>
So the problem is the load, not that it's slow, then?
<ill_logic>
Right, I'm just concerned that it's making my laptop slow, and the fans turn on.
<isd>
Ah, it's the load on the client.
<ill_logic>
Well the laptop is also running the server.
<ill_logic>
in virtualbox, because I'm developing
<ill_logic>
heh, and I hit the space limit.
<isd>
Ah. is it the client or server that's using cpu?
<ill_logic>
You know I didn't think to check haha
<isd>
ack, I need to head out the door.
<ill_logic>
Okay. Thanks for the suggestion.
<isd>
That would be the first thing to find out.
<isd>
best of luck.
<isd>
later.
harish has joined #sandstorm
Zarutian has quit [Quit: Zarutian]
TMM has quit [Ping timeout: 240 seconds]
TMM has joined #sandstorm
jemc has joined #sandstorm
jemc has quit [Ping timeout: 260 seconds]
ill_logic has quit [Remote host closed the connection]
jemc has joined #sandstorm
ill_logic has joined #sandstorm
ill_logic has quit [Read error: Connection reset by peer]
ill_logic has joined #sandstorm
ill_logic has quit [Read error: Connection reset by peer]
ill_logic has joined #sandstorm
ill_logic has quit [Read error: Connection reset by peer]
jemc has quit [Ping timeout: 240 seconds]
jadewang has quit [Remote host closed the connection]
_whitelogger has joined #sandstorm
jadewang has joined #sandstorm
griff_ has joined #sandstorm
griff_ has quit [Ping timeout: 260 seconds]
dograt_ has quit [Ping timeout: 240 seconds]
dograt has joined #sandstorm
TMM has quit [Quit: Ex-Chat]
harish has quit [Ping timeout: 260 seconds]
pie__ has joined #sandstorm
yeehi_ has joined #sandstorm
yeehi1 has quit [Ping timeout: 260 seconds]
<JonTheNiceGuy>
Does anyone know if there's some kind of CI Best Practice yet for publishing updates to the store?
mrshu has joined #sandstorm
ecloud has quit [Ping timeout: 246 seconds]
cevi has quit [Quit: ZNC 1.6.4+deb1+b1 - http://znc.in]
cevi has joined #sandstorm
afuentes has joined #sandstorm
ecloud has joined #sandstorm
harish has joined #sandstorm
griff_ has joined #sandstorm
leeola has joined #sandstorm
bodisiw has joined #sandstorm
JonTheNiceGuy has quit [Quit: Bridged client killed]
JonTheNiceGuy has joined #sandstorm
samba_ has joined #sandstorm
bodisiw has quit [Quit: This computer has gone to sleep]
bodisiw has joined #sandstorm
bodisiw has quit [Remote host closed the connection]
ill_logic has joined #sandstorm
jemc has joined #sandstorm
samba_ has quit [Ping timeout: 268 seconds]
griff_ has quit [Quit: griff_]
samba_ has joined #sandstorm
FredFredFred_ has joined #sandstorm
FredFredFred has quit [Ping timeout: 268 seconds]
demonimin has quit [Remote host closed the connection]