pipework has quit [Remote host closed the connection]
jfoy has joined #rubygems
havenwood has quit [Remote host closed the connection]
cowboyd has joined #rubygems
jcaudle has joined #rubygems
jcaudle has left #rubygems [#rubygems]
x1337807x has joined #rubygems
DanKnox_away is now known as DanKnox
reset has quit [Quit: Leaving...]
jfoy has quit [Quit: jfoy]
abuiles has joined #rubygems
jfoy has joined #rubygems
reset has joined #rubygems
cowboyd has quit [Remote host closed the connection]
jfoy has quit [Quit: jfoy]
reset has quit [Ping timeout: 256 seconds]
x1337807x has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
dvu has joined #rubygems
havenwood has joined #rubygems
dvu has quit [Ping timeout: 240 seconds]
yerhot has quit [Remote host closed the connection]
havenwood has quit [Remote host closed the connection]
havenwood has joined #rubygems
havenwood has quit [Remote host closed the connection]
havenwood has joined #rubygems
yerhot has joined #rubygems
kgrz has joined #rubygems
yerhot has quit [Ping timeout: 240 seconds]
havenwood has quit [Remote host closed the connection]
xymox has quit [Ping timeout: 246 seconds]
xymox has joined #rubygems
kgrz has quit [Remote host closed the connection]
huoxito has quit [Quit: Leaving]
kgrz has joined #rubygems
jstr has quit [Quit: Computer has gone to sleep.]
le_gars has joined #rubygems
jstr has joined #rubygems
dmison has quit [Ping timeout: 245 seconds]
dmison has joined #rubygems
kgrz_ has joined #rubygems
kgrz has quit [Ping timeout: 248 seconds]
DanKnox is now known as DanKnox_away
DanKnox_away is now known as DanKnox
kgrz_ has quit [Remote host closed the connection]
kgrz has joined #rubygems
DanKnox is now known as DanKnox_away
jstr has quit [Quit: Computer has gone to sleep.]
havenwood has joined #rubygems
kgrz_ has joined #rubygems
kgrz has quit [Ping timeout: 240 seconds]
workmad3 has joined #rubygems
jstr has joined #rubygems
kgrz has joined #rubygems
kgrz_ has quit [Ping timeout: 256 seconds]
workmad3 has quit [Ping timeout: 248 seconds]
jstr has quit [Quit: Computer has gone to sleep.]
stevenharman has quit [Read error: Connection reset by peer]
xerxas has quit [Ping timeout: 248 seconds]
kgrz has quit [Remote host closed the connection]
indirect has quit [Ping timeout: 256 seconds]
havenn_ has joined #rubygems
indirect has joined #rubygems
havenwood has quit [Ping timeout: 240 seconds]
havenn_ has quit [Remote host closed the connection]
le_gars has quit [Remote host closed the connection]
beli has joined #rubygems
<beli>
hi there. where to find a list of rubygems mirrors?
le_gars has joined #rubygems
kgrz has joined #rubygems
kgrz has quit [Remote host closed the connection]
karmi has joined #rubygems
kgrz has joined #rubygems
dmison has quit [Quit: Leaving...]
xerxas has joined #rubygems
lsegal has quit [Quit: Quit: Quit: Quit: Stack Overflow.]
sferik has joined #rubygems
kgrz has quit [Remote host closed the connection]
dmison has joined #rubygems
kgrz has joined #rubygems
charliesome has joined #rubygems
kgrz has quit [Ping timeout: 248 seconds]
khaase- is now known as khaase
khaase has quit [Changing host]
khaase has joined #rubygems
le_gars has quit [Remote host closed the connection]
khaase has left #rubygems ["Leaving..."]
Elhu has joined #rubygems
sferik has quit [Quit: Computer has gone to sleep.]
le_gars has joined #rubygems
dmison has quit [Quit: Leaving...]
karmi has quit [Quit: Leaving.]
le_gars has quit [Ping timeout: 248 seconds]
karmi has joined #rubygems
kgrz has joined #rubygems
le_gars has joined #rubygems
kgrz has quit [Ping timeout: 264 seconds]
dmison has joined #rubygems
dmison has quit [Ping timeout: 248 seconds]
Elhu has quit [Quit: Computer has gone to sleep.]
tcopeland has quit [Quit: Leaving.]
jstr has joined #rubygems
karmi has quit [Quit: Leaving.]
bbrowning has joined #rubygems
dmison has joined #rubygems
karmi has joined #rubygems
le_gars has quit [Remote host closed the connection]
le_gars has joined #rubygems
huoxito has joined #rubygems
karmi has quit [Quit: Leaving.]
beli has quit [Quit: Page closed]
jstr has quit [Quit: Computer has gone to sleep.]
kgrz has joined #rubygems
kgrz has quit [Ping timeout: 248 seconds]
yerhot has joined #rubygems
paulca has joined #rubygems
karmi has joined #rubygems
tekin has joined #rubygems
iamjarvo_ has joined #rubygems
dvu has joined #rubygems
Elhu has joined #rubygems
iamjarvo_ has quit [Client Quit]
yerhot has quit [Read error: Connection reset by peer]
yerhot has joined #rubygems
cowboyd has joined #rubygems
tcopeland has joined #rubygems
dvu has quit [Remote host closed the connection]
fromonesrc has joined #rubygems
dvu has joined #rubygems
fromonesrc_ has joined #rubygems
fromonesrc has quit [Ping timeout: 256 seconds]
samkottler has quit [Quit: leaving]
iamjarvo has joined #rubygems
stevenharman has joined #rubygems
whit537 has joined #rubygems
kgrz has joined #rubygems
kgrz has quit [Ping timeout: 245 seconds]
therealadam has joined #rubygems
le_gars has quit [Remote host closed the connection]
tbuehlmann has joined #rubygems
cowboyd has quit [Remote host closed the connection]
tcgden1 has joined #rubygems
iamjarvo has quit [Ping timeout: 246 seconds]
iamjarvo has joined #rubygems
catphish has joined #rubygems
cowboyd has joined #rubygems
<catphish>
qrush: are you around?
pipework has joined #rubygems
<qrush>
hi
<catphish>
do you have a moment to explain the http infrastructure that runs rubygems.org?
<catphish>
i had a very odd issue a couple of weeks back where rubygems.org was failing to respond to requests from one of my networks and i'm trying to get to the bottom of it
<qrush>
i'm not the right person to ask :S
<qrush>
it's all OSS now though, at github.com/rubygems/rubygems-aws
<catphish>
ah, i believed you were, who does the servers?
<catphish>
the OSS part doesn't explain the HTTP connectivity sadly, i did check it out at the time
<qrush>
skottler, vertis, dwradcliffe, evan would be best to ask
<catphish>
thanks
karmi has quit [Quit: Leaving.]
<evan>
hiya
<evan>
whats your question?
<evan>
my backlog doesn't show it
<catphish>
evan: i'm interested to know what the rubygems.org infrastructure is (in particular how HTTP requests are processed)
<catphish>
i had a strange issue a couple of weeks ago where http requests from one of my networks were ignored
<catphish>
and today it's happening again, but this time with flickr, and i'm really keen to get to the bottom of it
paulca has quit [Quit: paulca]
<evan>
ok
<evan>
well, HTTP goes from you to AWS US-West
<evan>
hits a server running nginx
<catphish>
yep, i was able to hit the nginx server during the failure
<catphish>
and get 400 errors if i sent invalid requests
<evan>
it then splits up where the request goes
<catphish>
does the response get proxied back through nginx?
<evan>
for most requests, they go get proxied to unicorn running no another EC2 server
<evan>
yes
<evan>
what was the 400 you got?
iamjarvo has quit [Read error: Operation timed out]
<catphish>
it's very strange, when i had trouble i could always get 400 responses to bad requests
<catphish>
but good requests were met with silence
<catphish>
but only from one of my networks
seacreat1re has quit [Remote host closed the connection]
<evan>
what was the body of that 400 response?
<evan>
if it happens again, i can always check the server logs
<evan>
so we can figure out whats going on
<catphish>
the 400 responses were basically "400 bad request nginx/x.x.x"
<catphish>
afraid i didn't record that
<catphish>
i assume nothing at the backend should respond differently depending on source ip?
iamjarvo has joined #rubygems
<evan>
thats correct
<evan>
well, it depends on the URL slightly
<evan>
if you hit the url for a .gem file
<evan>
we do a redirect
<evan>
that is based on the source ip
<evan>
to try and redirect you to the closest merror
<evan>
mirror
<catphish>
interesting, this was failing for the indexes and homepage
<catphish>
right now i'm experiencing the same issue with flickr
<catphish>
they respond to HEAD requests but not GET from some of my IPS
<evan>
almost sounds like there is a transparent proxy on your network
<catphish>
maybe an MTU problem actually
<evan>
that is malfunctioning
<catphish>
there's definitely no proxy :)
<catphish>
i test from a border router connected to multiple transits
<catphish>
well sadly its not affecting rubygems right now, so there's nothing i can ask you do do, thanks for the help though
<evan>
no prob
<catphish>
evan: do you keep logs a while?
<evan>
typically
<catphish>
if it's not too much trouble could you grep them for 185.22.208.1 and see if any errors occurred?
<catphish>
i can give you a time range where it happened if that is helpful
<catphish>
if you can't be bothered that's fine too, i'm just baffled
<catphish>
head requests from both IPs work (with a 405), but GET requests from 185.22.208.1 just timed out
<evan>
do a get
<catphish>
that's the last request
<evan>
ok
<evan>
i'll look
<catphish>
awesome, thanks
karmi has joined #rubygems
<evan>
the last entry for that ip is: 185.22.208.1 rubygems.org - [03/Jul/2013:14:45:07 +0000] "GET / HTTP/1.1" 200 9207 "-" "curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3" "-"
<catphish>
it did work eventually
<catphish>
after 6+ hours of failure
<evan>
it's 15:19 on that server
<evan>
there's nothing in the error log for it.
<catphish>
ok, the failure was shortly after: Date: Sun, 16 Jun 2013 16:51:12 GMT
<catphish>
oh that log is [03/Jul/2013 (today
<catphish>
the issue was at 16 Jun 2013 16:51:12 GMT
<evan>
yeah, thats today
<catphish>
as i said it's not occurring now, but wondered if there was anything in the historical lot
<catphish>
*log
huoxito has quit [Quit: Leaving]
<evan>
I don't have logs back that far
<evan>
we made a change a while back to how often the logs are rotated