00:31
jfoy has joined #rubygems
00:40
jfoy has quit [Remote host closed the connection]
00:41
jfoy has joined #rubygems
00:55
dvu has joined #rubygems
01:54
adkron has quit [Ping timeout: 240 seconds]
02:02
adkron has joined #rubygems
02:15
jfoy has quit [Ping timeout: 245 seconds]
03:12
havenwood has joined #rubygems
03:16
jfoy has joined #rubygems
03:16
havenwood has quit [Remote host closed the connection]
03:30
jfoy has quit [Ping timeout: 240 seconds]
03:57
havenwood has joined #rubygems
04:00
whit537 has quit [Quit: whit537]
04:28
kgrz has joined #rubygems
04:52
x1337807x has joined #rubygems
04:58
kgrz has quit [Ping timeout: 256 seconds]
05:01
kseifried has joined #rubygems
05:01
kseifried has joined #rubygems
05:04
pipework has joined #rubygems
05:24
huoxito has quit [Quit: Leaving]
05:27
huoxito has joined #rubygems
05:29
dvu has quit [Remote host closed the connection]
05:30
dvu has joined #rubygems
05:32
huoxito has quit [Quit: Leaving]
05:34
adkron has quit [Ping timeout: 264 seconds]
05:35
dvu has quit [Ping timeout: 268 seconds]
05:57
tbuehlmann has joined #rubygems
06:57
pipework has quit [Remote host closed the connection]
07:14
zzak_ has joined #rubygems
07:15
lmarburger_ has joined #rubygems
07:15
weeb1e has joined #rubygems
07:16
lmarburger has quit [Ping timeout: 264 seconds]
07:16
weeb1e_ has quit [Read error: Connection reset by peer]
07:16
zzak has quit [Read error: Connection reset by peer]
07:16
foohey has quit [Ping timeout: 264 seconds]
07:16
lmarburger_ is now known as lmarburger
07:16
foohey has joined #rubygems
07:16
Rennex has quit [Ping timeout: 264 seconds]
07:16
dfgg has quit [Ping timeout: 264 seconds]
07:16
dfgg has joined #rubygems
07:16
Rennex has joined #rubygems
07:23
dangerousdave has joined #rubygems
07:46
havenwood has quit [Remote host closed the connection]
07:46
havenwood has joined #rubygems
07:49
tenderlove has quit [Remote host closed the connection]
08:06
dangerousdave has quit [Read error: Connection reset by peer]
08:07
dangerousdave has joined #rubygems
08:19
tenderlove has joined #rubygems
08:28
tenderlove has quit [Ping timeout: 260 seconds]
08:55
tenderlove has joined #rubygems
08:59
tenderlove has quit [Ping timeout: 240 seconds]
09:06
havenwood has quit [Remote host closed the connection]
09:17
dmilith has quit [Ping timeout: 260 seconds]
09:21
dmilith has joined #rubygems
09:25
dangerousdave has quit [Read error: Connection reset by peer]
09:26
dangerousdave has joined #rubygems
09:26
tenderlove has joined #rubygems
09:30
tenderlove has quit [Ping timeout: 268 seconds]
09:36
dmilith has quit [Ping timeout: 256 seconds]
09:47
mikewintermute has joined #rubygems
09:57
tenderlove has joined #rubygems
10:00
stayarrr has joined #rubygems
10:07
tenderlove has quit [Ping timeout: 268 seconds]
10:07
havenwood has joined #rubygems
10:10
dangerousdave has quit [Read error: Connection reset by peer]
10:10
havenwood has quit [Remote host closed the connection]
10:10
Dave has joined #rubygems
10:10
havenwood has joined #rubygems
10:10
Dave is now known as Guest78738
10:31
havenwood has quit [Remote host closed the connection]
10:32
tenderlove has joined #rubygems
10:36
Guest78738 has quit [Quit: Leaving...]
10:37
tenderlove has quit [Ping timeout: 246 seconds]
12:04
tenderlove has joined #rubygems
12:07
mikewintermute has quit [Quit: mikewintermute]
12:09
tenderlove has quit [Ping timeout: 248 seconds]
12:22
tbuehlmann has quit [Remote host closed the connection]
12:32
dmilith has joined #rubygems
12:45
adkron has joined #rubygems
13:11
Aaaarg has left #rubygems [#rubygems]
13:36
tenderlove has joined #rubygems
13:41
tenderlove has quit [Ping timeout: 256 seconds]
14:07
tenderlove has joined #rubygems
14:12
tenderlove has quit [Ping timeout: 268 seconds]
14:13
Aaaarg has joined #rubygems
14:14
Aaaarg has left #rubygems [#rubygems]
14:38
tenderlove has joined #rubygems
14:40
adkron has quit [Ping timeout: 252 seconds]
14:42
tenderlove has quit [Ping timeout: 256 seconds]
14:54
vertis1 has joined #rubygems
14:57
vertis has quit [Ping timeout: 276 seconds]
15:09
tenderlove has joined #rubygems
15:13
tenderlove has quit [Ping timeout: 252 seconds]
15:39
tenderlove has joined #rubygems
15:44
tenderlove has quit [Ping timeout: 248 seconds]
16:01
havenwood has joined #rubygems
16:22
tbuehlmann has joined #rubygems
16:41
tenderlove has joined #rubygems
16:46
tenderlove has quit [Ping timeout: 260 seconds]
16:53
catphish has joined #rubygems
16:54
<
catphish >
i'm having trouble with rubygems not processing requests from my IP addresses :(
16:54
<
catphish >
are there any known problems, or does rubygems do any type of reverse lookup on IPs that could be hanging?
17:08
whit537 has joined #rubygems
17:09
whit537 has quit [Client Quit]
17:12
tenderlove has joined #rubygems
17:14
<
catphish >
qrush: are you around?
17:17
tenderlove has quit [Ping timeout: 276 seconds]
17:42
tenderlove has joined #rubygems
17:47
tenderlove has quit [Ping timeout: 248 seconds]
17:55
havenwood has left #rubygems [#rubygems]
18:43
tenderlove has joined #rubygems
18:47
tenderlove has quit [Ping timeout: 256 seconds]
19:01
catphish has quit [Ping timeout: 256 seconds]
19:14
tenderlove has joined #rubygems
19:17
catphish has joined #rubygems
19:18
tenderlove has quit [Ping timeout: 240 seconds]
19:49
tenderlove has joined #rubygems
19:49
catphish has quit [Ping timeout: 256 seconds]
19:56
tbuehlmann has quit [Remote host closed the connection]
19:58
martinisoft has joined #rubygems
19:59
catphish has joined #rubygems
20:06
workmad3 has joined #rubygems
20:34
stayarrr has quit [Quit: Leaving...]
21:09
workmad3 has quit [Ping timeout: 245 seconds]
21:16
adkron has joined #rubygems
21:20
catphish has quit [Ping timeout: 256 seconds]
21:21
catphish has joined #rubygems
21:27
workmad3 has joined #rubygems
21:57
<
catphish >
any staff around?
22:00
workmad3 has quit [Ping timeout: 240 seconds]
22:00
<
raggi >
catphish: i might be able to help if it's a client side problem, but i have no access to fix things
22:01
<
raggi >
catphish: what does your output look like wiht -V ?
22:04
<
catphish >
raggi: thanks, i'm fairly sure it's a server side issue, but i'll paste some output
22:04
<
catphish >
then it just hangs forever
22:09
<
catphish >
it's as if a proxy / loadbalancer is accepting my connection initially, then passing it to another server that can't respond to me, or perhaps the app does some kind of reverse lookup on the address, which never finishes
22:14
workmad3 has joined #rubygems
22:22
vertis1 has quit [Quit: Leaving.]
22:23
workmad3 has quit [Ping timeout: 240 seconds]
22:24
mikewintermute has joined #rubygems
22:28
adkron has quit [Ping timeout: 252 seconds]
22:29
mikewintermute has quit [Quit: mikewintermute]
22:36
<
raggi >
catphish: what does traceroute for 54.245.255.174 say?
22:38
lsegal` has joined #rubygems
22:39
lsegal` has quit [Client Quit]
22:39
lsegal has joined #rubygems
22:40
<
raggi >
looks right
22:41
<
catphish >
not sure why it gets lost at the end, but i'm certain i can reach 54.245.255.174
22:42
<
catphish >
it accepts the http connection, but just doesn't respond to requests
22:42
<
catphish >
do you know if it's loadbalanced in some way?
22:43
<
catphish >
another interesting fact is that if i hit it with an HTTP method it doesn't like (for some reason it doesn't like HEAD), it responds wth a 405 Not Allowed
22:43
<
raggi >
yes, that's ELB
22:44
<
catphish >
so i can get as far as nginx
22:44
<
catphish >
but it i make a valid GET request from that network it hangs
22:44
<
catphish >
so my guess: the loadbalancer (nginx) has a route to my network, but the app servers don't :(
22:45
<
catphish >
this only started happening today
22:45
<
raggi >
can you drop a basic request into a telnet session?
22:46
<
raggi >
how comes your curl is outputting headers wihtout -D-?
22:47
<
catphish >
curl -I does a HEAD request, which would only return headers
22:47
<
catphish >
so it prints them by default
22:48
<
catphish >
i can telnet in, but if i issue a GET / HTTP/1.1 it still hangs
22:50
<
raggi >
i'd suggesting checking all the regular things, firewall rules, make sure you don't have proxies etc
22:52
<
catphish >
the host i was testing from is a border router with no firewall, so i'm pretty confident it's not my end
22:53
<
catphish >
and my connection to the http endpoint is ok as i can get 405 responses
22:53
<
raggi >
right, all the load balancing is done at that point
22:54
<
raggi >
if there was an app problem from there, it would be systemic - affecting everyone
22:54
<
catphish >
so it's definitely the app, but it only affects me :(
22:55
<
catphish >
i realise that's incredibly unlikely
22:56
<
catphish >
and it's fixed
22:57
<
catphish >
after about 7 hours it suddenly works
22:57
<
catphish >
well that's baffling but great
22:58
<
raggi >
i've had bad ELBs before, but not with this kind of pattern, but more than that, you were hitting the same ELB as everyone else
22:58
<
raggi >
(i checked from UK, east coast, and west coast USA)
22:59
<
catphish >
since i was able to get nginx errors, i assume the ELB is OK
22:59
<
catphish >
and it appeared the app was hanging when it saw my IP, which is obviously incredibly unlikely
22:59
dmison has joined #rubygems
23:00
<
catphish >
my IP range is quite new, so i do see occasional routing problems
23:00
<
catphish >
good to know
23:01
<
catphish >
the app looks to simple to be the cause of the problem too
23:01
<
raggi >
it's not the app
23:02
<
raggi >
it is a routing or filtering problem
23:02
<
raggi >
as i say, i'd be willing to suspect ELB, as they have been suffering recently, but, that would affect otehrs
23:02
<
catphish >
yet the app server could respond with 405
23:02
<
raggi >
or you'd be shifted to a different host, if there was "an event" ongoing
23:03
<
catphish >
unless ELB sends invalid requests to a different host, or knows how to respond to invalid requests without going to an app server
23:03
<
raggi >
catphish: session loss due to routing problems often results in short messages making it back
23:04
<
catphish >
well that would certainly explain this
23:04
<
catphish >
could be a MTU issue on a backup route i guess
23:04
<
catphish >
that would definitely cause it if that's a known issue resulting from routing changes in amazon
23:05
<
raggi >
the level3 transatlantic was a pain in my ass for years
23:05
<
raggi >
part of the reason i will never take a job in netops
23:05
<
raggi >
but it wasn't those routes in this case
23:05
<
catphish >
what do you do? you seem to know networks :)
23:07
<
raggi >
haha, i've forgotten a lot of stuff about it
23:08
<
raggi >
i'm technically a staff software engineer
23:08
<
raggi >
in reality, i support 7 development teams at google
23:08
<
catphish >
i'm a rails developer and a server engineer and a network engineer (small team)
23:12
havenwood has joined #rubygems
23:15
havenn_ has joined #rubygems
23:15
havenwood has quit [Read error: Connection reset by peer]
23:23
<
catphish >
sleep now, thanks :)
23:23
catphish has quit [Quit: Leaving]
23:56
vertis has joined #rubygems