huoxito has quit [Remote host closed the connection]
huoxito has joined #rubygems
huoxito has quit [Remote host closed the connection]
huoxito has joined #rubygems
jkline has quit [Quit: jkline]
TheComrade has quit [Quit: TheComrade]
redmenace has quit [Ping timeout: 265 seconds]
lsegal has joined #rubygems
<vertis>
tmoore thanks, I'll check it and deploy it as soon as I finish work
<drbrain>
imperator: thanks!
<imperator>
drbrain, were you able to duplicate it?
<drbrain>
imperator: I haven't tried in ages
tjbiddle has joined #rubygems
redmenace has joined #rubygems
huoxito has quit [Remote host closed the connection]
huoxito has joined #rubygems
redmenace has quit [Ping timeout: 246 seconds]
huoxito has quit [Read error: Connection reset by peer]
huoxito has joined #rubygems
havenwood has joined #rubygems
huoxito has quit [Remote host closed the connection]
huoxito_ has joined #rubygems
DanKnox is now known as DanKnox_away
TheComrade has joined #rubygems
tmoore has quit [Quit: Page closed]
arthurnn has quit [Remote host closed the connection]
arthurnn has joined #rubygems
DanKnox_away is now known as DanKnox
tjbiddle has quit [Quit: tjbiddle]
arthurnn has quit [Ping timeout: 248 seconds]
imperator has left #rubygems ["Leaving"]
stevenharman has joined #rubygems
stevenharman has quit [Client Quit]
havenwood has quit [Remote host closed the connection]
jalessio has joined #rubygems
jalessio has quit [Client Quit]
jalessio has joined #rubygems
havenwood has joined #rubygems
huoxito_ has quit [Remote host closed the connection]
huoxito has joined #rubygems
havenwood has quit [Ping timeout: 272 seconds]
huoxito has quit [Ping timeout: 265 seconds]
arthurnn has joined #rubygems
arthurnn has quit [Ping timeout: 272 seconds]
Elhu has joined #rubygems
graphex_ has quit [Ping timeout: 248 seconds]
tbuehlmann has joined #rubygems
graphex has joined #rubygems
vertis has quit [Quit: vertis]
graphex has quit [Read error: Connection reset by peer]
jalessio has quit [Quit: Leaving.]
Elhu has quit [Quit: Computer has gone to sleep.]
lsegal has quit [Read error: Connection reset by peer]
lsegal has joined #rubygems
DanKnox is now known as DanKnox_away
DanKnox_away is now known as DanKnox
workmad3 has joined #rubygems
lsegal has quit [Read error: Connection reset by peer]
lsegal has joined #rubygems
DanKnox is now known as DanKnox_away
Elhu has joined #rubygems
workmad3 has quit [Ping timeout: 252 seconds]
lsegal has quit [Quit: Quit: Quit: Quit: Stack Overflow.]
roidrage has joined #rubygems
roidrage has quit [Client Quit]
jigfox has joined #rubygems
adambeynon has joined #rubygems
vertis has joined #rubygems
workmad3 has joined #rubygems
jigfox has quit [Remote host closed the connection]
tbuehlmann has quit [Ping timeout: 240 seconds]
tbuehlmann has joined #rubygems
<mpapis>
drbrain, #677 I would ask headius if he want's to keep it open, I'm interested in it as tool that manages ruby - to install gems that do extra things - but then rubygems_plugin.rb from multiple gems would colide if merged multiple gems in one location
Elhu has quit [Quit: Computer has gone to sleep.]
tcopeland has quit [Quit: Leaving.]
mnufweli has joined #rubygems
tbuehlmann has quit [Ping timeout: 272 seconds]
huoxito has joined #rubygems
huoxito has quit [Remote host closed the connection]
<jnimety>
OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server hello A: wrong version number
<jnimety>
Gem::RemoteFetcher::FetchError: Errno::ETIMEDOUT: Connection timed out - connect(2)
<jnimety>
Bundler::Fetcher::CertificateFailureError: Could not verify the SSL certificate for https://rubygems.org/.
<roidrage>
samkottler: our current plague is the first one in the list
<samkottler>
roidrage: are you forcing tlsv1.2?
<roidrage>
samkottler: how would i do that?
<samkottler>
roidrage: you'd know if you were doing it
<samkottler>
roidrage: but that top issue appears to be an issue with your CA bundle
ZachBeta has joined #rubygems
graphex has joined #rubygems
<jnimety>
samkottler, roidrage I spent too many hours last week debugging that issue on fedora 18 servers. we had the latest ca bundle. the solution was to force reinstall the ca bundle
<jnimety>
on my mac I still get that error intermittently, even after running the rvm ca update command
<roidrage>
samkottler: the baffling bit is that this error came up today
<roidrage>
we haven't seen this error before
<roidrage>
at least not in a way that we'd get an influx of reports
<samkottler>
roidrage: did you upgrade machines recently?
<roidrage>
samkottler: nope
<samkottler>
the weird thing about this bug is that no one can consistently reproduce it, which makes fixing it borderline impossible
<roidrage>
yup :)
<roidrage>
i feel you, big time
<samkottler>
like I can't even say that moving away from nginx altogether or tweaking our SSL config will fix stuff
<samkottler>
because I can't test
stevenharman has joined #rubygems
<roidrage>
yup, the unfortunate nature of these types of bugs
<samkottler>
roidrage: if you have a machine you can give me access to that consistently has the problem that'd be a massive help in getting it fixed
<roidrage>
samkottler: i'll have a look, i'm afraid it won't be anything consistent
<samkottler>
roidrage: okay, np. if you can email me details about the machines (shk [at] redhat.com) and their ca-certificates version that'd be helpful, too
roidrage has quit [Read error: Connection reset by peer]
roidrage has joined #rubygems
<roidrage>
lmao broken hotel wifi
jnimety has quit [Quit: Computer has gone to sleep.]
roidrage has quit [Read error: Connection reset by peer]
roidrage has joined #rubygems
rafaelfr_ has joined #rubygems
roidrage has quit [Read error: Connection reset by peer]
roidrage has joined #rubygems
ZachBeta has quit [Quit: Computer has gone to sleep.]
<roidrage>
trying to get the problem with a bigger bundle
<roidrage>
samkottler: to come back to one of the questions, how would i force bundler to use tls?
jnimety has joined #rubygems
rafaelfranca has quit [Ping timeout: 245 seconds]
<dwradcliffe>
morning samkottler
einarj has joined #rubygems
ZachBeta has joined #rubygems
<roidrage>
jnimety: wow, there are really all three errors in one build, sorry :(
<jnimety>
roidrage: not a problem. I'm seeing it everywhere and just want to get to the bottom of it (for the greater good!)
<roidrage>
yeah, i'd love to figure it out too
<jnimety>
I'm planning on implementing some timeout and ssl retry logic tonight at our local ruby meetup. Hopefully that will help curb frustration. Still need to find root cause though.
<jnimety>
*in rubygems
<roidrage>
there is a retry patch for bundler
<roidrage>
not sure if it's made it into master yet
<jnimety>
cool, do you have a link? If nothing else I might be able to use similar logic/behavior
<tenderlove>
I'm getting 500s when pushing gems again
<dwradcliffe>
tenderlove: let me take a look
<lmarburger>
dwradcliffe: after you track that down, do you know anything about the .gem and .gemspec.* redirects to fastly that were in nginx being removed last night?
<dwradcliffe>
tenderlove: try again
<tenderlove>
worked that time
<tenderlove>
thanks!
<dwradcliffe>
tenderlove: np, sorry, again. We're actively working on ops this week.
<tenderlove>
np. Thanks for fixing it! :D
<dwradcliffe>
lmarburger: yep. I know why it's gone now.
<dwradcliffe>
lmarburger: who deployed that config change?
<lmarburger>
not in the chef config?
<lmarburger>
dwradcliffe: evan
<dwradcliffe>
lmarburger: bingo
<dwradcliffe>
I'll probably be pushing out some changes tonight, so hopefully we can get that back in
<dwradcliffe>
I never saw the config so I'm not sure what needs to get added
dvu has quit [Remote host closed the connection]
dvu has joined #rubygems
<lmarburger>
dwradcliffe: thanks for the help!
<dwradcliffe>
lmarburger: np!
<lmarburger>
i know the domain that gem and gemspec requests need to hit but i don't know what the nginx config looked like.
<lmarburger>
so if you need my help with any of that, let me konw
dvu has quit [Ping timeout: 272 seconds]
<dwradcliffe>
lmarburger: if I don't hear back from evan before tonight I might try to rebuild it myself
jfoy has joined #rubygems
<imperator>
hm, i did get the "invalid switch in RUBYOPT: -a" on winders
<imperator>
but only when i had GEM_PATH with a space in it
<roidrage>
it's very random but it comes up way too often
<drbrain>
roidrage: that's likely due to an network issue on travis' end
<roidrage>
drbrain: negative
<drbrain>
if it doesn't get to reading the version number something is shutting down the connection
<roidrage>
we've had reports of that from other platforms as well
<drbrain>
where? I've only seen the travis reporting it
<roidrage>
how would we go about debugging this better? the error itself is unfortunately quite unspecific
<drbrain>
yesterday a user was in here asking about retrying HTTP requests, I may have to add that, but it's no guarantee that your won't get a hangup on your TCP connection for both requests
workmad3 has joined #rubygems
<imperator>
roidrage, what happens if you do openssl -connect rubygems.org:443 ?
<drbrain>
roidrage: I suspect that if you do ↑ often enough inside travis-ci you'll see the same error
<roidrage>
as i said, these errors are very spurious, they're frickin hard to pin down. 15 gems install properly, the 16th fails
<drbrain>
and unfortunately, OpenSSL doesn't propagate up the true error it discovered
TheComrade has joined #rubygems
<drbrain>
it could be corruption in the stream, or it could be a server misconfiguration or it could be that the connection hung up. The only way to tell is with tcpdump ☹
<drbrain>
also, does bundler have its own HTTP code?
<drbrain>
indirect: ↑
<roidrage>
bundler now fortunately has a built-in retry
<roidrage>
just waiting for indirect to ship a new version :)
<drbrain>
since its random, are you sure it isn't a network issue between travis and rubygems.org/S3?
<roidrage>
but thanks for the pointer, i'll raise this issue with our infrastructure team. there's been a network maintenance today
<drbrain>
I ask this because I haven't seen reports of the "wrong version number" error outside travis
<roidrage>
was there any reason why fastly was disabled?
<drbrain>
it may be that evan went to Paris and didn't want to mess with it
<drbrain>
or risk it breaking when he couldn't work oni t
<roidrage>
okay, cool. would be a bit too curious if that was related
<drbrain>
pure speculation on my part
stevenharman has quit [Quit: Leaving...]
<dwradcliffe>
roidrage: fastly was accidentally disabled because it wasn't setup properly the first time
<roidrage>
yeah, unfortunately speculation is all one can do in this setup :)
hakunin_ is now known as hakunin
<drbrain>
roidrage: can you create a travis task that creates tcpdump captures of multiple runs of openssl s_client ?
<roidrage>
i can look into that
<drbrain>
roidrage: retrieving pcap files (tcpdump -w) from a failed run would be most helpful