drbrain changed the topic of #rubygems to: RubyGems 2.1.11: http://bit.ly/rubygems-2-1-11 CVE-2013-4363: http://bit.ly/CVE-2013-4363 - Latest status: http://twitter.com/rubygems_status and http://status.rubygems.org
bbrowning has quit [Remote host closed the connection]
bbrowning has joined #rubygems
bf4 has quit [Ping timeout: 264 seconds]
workmad3 has quit [Ping timeout: 260 seconds]
pglombardo has quit []
ZachBeta has joined #rubygems
ddd has joined #rubygems
arohner has quit [Remote host closed the connection]
arohner has joined #rubygems
huoxito has joined #rubygems
stevenharman has quit [Quit: Leaving...]
arohner has quit [Ping timeout: 252 seconds]
<dwradcliffe> samkottler: any progress over the weekend?
stevenharman has joined #rubygems
seanlinsley has joined #rubygems
antichatter has joined #rubygems
huoxito has quit [Remote host closed the connection]
ZachBeta has quit [Quit: Computer has gone to sleep.]
tjbiddle_ has joined #rubygems
tjbiddle has quit [Ping timeout: 272 seconds]
tjbiddle_ is now known as tjbiddle
antichatter has quit [Quit: Colloquy for iPhone - http://colloquy.mobi]
seanlinsley has quit [Quit: seanlinsley]
seanlinsley has joined #rubygems
seanlinsley has quit [Client Quit]
tjbiddle has quit [Quit: tjbiddle]
seanlinsley has joined #rubygems
seanlinsley has quit [Quit: seanlinsley]
seanlinsley has joined #rubygems
stevenharman has quit [Quit: Leaving...]
sferik has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
bbrowning has quit [Quit: Leaving]
huoxito has joined #rubygems
huoxito has quit [Ping timeout: 248 seconds]
baburdick has joined #rubygems
lsegal has joined #rubygems
arohner has joined #rubygems
stevenharman has joined #rubygems
huoxito has joined #rubygems
baburdick has quit [Quit: Leaving.]
baburdick has joined #rubygems
tjbiddle has joined #rubygems
josh-k has joined #rubygems
stevenharman has quit [Quit: Leaving...]
huoxito has quit [Remote host closed the connection]
baburdick has quit [Quit: Leaving.]
josh-k has quit [Remote host closed the connection]
baburdick has joined #rubygems
csaunders has quit [Ping timeout: 240 seconds]
csaunders has joined #rubygems
baburdick has quit [Quit: Leaving.]
deryl has quit [Quit: Leaving]
josh-k has joined #rubygems
arohner has quit [Read error: Connection reset by peer]
arohner has joined #rubygems
arohner has quit [Ping timeout: 250 seconds]
niska has quit [Ping timeout: 252 seconds]
niska has joined #rubygems
jonahR has quit [Quit: jonahR]
tjbiddle has quit [Quit: tjbiddle]
josh-k has quit [Remote host closed the connection]
josh-k has joined #rubygems
josh-k has quit [Ping timeout: 245 seconds]
arohner has joined #rubygems
josh-k has joined #rubygems
arohner has quit [Remote host closed the connection]
arohner has joined #rubygems
kanobe has quit [Ping timeout: 246 seconds]
alindeman has quit [Ping timeout: 240 seconds]
arohner has quit [Remote host closed the connection]
alindeman has joined #rubygems
alindeman has quit [Ping timeout: 240 seconds]
alindeman has joined #rubygems
kanobe has joined #rubygems
tbuehlmann has joined #rubygems
jordia65 has joined #rubygems
alindeman has quit [Ping timeout: 240 seconds]
le_gars has joined #rubygems
alindeman has joined #rubygems
Elhu has joined #rubygems
le_gars has quit [Remote host closed the connection]
le_gars has joined #rubygems
le_gars has quit [Ping timeout: 252 seconds]
alindeman has quit [Ping timeout: 240 seconds]
alindeman has joined #rubygems
lsegal has quit [Quit: Quit: Quit: Quit: Stack Overflow.]
workmad3 has joined #rubygems
graphex has quit [Ping timeout: 248 seconds]
graphex has joined #rubygems
graphex has quit [Ping timeout: 248 seconds]
graphex has joined #rubygems
graphex has quit [Ping timeout: 248 seconds]
graphex has joined #rubygems
graphex has quit [Ping timeout: 248 seconds]
graphex has joined #rubygems
closer has quit [Ping timeout: 256 seconds]
graphex has quit [Ping timeout: 248 seconds]
closer has joined #rubygems
huoxito has joined #rubygems
alindeman has quit [Ping timeout: 240 seconds]
graphex has joined #rubygems
alindeman has joined #rubygems
graphex has quit [Ping timeout: 248 seconds]
graphex has joined #rubygems
graphex has quit [Ping timeout: 248 seconds]
jordia65 has quit [Remote host closed the connection]
graphex has joined #rubygems
jordia65 has joined #rubygems
jordia65 has quit [Remote host closed the connection]
jordia65 has joined #rubygems
graphex has quit [Ping timeout: 248 seconds]
graphex has joined #rubygems
graphex has quit [Read error: Connection reset by peer]
alindeman has quit [Ping timeout: 240 seconds]
alindeman has joined #rubygems
workmad3 is now known as wm3|lunch
Elhu has quit [Quit: Computer has gone to sleep.]
redmenace has quit [Ping timeout: 252 seconds]
redmenace has joined #rubygems
Elhu has joined #rubygems
huoxito has quit [Remote host closed the connection]
huoxito has joined #rubygems
wm3|lunch is now known as workmad3
bbrowning has joined #rubygems
stevenharman has joined #rubygems
stevenharman has quit [Quit: Leaving...]
josh-k has quit [Ping timeout: 248 seconds]
josh-k has joined #rubygems
josh-k_ has joined #rubygems
josh-k has quit [Ping timeout: 264 seconds]
ZachBeta has joined #rubygems
Authenticator has quit [Ping timeout: 272 seconds]
Authenticator has joined #rubygems
josh-k_ has quit [Remote host closed the connection]
bf4 has joined #rubygems
shtirlic has quit [Quit: ZNC - http://znc.in]
shtirlic has joined #rubygems
alindeman has quit [Ping timeout: 240 seconds]
alindeman has joined #rubygems
seanlinsley has quit [Quit: seanlinsley]
shtirlic has quit [Ping timeout: 240 seconds]
shtirlic has joined #rubygems
baburdick has joined #rubygems
graphex has joined #rubygems
graphex has quit [Quit: ZNC - http://znc.in]
graphex has joined #rubygems
Elhu has quit [Quit: Computer has gone to sleep.]
seanlinsley has joined #rubygems
huoxito has quit [Remote host closed the connection]
Elhu has joined #rubygems
jonahR has joined #rubygems
havenwood has joined #rubygems
seanlinsley has quit [Quit: seanlinsley]
seanlinsley has joined #rubygems
alindeman has quit [Ping timeout: 240 seconds]
kalleth has quit [Quit: http://quassel-irc.org - Chat comfortably. Anywhere.]
kalleth has joined #rubygems
alindeman has joined #rubygems
alindeman has quit [Ping timeout: 240 seconds]
alindeman has joined #rubygems
arohner has joined #rubygems
lmarburger has quit [Ping timeout: 272 seconds]
Elhu has quit [Quit: Computer has gone to sleep.]
mjc_ has joined #rubygems
Elhu has joined #rubygems
indirect has quit [Ping timeout: 252 seconds]
lmarburger has joined #rubygems
lmarburger is now known as lmarburge
indirect has joined #rubygems
lmarburge is now known as lmarburger
baburdick has quit [Quit: Leaving.]
ZachBeta has quit [Quit: Computer has gone to sleep.]
workmad3 is now known as wm3|away
jfelchner has quit [Read error: Connection reset by peer]
huoxito has joined #rubygems
jfelchner has joined #rubygems
alindeman has quit [Ping timeout: 240 seconds]
alindeman has joined #rubygems
arohner has quit [Remote host closed the connection]
jeremydei has left #rubygems ["WeeChat 0.4.1"]
_whitelogger has joined #rubygems
arohner has joined #rubygems
alindeman has quit [Ping timeout: 240 seconds]
alindeman has joined #rubygems
Elhu has joined #rubygems
arohner has quit [Remote host closed the connection]
arohner has joined #rubygems
arohner has quit [Remote host closed the connection]
Elhu has quit [Ping timeout: 240 seconds]
tjbiddle has joined #rubygems
Elhu has joined #rubygems
alindeman has quit [Ping timeout: 240 seconds]
bf4 has quit [Ping timeout: 246 seconds]
alindeman has joined #rubygems
baburdick has joined #rubygems
stevenharman has joined #rubygems
arohner has joined #rubygems
baburdick has quit [Ping timeout: 264 seconds]
ZachBeta has joined #rubygems
baburdick has joined #rubygems
alindeman has quit [Ping timeout: 240 seconds]
alindeman has joined #rubygems
jfelchner has quit [Quit: zzzzzzzzzzzzz………]
bf4 has joined #rubygems
havenn has joined #rubygems
havenn has quit [Read error: Connection reset by peer]
havenwood has quit [Ping timeout: 250 seconds]
havenwood has joined #rubygems
priyam_ has left #rubygems [#rubygems]
josh-k has joined #rubygems
jfelchner has joined #rubygems
alindeman has quit [Ping timeout: 240 seconds]
alindeman has joined #rubygems
jfelchner has quit [Ping timeout: 245 seconds]
havenwood has quit [Read error: Connection reset by peer]
jfelchner has joined #rubygems
jordia65 has left #rubygems [#rubygems]
arohner has quit [Remote host closed the connection]
arohner has joined #rubygems
arohner has quit [Ping timeout: 264 seconds]
jfelchner has quit [Quit: zzzzzzzzzzzzz………]
huoxito has quit [Remote host closed the connection]
jfelchner has joined #rubygems
huoxito has joined #rubygems
jfelchner has quit [Quit: zzzzzzzzzzzzz………]
arohner has joined #rubygems
arohner has quit [Remote host closed the connection]
arohner has joined #rubygems
Elhu has quit [Quit: Computer has gone to sleep.]
Elhu has joined #rubygems
popugaeva has joined #rubygems
Elhu has quit [Quit: Computer has gone to sleep.]
popugaeva has quit [K-Lined]
Elhu has joined #rubygems
tbuehlmann has quit [Remote host closed the connection]
alindeman has quit [Ping timeout: 240 seconds]
alindeman has joined #rubygems
jfelchner has joined #rubygems
Elhu has quit [Quit: Computer has gone to sleep.]
bbrowning is now known as bbrowning_away
wm3|away has quit [Ping timeout: 246 seconds]
Elhu has joined #rubygems
huoxito has quit [Remote host closed the connection]
<indirect> drbrain: ping?
<drbrain> indirect: hello
<indirect> rubygems isn't honoring bundler disabling dependency resolution
<indirect> and it is slowing things down by 2-4x
<indirect> which is rather unpleasant
<drbrain> for install or activation?
<indirect> for install
<indirect> I used that test to isolate the 4x slowdown to this commit: https://github.com/rubygems/rubygems/commit/bce182220aa188333fcaf095d8d4bbb725c32b8c
<indirect> and we directly instantiate GemInstaller
<drbrain> what does bundler use to install gems?
<indirect> we don't use DependencyInstaller
<drbrain> I thought it was Gem::Installer
<indirect> yes
<indirect> module Bundler
<indirect> class GemInstaller < Gem::Installer
<indirect> that is how we install gems
<indirect> and yet
<indirect> 4x slowdown
<indirect> from that commit
<indirect> specifically, we call Bundler::GemInstaller.new(path, options_hash).install
<indirect> some insight into how that slowdown is possible would be great
<indirect> because it affects all bundlers on rubygems 2.1+, including ruby 2.1
<indirect> and I would like to not be 4x slower at installing when ruby 2.1 is released :/
<drbrain> yeah
<drbrain> hold on
<indirect> k
ZachBeta has quit [Quit: Computer has gone to sleep.]
tjbiddle_ has joined #rubygems
tjbiddle has quit [Ping timeout: 252 seconds]
tjbiddle_ is now known as tjbiddle
bbrowning_away is now known as bbrowning
<drbrain> indirect: offhand, I don't see how this commit changed the install speed
<indirect> drbrain: me either!
<indirect> I just git bisected it :(
Elhu has quit [Quit: Computer has gone to sleep.]
* drbrain nods
<drbrain> what do I need to do to get this spec to run?
<indirect> rake spec:deps
<indirect> RUBYOPT="-I /Users/andre/src/rubygems/rubygems/lib" rspec spec --profile -- spec/install/path_spec.rb:17
<indirect> that's it
<indirect> the other thing that spec does is build a gem
<indirect> but every spec builds tons of gems
<indirect> so I am very unsure how that could be it
<drbrain> indirect: oh, I get hung up on the "duplicate dependencies" validation
<drbrain> let me go back in time
<indirect> oh, yeah
<indirect> I tested against that actually commit
<indirect> and it's parent
<indirect> there is a fix for the dup deps in the 1-5-stable branch
<indirect> if you need it
<drbrain> I'm on bundler master
<indirect> I bisected from v2.0.0 to v2.1.0
<indirect> using bundler master
<indirect> and git said it was that commit
<indirect> the times were super stable
<drbrain> ok, with 1-5-stable I see 3.78s
<indirect> 0.8s vs 2.5-3.5s
<indirect> before and after that commit
<drbrain> hrm
<drbrain> I get 2.86s for 964b2bc (parent of bce1822)
<drbrain> I'm seeing no significant difference between these two commits
<drbrain> maybe my -I is wrong
<drbrain> must have been
<indirect> ah
<indirect> yeah, it's very dramatic
<drbrain> something is happening with the options
<drbrain> I see ignore_dependencies: true arriving in Gem::Installer#new
<drbrain> but it's gone for process_options
<indirect> that seems bad
<indirect> :)
seanlinsley has quit [Quit: seanlinsley]
<drbrain> what's extra disconcerting is I added `raise @options.inspect` to Gem::Installer#initialize, but the spec does not show that as the raised exception
<drbrain> is bundler handling that exception somehow?
<drbrain> hrm, looks like process_options isn't at fault
<drbrain> for 964b2bc (parent) if I raise options.inspect after the require in Gem::Installer#initialize I get:
zzak has joined #rubygems
bf4 has quit [Ping timeout: 250 seconds]
<drbrain> {:bin_dir=>nil, :development=>nil, :env_shebang=>false, :force=>false, :format_executable=>false, :ignore_dependencies=>true, :install_dir=>"/Users/drbrain/Work/git/bundler/tmp/gems/system", :security_policy=>nil, :user_install=>nil, :wrappers=>true, :build_args=>nil}
<drbrain> for bce1822 if I do the same I get:
<drbrain> {:install_dir=>"/Users/drbrain/Work/git/bundler/tmp/gems/system", :only_install_dir=>true}
<zzak> drbrain: hi
<drbrain> zzak: hi
<zzak> do we have tickets?
<drbrain> tickets for what?
<zzak> gems bugs
<zzak> naruse is waiting on rc until we can fix this stuff
<drbrain> zzak: ↑ this stuff?
<drbrain> i.e. the bundler slowness?
<zzak> is there a bug in options too?
<drbrain> there's no bug in options on rubygems' side that I can see
<drbrain> I don't think that RC needs to be delayed for this issue since RubyGems 2.1 also has the same bug
<indirect> zzak: drbrain: so 2.1 will ship with the slow, unfixed version? :(
<indirect> or you plan to fix it between rc and final?
<drbrain> indirect: ↑ fix between RC and final
<indirect> gotcha
<indirect> cool
<indirect> thanks for looking at it
<drbrain> indirect: can you look at the bundler side and figure out why it isn't passing down ignore_dependencies: true ?
<drbrain> or any of the options?
<zzak> drbrain: #9250
<drbrain> indirect: I will apply bce1822 in parts to see which diff chunk(s) cause the slowdown
<indirect> uhhh
<indirect> how is that possible?
<zzak> drbrain: you mean < 1 week? :/
<indirect> the bundler class doesn't override the initializer: https://github.com/bundler/bundler/blob/master/lib/bundler/gem_installer.rb
<drbrain> zzak: 9250 is marked closed?
<drbrain> indirect: if I `raise` inside Gem::Installer.new, that doesn't show up as a proper exception in the spec which is very troubling to me
<zzak> drbrain: was it merged upstream?
<drbrain> zzak: yes
<indirect> drbrain: the bundler spec doesn't run the code
<indirect> it uses Open3 to run `bundle install`
<indirect> and collects the output
<indirect> you will not see a raised exception
<drbrain> ☹ ☹ ☹
<indirect> this is an integration test
<indirect> I don't have a unit test that easily exhibits the exact same behaviour
<drbrain> I know
<drbrain> zzak: AFAICT, rubygems is ready to go for RC
<indirect> drbrain: you can see the backtrace from your exception by applying this patch: https://gist.github.com/80ca6888e01bacb3df80
<drbrain> indirect: thanks
<drbrain> indirect: it's bundler's bug
<drbrain> this is going through Bundler::Source::Path::Installer
<drbrain> which overrides #initialize but does not super
<drbrain> well, in part is bundler's bug
josh-k has quit [Remote host closed the connection]
<indirect> ugh
<indirect> yeah
<indirect> not sure if super()ing will break everything
<indirect> but I am happy to patch as is helpful :P
<drbrain> I think super() for Path::Installer is wrong
<drbrain> but you probably want to process_options in there
<indirect> drbrain: calling super() does not help, yeah
<drbrain> there's no .gem for a Path gem, so Gem::Package.new will fail
<drbrain> (in Gem::Installer#initialize)
<indirect> drbrain: process_options(options) does not help either
<drbrain> @options = options; process_options, I think
<indirect> ah
<indirect> okay
<drbrain> but I don't think that's going to fix our speed problem anyhow
<indirect> yeah, that doesn't solve the speed problem :/
<drbrain> indirect: haha
<drbrain> indirect: I think the problem is require
<indirect> ahh, interesting
<drbrain> +require 'rubygems/dependency_resolver'
<drbrain> +require 'rubygems/dependency_resolver/installer_set'
<drbrain> I think adding those two requires causes lots more files to get loaded
<drbrain> I had noticed this last week and shuffled some things around on master
<indirect> interesting
<indirect> but master is just as slow
<drbrain> but maybe it was insufficient?
<drbrain> I was all "why does this take so long now?"… "oh, requiring files I'm not using"
havenwood has joined #rubygems
<drbrain> let me check
<indirect> I see
<indirect> yeah, I just checked rg master against 1-5-stable
<indirect> and it's still slow
<drbrain> maybe I didn't get them all?
<indirect> drbrain: it's actually worse on rg master
<drbrain> ok
<drbrain> hrm, my theory seems wrong
<drbrain> adding just the requires shows no difference
<indirect> 1.1s vs 7s on master :(
<drbrain> hrm, somebody is executing Gem::DependencyInstaller#install
<drbrain> bundler is using Gem::DependencyInstaller via `gem`:
seanlinsley has joined #rubygems
<drbrain> indirect: spec/support/builders.rb:616
<drbrain> so maybe this is only a spec bug?
<drbrain> and, any idea why this doesn't use Gem::Installer directly here?
<indirect> what is using `gem`?
<indirect> hmmm
<drbrain> the bundler spec helpers are
<indirect> gem_command :install, "--no-rdoc --no-ri --ignore-dependencies #{path}"
<indirect> you mean that?
<indirect> oh I see
<indirect> but yes
<indirect> that also passes --ignore-dependencies
<indirect> iirc we use gem install because we're trying to create an integration test that references system gems
<indirect> which are installed via `gem install`
<indirect> if we are passing --ignore-dependencies every place we call `gem install`, why is it slower?
<drbrain> I will continue to investigate, but in the mean time:
<drbrain> using Gem::Installer is a 75% speedup
<indirect> wow
<indirect> super strange
<indirect> thanks
<indirect> that is also very surprising
<drbrain> I don't know if it will break other specs though
<indirect> I would not have expected going through the gem binary to slow things down by so much
<drbrain> it shouldn't because `gem install` and Gem::Installer.new().install look at GEM_HOME just the same
<indirect> yeah
<indirect> exactly
<drbrain> reloading all that code takes a good while
<indirect> heh
<indirect> I guess the problem is that I didn't realize how big rubygems is :P
<drbrain> I think just loading DependencyInstaller, even if dependencies are ignored, takes a while
<indirect> oh
<indirect> that is unfortunate
<drbrain> autoload might help
<indirect> do --no-ri and --no-rdoc also need to be passed in to Gem::Installer?
<drbrain> no, that's happens from DependencyInstaller
<indirect> ah, okay
<indirect> great
stevenharman has quit [Quit: Leaving...]
stevenharman has joined #rubygems
<drbrain> compared to rubygems 2.0.0, looks like master does the resolution even with --ignore-dependencies