hsbt_away changed the topic of #ruby-core to: check the latest release candidate for 1.9.1 release ftp.ruby-lang.org:/home/yugui/ruby-1.9.1-r26021+1.tar.bz2
<znz_jp> biff: [ruby-changes:31467] ko1:r43546 (trunk): * gc.c: define RGENGC_ESTIMATE_OLDSPACE == 0 if USE_RGENGC is 0. - http://mla.n-z.jp/?ruby-changes=31467
<znz_jp> biff: [ruby-changes:31468] ko1:r43547 (trunk): * benchmark/bm_vm1_gc_short_lived.rb: added. - http://mla.n-z.jp/?ruby-changes=31468
<znz_jp> biff: [ruby-changes:31469] nobu:r43548 (trunk): * properties. - http://mla.n-z.jp/?ruby-changes=31469
nagachika has joined #ruby-core
n0kada has joined #ruby-core
eLobato has quit [Ping timeout: 245 seconds]
tylersmith has quit [Remote host closed the connection]
nari has joined #ruby-core
n0kada has quit [Remote host closed the connection]
<znz_jp> biff: [ruby-changes:31470] glass:r43549 (trunk): * array.c (rb_ary_and): defer hash creation and some refactoring. - http://mla.n-z.jp/?ruby-changes=31470
knu has quit [Ping timeout: 246 seconds]
tylersmith has joined #ruby-core
tylersmith has quit [Remote host closed the connection]
tylersmith has joined #ruby-core
<znz_jp> biff: [ruby-changes:31471] glass:r43550 (trunk): * array.c (rb_ary_sample): use RARRAY_AREF() and RARRAY_PTR_USE() - http://mla.n-z.jp/?ruby-changes=31471
DanKnox is now known as DanKnox_away
shinnya has quit [Ping timeout: 240 seconds]
wycats has joined #ruby-core
<wycats> question:
<wycats> b = "\u{e9}".force_encoding("BINARY")
<wycats> I would expect b.encode("UTF-8", "BINARY") to work
<wycats> but I get an exception
<wycats> it seems like .encode is going byte-by-byte
<wycats> instead of looking for UTF-8 character sequences
<drbrain> wycats: I get: -e:1:in `encode': U+00E9 from UTF-8 to ASCII-8BIT (Encoding::UndefinedConversionError)
<drbrain> so there's no mapping from Unicode to binary, which seems sane
<wycats> that isn't what it's saying
<wycats> you can do stuff like define replacement characters etc.
<wycats> and it works
<wycats> but if you define replacement characters, it will just return two replacement characters
<wycats> I'm looking for a way to take a binary string and replace only the invalid UTF-8 sequences with replacement characters
<wycats> but not valid UTF-8 sequences
soba has joined #ruby-core
<drbrain> "\u{e9}".encode "BINARY", undef: :replace, replace: "X" #=> "X"
<drbrain> I only get one replacement character after providing a mapping
<drbrain> and, it is saying there's no mapping from Unicode to binary for U+00E9
<wycats> maybe it changed in 2.1?
<drbrain> well, I guess you could s/mapping/conversion
<drbrain> let me try in 2.1, I ran the above in 2.0.0p247
<drbrain> with: ruby 2.1.0dev (2013-10-28 trunk 43454) [x86_64-darwin13.0.0] I also get one "X"
<wycats> hm interesting
<wycats> it works if you encode from UTF-8->BINARY
<wycats> iiinteresting
<wycats> makes sense
<wycats> wait
<wycats> no
<drbrain> I think only 7-bit characters are OK for UTF-8 to BINARY
<wycats> right
<wycats> I'm looking for a way to just remove invalid sequences
<wycats> the Internet says ^^
<wycats> but the internet is wrong
<wycats> people didn't test with valid sequences
<wycats> they just did "some ascii <invalid lol> other ascii"
<drbrain> so you have a string that is in some unknown encoding, maybe UTF-8
<wycats> and were like: WHEE NORMALIZED
<drbrain> and you want to strip all bad characters?
<wycats> right
<wycats> but I know the valid characters are UTF-8
<drbrain> I have that around here somewhere in RDoc I cthink
<drbrain> let me check
<wycats> I want to strip characters that are not valid UTF-8 sequences
<drbrain> OK
<wycats> :D
<wycats> I found that to be very surprising
<wycats> _ko1: ^^
<wycats> AllocationCounter is just a script I wrote up
<wycats> to figure out how many allocations some block of code generates
<wycats> tl;dr if you have a UTF-8 string scanner and try to use a regular ASCII regex on it, you get a string allocation
<wycats> I've been working on reducing the allocations in the Skylight Ruby agent
<drbrain> nice
<wycats> socket.syswrite string.freeze -> 0 allocations; socket.print string -> 2 allocations
<drbrain> wycats: in general, encode through a compatible encoding to eliminate bad characters
<wycats> interesting you round-trip to UTF-16
<drbrain> yes
<drbrain> UTF-8 to UTF-8 is a no-op :/
<wycats> makes sense
<wycats> yeah
<wycats> it seems like there should be something built-in for this
<drbrain> yeah
<wycats> I guess I could encode! to reduce allocations, but it's going to have to double the string size :/
<drbrain> there's a fancier API that I think runpaint has (had?) documentation on that sets up a full encoder
<drbrain> I doubt it'll reduce your allocations though
<wycats> definitely not
<wycats> there are lots of little things: slice! allocates two strings (an intermediate one that nobody ever sees)
<wycats> I'll do a full writeup eventually
<drbrain> (unrelated) in drbdump (tcpdump for drb) I track allocations by walking the Marshal stream
<drbrain> I can say "this stream of objects causes N allocations" without loading it
<drbrain> (but now that we have Flonum it's an upper bound)
<wycats> Cool :)
<wycats> Oh another one: fixnum < bignum allocates a bignum
<wycats> It should just return true
<wycats> That one was killing the protobuf library we were using because it was doing range checks against MaxUint64
<drbrain> it can't because of Bignum#coerce
<drbrain> you can get a Bignum for small numbers
<wycats> But why is that coercion necessary?
<drbrain> I'm not sure if it happens only for <=>
<drbrain> I mean, some other math operation could cause the coercion
<wycats> I just do a check for MaxFixnum first
<wycats> But meh
<drbrain> … maybe it could be eliminated
<wycats> It was surprising
knu has joined #ruby-core
* drbrain nods
<wycats> I should file these
<wycats> I filed a few already
<drbrain> p (2**64).coerce(5).first.class #=> Bignum
<wycats> Yeah
<wycats> But the compare should be doable in C
<wycats> Should not require coercing the Fixnum into a heap-allocated bignum just to compare
<wycats> Similarly :"foo#{bar}baz" where bar is a T_STRING should not require any allocations
xibbar_ie has joined #ruby-core
ItSAN____ has joined #ruby-core
r0bglees0n has joined #ruby-core
tylersmi_ has joined #ruby-core
<drbrain> historically those types of optimizations (interpolation) haven't been done in case someone overrides String#to_s
DanKnoxz has joined #ruby-core
DanKnoxz is now known as DanKnox
tylersmith has quit [*.net *.split]
ItSANgo_ has quit [*.net *.split]
r0bgleeson has quit [*.net *.split]
xibbar_i_ has quit [*.net *.split]
DanKnox_away has quit [*.net *.split]
knu has quit [Ping timeout: 246 seconds]
<wycats> drbrain: I don't know if that's a good enough reason
<wycats> you can just check that
<wycats> keep a global flag ;)
<wycats> vm->stupid_user_overrides_string_to_s
<drbrain> now that we have a VM instead of an interpreter you can do fancier things than that
<drbrain> like caching method lookups
<drbrain> so everybody wins, not just interpolation
<wycats> <3
<wycats> interpolation is a pretty common cause of string allocations
<wycats> send :"dispatch_#{type}" isn't uncommon
r0bglees0n has quit [Ping timeout: 245 seconds]
nagachik_ has joined #ruby-core
nagachika has quit [Read error: Operation timed out]
<znz_jp> biff: [ruby-changes:31472] naruse:r43551 (trunk): Suppress Warnings: instance variable @installed_by_version not initialized - http://mla.n-z.jp/?ruby-changes=31472
<znz_jp> biff: [ruby-changes:31473] naruse:r43552 (trunk): Suppress warnings: File.exists? is a deprecated name - http://mla.n-z.jp/?ruby-changes=31473
<znz_jp> biff: [ruby-changes:31474] naruse:r43553 (trunk): Suppress warnings: ambiguous first argument; put parentheses or even spaces - http://mla.n-z.jp/?ruby-changes=31474
nagachik_ has quit [Remote host closed the connection]
nagachika has joined #ruby-core
knu has joined #ruby-core
marcandre has quit [Remote host closed the connection]
flgr has quit [Remote host closed the connection]
tylersmi_ has quit [Remote host closed the connection]
DanKnox is now known as DanKnox_away
<znz_jp> biff: [ruby-changes:31475] duerst:r43554 (trunk): lib/open3.rb: tweaked grammar in comments - http://mla.n-z.jp/?ruby-changes=31475
nari has quit [Ping timeout: 245 seconds]
tylersmith has joined #ruby-core
tylersmith has quit [Ping timeout: 265 seconds]
tylersmith has joined #ruby-core
tylersmith has quit [Ping timeout: 260 seconds]
ItSAN____ has quit [Ping timeout: 272 seconds]
r0bglees0n has joined #ruby-core
<lopex> wycats: isnt that what scrub does ?
<lopex> removing bad chars
r0bglees0n has quit [Ping timeout: 272 seconds]
ItSANgo has joined #ruby-core
soba has quit [Ping timeout: 240 seconds]
tylersmith has joined #ruby-core
tylersmith has quit [Ping timeout: 272 seconds]
<znz_jp> biff: [ruby-changes:31476] tarui:r43555 (trunk): * common.mk (help): add texts about gcbench. - http://mla.n-z.jp/?ruby-changes=31476
ItSANgo has quit [Ping timeout: 264 seconds]
tylersmith has joined #ruby-core
tylersmith has quit [Ping timeout: 240 seconds]
<drbrain> lopex: Ruby 2.1 only :(
<drbrain> wycats: ↑
<znz_jp> biff: [ruby-changes:31477] glass:r43556 (trunk): * array.c (rb_ary_reverse): use RARRAY_PTR_USE(). - http://mla.n-z.jp/?ruby-changes=31477
<znz_jp> biff: [ruby-changes:31478] glass:r43557 (trunk): * array.c (rb_ary_shift_m): use RARRAY_PTR_USE() without WB because - http://mla.n-z.jp/?ruby-changes=31478
r0bglees0n has joined #ruby-core
tylersmith has joined #ruby-core
tylersmith has quit [Ping timeout: 272 seconds]
<znz_jp> biff: [ruby-changes:31479] tarui:r43558 (trunk): * gc.c (gc_before_sweep): Change algorithm of malloc_limit to - http://mla.n-z.jp/?ruby-changes=31479
<lopex> drbrain: but hey, ruby likes to backport everything
ItSANgo has joined #ruby-core
tylersmith has joined #ruby-core
tylersmith has quit [Ping timeout: 245 seconds]
eLobato has joined #ruby-core
shinnya has joined #ruby-core
marcandre has joined #ruby-core
nari has joined #ruby-core
ItSANgo has quit [Ping timeout: 246 seconds]
nari has quit [Ping timeout: 260 seconds]
tylersmith has joined #ruby-core
tylersmith has quit [Ping timeout: 265 seconds]
nari has joined #ruby-core
nari has quit [Ping timeout: 272 seconds]
kosaki2 has joined #ruby-core
kosaki2 has quit [Remote host closed the connection]
kosaki2 has joined #ruby-core
nagachika has quit [Remote host closed the connection]
eLobato has quit [Quit: WeeChat 0.4.1]
ZachBeta has joined #ruby-core
ZachBeta has quit [Quit: Computer has gone to sleep.]
nagachika has joined #ruby-core
nagachika has quit [Remote host closed the connection]
tylersmith has joined #ruby-core
tylersmith has quit [Ping timeout: 244 seconds]
ZachBeta has joined #ruby-core
n0kada has joined #ruby-core
n0kada has quit [Remote host closed the connection]
<zzak> not new features
shinnya has quit [Ping timeout: 246 seconds]
enebo has joined #ruby-core
kosaki2_ has joined #ruby-core
kosaki2 has quit [Ping timeout: 272 seconds]
ZachBeta has quit [Quit: Computer has gone to sleep.]
ZachBeta has joined #ruby-core
<wycats> scrub sounds good
kosaki2_ has quit [Remote host closed the connection]
kosaki2 has joined #ruby-core
DanKnox_away is now known as DanKnox
tylersmith has joined #ruby-core
tylersmith has quit [Remote host closed the connection]
tylersmith has joined #ruby-core
lopex has quit [Remote host closed the connection]
guilleiguaran_ has quit [Remote host closed the connection]
Guest85414___ has quit [Write error: Broken pipe]
kosaki2 has quit [Remote host closed the connection]
kosaki2 has joined #ruby-core
wycats has quit [Remote host closed the connection]
Domon_ has quit [Remote host closed the connection]
wycats has joined #ruby-core
ggherdov has quit [Remote host closed the connection]
kosaki2 has quit [Ping timeout: 264 seconds]
ZachBeta has quit [Quit: Computer has gone to sleep.]
Guest85414___ has joined #ruby-core
guilleiguaran_ has joined #ruby-core
lopex has joined #ruby-core
ggherdov has joined #ruby-core
Domon_ has joined #ruby-core
ZachBeta has joined #ruby-core
kosaki2 has joined #ruby-core
kbsa__ has joined #ruby-core
kbsa has quit [Ping timeout: 240 seconds]
kbsa__ is now known as kbsa
kosaki2 has quit [Remote host closed the connection]
kosaki2 has joined #ruby-core
kosaki2 has quit [Read error: Connection reset by peer]
n0kada has joined #ruby-core
n0kada_ has joined #ruby-core
n0kada has quit [Ping timeout: 240 seconds]
<znz_jp> biff: [ruby-changes:31480] nobu:r43559 (trunk): * .gdbinit (rp): fix flag name. - http://mla.n-z.jp/?ruby-changes=31480
<znz_jp> biff: [ruby-changes:31481] svn:r43560 (trunk): * 2013-11-07 - http://mla.n-z.jp/?ruby-changes=31481
kosaki2 has joined #ruby-core
enebo has quit [Quit: enebo]
kosaki2_ has joined #ruby-core
kosaki2 has quit [Read error: Connection reset by peer]
kosaki2 has joined #ruby-core
kosaki2_ has quit [Read error: Operation timed out]
kosaki2 has quit [Read error: Operation timed out]
ZachBeta has quit [Quit: Computer has gone to sleep.]