sipa changed the topic of #bitcoin-wizards to: This channel is for discussing theoretical ideas with regard to cryptocurrencies, not about short-term Bitcoin development | http://bitcoin.ninja/ | This channel is logged. | For logs and more information, visit http://bitcoin.ninja
<aj>
gmaxwell: why would you pass a vector cost? you'd either calculate it from scratch, or just pass the pre-calculated value x=f(....)?
<gmaxwell>
aj: if the precalculated value is normative and inside the transaction, then the change would be a hardfork that killed all in-flight transactions.
<gmaxwell>
You can think of it in terms of the "regret" you would have from an inaccurate cost model that lets someone pay a much lower price for abuse... which effectively means that for a cost bound attacker you need some factor safty margin for mispricing.
dEBRUYNE_ has quit [Ping timeout: 256 seconds]
c0rw1n is now known as c0rw|zZz
<gmaxwell>
Turns out that you can be pretty far off on your metrics and not end up with crazy required safty margins.
<aj>
gmaxwell: you'd need to reorder all the transactions anyway when the change hit
<gmaxwell>
aj: yea, but that doesn't synchronize every user of the bitcoin system.
DougieBot5000 has quit [Quit: Leaving]
<gmaxwell>
having to resign transactions, which might require manual intervention.. much bigger step.
bliljerk101 has quit [Ping timeout: 272 seconds]
<aj>
gmaxwell: oh, i was thinking x=f() would just be attached to the txn loosely, not signed
<aj>
gmaxwell: (and validated as part of running the script, or otherwise validating the txn)
Newyorkadam has joined #bitcoin-wizards
<gmaxwell>
yea, part of the relay scheme? I think in the past we'd assumed the cost would be in the transaction; but now that we discuss it I don't recall if there was a good reason for that. Well so, fraud proofs require a MST for each globally limited constraint, but that doesn't mean thet they need to be part of the transaction either.
<gmaxwell>
aj: what you do with them is run with them, bound the execution cost of the transaction with them, if violated you ban the peer.
bliljerk101 has joined #bitcoin-wizards
c0rw|zZz_ has joined #bitcoin-wizards
<aj>
gmaxwell: right, so you'd need a vector MST commitment for the constraints as part of the block; i guess you could extend the vector to add new constraints as a soft fork though
<gmaxwell>
aj: yea, actually the mst style being talked about for segwit fraud proof was arbritarily extendable.
c0rw|zZz has quit [Ping timeout: 272 seconds]
<gmaxwell>
I posed a question to some people previously: do we ever care about a reduction operator other than sum?
strider has joined #bitcoin-wizards
<tromp__>
maybe min or max?
<gmaxwell>
Thats also what I suggested as a plausable candidate.
<tromp__>
or less likely, xor
<gmaxwell>
e.g. any communitive two input combining function is a potentially logical thing.. but I can't come up with uses for things other than min/max and even those are a little tortured.
<gmaxwell>
and min and max are the same function; just invert your values, so only one would be needed.
<aj>
gmaxwell: i think anything other than linear summation would make optimisation unnecessarily hard?
CubicEarth has quit [Remote host closed the connection]
<gmaxwell>
aj: multiple dimensions makes optimization unnecessarily hard. I expect they'd be reduced to a single cost at the end.
GGuyZ has joined #bitcoin-wizards
zookolaptop has joined #bitcoin-wizards
<aj>
gmaxwell: that's what i mean, linear sum lets you reduce multi-dimensions to a single cost easily
<aj>
gmaxwell: (and yeah, sure, even /that's/ still hard enough...)
<gmaxwell>
aj: I mean you might rationally have a rule that says "no script execution in a block can use more than 12 MB ram"
<gmaxwell>
But while saying the justification for max outloud I realize that a max actually doesn't really need a MST for a fraud proof.
<aj>
gmaxwell: or X elements of stack
<gmaxwell>
Just show the naughty transaction directly.
c0rw|zZz_ is now known as c0rw|zZz
<gmaxwell>
hurrah. I think max/min actually has no utility for fraud proofing a resource constraint.
<gmaxwell>
because though max is a global constrant, profing you've exceeded a maximum has an O(1) proof.
<gmaxwell>
(well log n, of course, but without any extra commitments)
CubicEarth has joined #bitcoin-wizards
jannes has quit [Quit: Leaving]
GGuyZ has quit [Remote host closed the connection]
GGuyZ has joined #bitcoin-wizards
<aj>
gmaxwell: does elements do any fancier costing, given it enables OP_CAT and OP_SUBSTR? i guess "blah" {DUP CAT} SHA256 could be worth accounting
<gmaxwell>
aj: No, because CAT can only produce an object no larger than you could have just pushed in.
<gmaxwell>
(Cat has to be limited or you easily get effectively unbounded memory use)
<gmaxwell>
We did some benchmarking with a thought of also increasing the operations limit; and concluded that it wasn't obviously safe to do so, so we didn't... but it deserved more exploration.
jannes has joined #bitcoin-wizards
<gmaxwell>
Basically we used AFL to try to find the slowest scripts to execute, and I think our results were goofed. Also: malloc sucks.
<aj>
gmaxwell: "AFL" ?
<gmaxwell>
American fuzzy lop; a fast instrumentation based whitebox fuzzer.
CubicEarth has quit [Remote host closed the connection]
GGuyZ has quit [Quit: GGuyZ]
Sleepnbum has quit [Ping timeout: 240 seconds]
Newyorkadam has quit [Quit: Newyorkadam]
Newyorkadam has joined #bitcoin-wizards
Yoghur114_2 has quit [Remote host closed the connection]
DougieBot5000 has joined #bitcoin-wizards
joecool has quit [Ping timeout: 240 seconds]
CubicEarth has joined #bitcoin-wizards
CubicEar_ has joined #bitcoin-wizards
dabura667 has joined #bitcoin-wizards
CubicEarth has quit [Ping timeout: 272 seconds]
saibog38 has joined #bitcoin-wizards
jcorgan|away is now known as jcorgan
jannes has quit [Ping timeout: 256 seconds]
justanotheruser has quit [Read error: Connection reset by peer]
justanotheruser has joined #bitcoin-wizards
GGuyZ has joined #bitcoin-wizards
GGuyZ has quit [Client Quit]
ebfull has joined #bitcoin-wizards
GGuyZ has joined #bitcoin-wizards
GGuyZ has quit [Client Quit]
eudoxia has joined #bitcoin-wizards
tripleslash_a has quit [Read error: Connection reset by peer]
kumavis has quit [Ping timeout: 264 seconds]
jbenet has quit [Ping timeout: 264 seconds]
tripleslash has joined #bitcoin-wizards
Ylbam has quit [Ping timeout: 264 seconds]
kumavis has joined #bitcoin-wizards
jbenet has joined #bitcoin-wizards
kelly has quit [Ping timeout: 252 seconds]
matsjj has joined #bitcoin-wizards
licnep has joined #bitcoin-wizards
bendavenport has quit [Quit: bendavenport]
zookolaptop has quit [Ping timeout: 250 seconds]
TBI_ has joined #bitcoin-wizards
matsjj has quit [Remote host closed the connection]
TBI has quit [Ping timeout: 240 seconds]
gielbier has quit [Read error: Connection reset by peer]
liteIRC_ has joined #bitcoin-wizards
zooko has quit [Ping timeout: 240 seconds]
liteIRC_ is now known as zooko
saibog38 has quit [Quit: Page closed]
gobias_industrie has joined #bitcoin-wizards
<maaku>
petertodd: I do not believe anyone has observed that you can get sharding via linearized coin histories, that is novel. you also need a relaxation of validation conditions, which I believe is also novel
<maaku>
but yes mizrahi's approach to coin coloring is to establish a linearized coin history, in order to keep the colored coin proof small
gobias_industrie is now known as gobiasindustries
<maaku>
kanzure: stateless mining / stateless validation is the term I've been using
<maaku>
gmaxwell: the 20k sigop limit is for outputs, not the sigops actually executed by the block..
liteIRC_ has joined #bitcoin-wizards
strider has quit [Quit: Leaving]
<gmaxwell>
maaku: no, for P2SH it is counted correctly.
<gmaxwell>
I explained this above.
<maaku>
right but a block isn't limited to p2sh spends...
<gmaxwell>
Right but seg wit area is limited to segwit spends.
spinza has quit [Ping timeout: 250 seconds]
zooko has quit [Ping timeout: 272 seconds]
liteIRC_ is now known as zooko
hackerman has quit [Read error: Connection reset by peer]
<gobiasindustries>
I'm wondering why there appears to be one bitcoin rich entity arguing with itself on bitcoinocracy.com
tachys_ has quit [Remote host closed the connection]
<gmaxwell>
gobiasindustries: whats the 'arguing with itself' refer to?
tachys_ has joined #bitcoin-wizards
wizkid057 has quit [Quit: brb]
<gobiasindustries>
there's a bunch of 500.00051 addresses voting on both sides, they were all funded around the same time in the same increments so I'm guessing it's the same person/entity/whatever
wizkid057 has joined #bitcoin-wizards
<gmaxwell>
well the 0.0051 is dust that people were spamming to large outputs.
<gobiasindustries>
ok, that makes more sense then.
liteIRC_ has joined #bitcoin-wizards
<gmaxwell>
gobiasindustries: Though one of the bitcoinXT trolls was hammering me in PM to "commit to making this binding"; might have been playing the other side to convince me to agree to something outright dumb (and beyond my power in any case) because I thought I would "win".
CubicEarth has joined #bitcoin-wizards
zooko has quit [Ping timeout: 272 seconds]
liteIRC_ is now known as zooko
CubicEar_ has quit [Ping timeout: 272 seconds]
matsjj has joined #bitcoin-wizards
zwick has quit [Quit: WeeChat 1.0.1]
spinza has joined #bitcoin-wizards
go1111111 has quit [Ping timeout: 240 seconds]
matsjj has quit [Remote host closed the connection]
<petertodd>
maaku: thanks, I sent alex an email to clarify; I'll post to bitcoin-dev to correct that if that's the case
<petertodd>
maaku: hopefully he didn't already tell me the linearization thing - that'd be kinda embarassing :)
tachys_ has quit [Ping timeout: 240 seconds]
mrkent_ has quit []
<kanzure>
"i have no idea what a whuffie is" -- maaku
<gmaxwell>
fortunate person.
<gmaxwell>
Thats a terrible story.
<gmaxwell>
:)
* gmaxwell
sends a bunch of anti-left-spin-counterclockwise whuffie in some random direction
<kanzure>
was doctorow the actual origin of whuffie stuff?
Sleepnbum has joined #bitcoin-wizards
Sleepnbum is now known as Guest63220
liteIRC_ has joined #bitcoin-wizards
<gmaxwell>
kanzure: the idea has existed elsewhere of course; but I'm pretty sure that name comes from DaOitMK
<gmaxwell>
(to be fair, I don't really think the story is terrible; it's not great and it's over hyped)
go1111111 has joined #bitcoin-wizards
<kanzure>
was there ever a whuffie implementation of any kind?
<gmaxwell>
'reddit' ?
<kanzure>
oh :-/
zooko has quit [Ping timeout: 272 seconds]
liteIRC_ is now known as zooko
GfxdjGFhgF has joined #bitcoin-wizards
GGuyZ has joined #bitcoin-wizards
<tulip>
gobiasindustries: if you have any value of privacy you should stay far away, especially given that voting with proof of stake like that is mostly meaningless anyway.
liteIRC_ has joined #bitcoin-wizards
zooko has quit [Read error: Connection reset by peer]
liteIRC_ is now known as zooko
<jcorgan>
afraid now to say anything around kanzure because his records are more detailed than the NSA's
liteIRC_ has joined #bitcoin-wizards
RoboTeddy has quit [Remote host closed the connection]
<kanzure>
yeah i'm surprised more people haven't talked with me about that or tried to convince me against not doing that
<kanzure>
probably a selection effect of some kind :-/
<CubicEarth>
I liked Gavin's suggestion we call "Segregated Witness" by another name. How about "Split Witness" or "Separate Witness"
RoboTeddy has joined #bitcoin-wizards
<jcorgan>
is it a term of art?
<jcorgan>
(well known, specific meaning within a field or community)
liteIRC__ has joined #bitcoin-wizards
zooko has quit [Read error: Connection reset by peer]
liteIRC__ is now known as zooko
<CubicEarth>
jcorgan: Segregated Witness?
liteIRC__ has joined #bitcoin-wizards
liteIRC_ has quit [Ping timeout: 240 seconds]
<jcorgan>
yes. i thought it was a well-defined cryptographic term, could be wrong
<jcorgan>
if so, would rather keep it as-is
<aj>
witness is, segregated is just for bitcoin afaik
<CubicEarth>
Maybe, I'm unfamiliar if it is. I do know in American English, 'Segregation' is commonly used to refer to a long period of institutionalized racial discrimination.
<gmaxwell>
it's not actually detatched in most cases.
<jcorgan>
CubicEarth: i think context makes that connotaton unlikely
<kanzure>
words cannot be tainted by racism
<CubicEarth>
jcorgan: you might be right, but what do we have invested in the term "segregated". Wouldn't "Sepratate" or "Split" be just as good in this case?
<moa>
segregate verb set apart from the rest or from each other; isolate or divide.
<kanzure>
i suggest we kickban before this gets out of hand
<jcorgan>
really don't want to bikeshed on this
zooko has quit [Ping timeout: 272 seconds]
liteIRC__ is now known as zooko
<aj>
("remote witness" is actual legal terminology in .au apparently)
<aj>
ooo, ooo! "distinguished witness"
<jcorgan>
not that i have any say, but i'm perfectly happy with segregated witness
<kanzure>
this is off-topic
<CubicEarth>
kanzure: where should it be discussed?
rusty has joined #bitcoin-wizards
<amiller>
only two hard problems in computer science, cash invalidation and naming things
<CubicEarth>
amiller: +1
<kanzure>
i wonder if it would be possible for a deployed lightning network to be in the state such that broad amounts of money routing make it cheaper for people to earn money providing money in their channels (as negative fees) rather than actually purchasing coffee.
<kanzure>
or er, for the money they would earn to be more valuable than the coffee they would have had if they didn't provide their BTC as negative fee.
<CubicEarth>
I don't really care much myself. But, Bitcoin doesn't have a PR department, we'd probably be better off if we did, but we don't, so sometimes we may just have to consider how thing might appear to others on our own.
<gmaxwell>
If anyone but bitcoin developers and people spectating development see the term something has gone wrong.
eudoxia has quit [Quit: Leaving]
<aj>
amiller: "cash invalidation" eh...
GGuyZ has quit [Quit: GGuyZ]
<amiller>
you know, like preventing double-spends
<jcorgan>
*groan*
<oldbrew>
yea not enough beer
<phantomcircuit>
amiller, (and off by one errors)
<gmaxwell>
In any case, I don't care though segregated is a common term in several domains, including finance and law. (E.g. maintaining segregated client funds)
nubbins` has quit [Quit: Quit]
<aj>
amiller: hmm, i heard my cpu had multiple megabits of cash, but 1 megabit is 1BTC right? there must be some way to monetise this
<aj>
anyone know what the deal with segwit and p2pkh is? do we just end up with 32B addresses for everything?
<rusty>
aj: naah, IIUC there's a type byte so you can put script in directly for compact scripts like that
<aj>
rusty: oh, i thought type byte was just "1" for the first version
nuke1989 has quit [Remote host closed the connection]
Guest63220 has quit []
el33th4x0r has joined #bitcoin-wizards
liteIRC_ has joined #bitcoin-wizards
zooko has quit [Read error: Connection reset by peer]
liteIRC_ is now known as zooko
dabura667 has quit [Quit: Connection closed for inactivity]
p15x has joined #bitcoin-wizards
Newyorkadam has joined #bitcoin-wizards
Newyorkadam has quit [Client Quit]
liteIRC_ has joined #bitcoin-wizards
zooko has quit [Ping timeout: 272 seconds]
liteIRC_ is now known as zooko
alex_leishman has joined #bitcoin-wizards
sparetire_ has quit [Quit: sparetire_]
p15x has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
zooko has quit [Remote host closed the connection]
NewLiberty has quit [Ping timeout: 250 seconds]
c-cex-yuriy has quit [Quit: Connection closed for inactivity]
licnep has quit [Quit: Connection closed for inactivity]
p15x has joined #bitcoin-wizards
DigiDreamer has joined #bitcoin-wizards
tromp has quit [Remote host closed the connection]
tromp has joined #bitcoin-wizards
zooko has joined #bitcoin-wizards
GfxdjGFhgF has quit [Ping timeout: 272 seconds]
tromp has quit [Ping timeout: 250 seconds]
archobserver has quit [Ping timeout: 264 seconds]
tromp has joined #bitcoin-wizards
rusty has quit [Ping timeout: 256 seconds]
tromp has quit [Ping timeout: 250 seconds]
ThomasV has joined #bitcoin-wizards
bendavenport has joined #bitcoin-wizards
AaronvanW has joined #bitcoin-wizards
AaronvanW has quit [Changing host]
AaronvanW has joined #bitcoin-wizards
GGuyZ has joined #bitcoin-wizards
DigiDreamer has quit [Ping timeout: 246 seconds]
DigiDreamer has joined #bitcoin-wizards
p15x has quit [Ping timeout: 250 seconds]
bit2017 has joined #bitcoin-wizards
CubicEarth has quit [Remote host closed the connection]
CubicEarth has joined #bitcoin-wizards
CubicEarth has quit [Remote host closed the connection]
zooko has quit [Remote host closed the connection]
CubicEar_ has joined #bitcoin-wizards
AaronvanW has quit [Ping timeout: 250 seconds]
AaronvanW has joined #bitcoin-wizards
AaronvanW has quit [Changing host]
AaronvanW has joined #bitcoin-wizards
<alex_leishman>
i would like to discuss an updated signature scheme for bitcoin that has the potential to effectively remove most signature data from a block.
<alex_leishman>
are any of the "wizards" online?
<phantomcircuit>
alex_leishman, the standard for irc is to state your question (or statement) not to ask whether people are available to discuss the question
<alex_leishman>
oh ok. sorry. new to this
<phantomcircuit>
alex_leishman, it's kind of like writing your question on a chalkboard and then coming back in an hour :)
<alex_leishman>
Got it! Let me compose my question. I'll post soon
p15 has joined #bitcoin-wizards
<sipa>
i've been working on schnorr signatures, and combining them with a merkle tree of combinations, which can efficiently encode any and/or combinations of required signatures over public keus
<alex_leishman>
have you considered BLS signatures?
<sipa>
if you're interested, look up "key tree signatures"
<sipa>
BLS signatures are pretty inefficient to verify
<gmaxwell>
alex_leishman: they're not batch verifyable in a context which is interesting to us. And pairing is very slow. (also putting aside the new security assumptions for a bit)
<gmaxwell>
alex_leishman: (BLS signatures are batchable if they all sign the same message or all are the same key)
<alex_leishman>
Actually, I do not think this is the case
AaronvanW has quit [Ping timeout: 250 seconds]
bit2017 has quit [Ping timeout: 256 seconds]
<alex_leishman>
I was speaking with Dan Boneh about this yesterday
<alex_leishman>
this is his idea, not mine
<gmaxwell>
alex_leishman: Boneh agreed with me that it was not so many weeks ago; so perhaps he's run into something new.
<alex_leishman>
i'm simply working on figuring out the details
<alex_leishman>
hmmm.
<alex_leishman>
perhaps there is a miscommunication
<alex_leishman>
from my understanding, all input signatures in a block can be verified as a batch
<alex_leishman>
so a block may only need to include one aggregate signature
<sipa>
that would be pretty awesome
<gmaxwell>
Yes, one can aggregate signatures, I've implemented this-- but it is one pairing operation per signature.
<alex_leishman>
anyone can verify this single signature if they know all of the public keys associated with the signatures in the group used to make the aggregate signature, i.e. the input public keys
<gmaxwell>
which is fantastically slower than today.
<alex_leishman>
how much slower do you think it is?
<alex_leishman>
Dan said it was ~20%
<alex_leishman>
that was my understanding
RoboTeddy has joined #bitcoin-wizards
<gmaxwell>
oh dear no, state of the art pairing implementats right now are in the ballpart of 0.5 ms on hardware where our ecdsa verify is 70 microseconds.
tromp has joined #bitcoin-wizards
<gmaxwell>
(and maybe twice that speed for batch schnorr verification)
GGuyZ has quit [Quit: GGuyZ]
<alex_leishman>
interesting. so the big deal here is the pairing function speed?
<alex_leishman>
you're saying it's too slow?
bit2017 has joined #bitcoin-wizards
CubicEar_ has quit [Remote host closed the connection]
<gmaxwell>
in any case the verification equation for the aggregate signature is just pairing(m1,px1)*pairing(m2,px2)*pairing(m3,px3) == pairing(ag,g); and the aggregate is the product of the regular BLS signatures, (to_point(message)^secret).
<gmaxwell>
So thats one pairing per signature plus one.
CubicEarth has joined #bitcoin-wizards
<alex_leishman>
yes
<gmaxwell>
alex_leishman: right, it wouldn't be an improvement for most bandwidth/cpu limits; at the tip of the chain our performance is already not far from cpu limited. (it's annoying, there are different limits in different places... long time storage of signatures is a non-issue because of pruning)
tromp has quit [Ping timeout: 256 seconds]
<alex_leishman>
ok, so basically the CPU cost outweighs the bandwidth savings?
<alex_leishman>
let's set the pairing function speed aside
<alex_leishman>
what then?
<gmaxwell>
0.5ms per signature times transaction size implies a speed of under 2mbit/sec. (so say that per core.)
<alex_leishman>
i'm sorry. what is that message in reference to?
<gmaxwell>
Then it would be a new security assumption; (discrete log security in the BLS group) Though this doesn't bother me that much if its optional.
<alex_leishman>
yes
<gmaxwell>
alex_leishman: It was a translation of my example pairing speed into how small amount of bandwidth you'd need before cpu becomes the bottleneck.
<alex_leishman>
oh ok
<alex_leishman>
this scheme would effectively remove all signature data from a block
<alex_leishman>
what's your gut opinion? Is this idea worth formalizing and exploring further?
<gmaxwell>
Yes, I know, I've actually _implemented_ it (for privacy reasons).
<alex_leishman>
oh ok
<gmaxwell>
This kind of aggregation can be turned into a powerful privacy scheme. But it is pretty slow.
<gmaxwell>
In bitcoin storage is not a concern.
<alex_leishman>
but what about block propagation speed?
<alex_leishman>
network latency for miners. isn't that their concern?
<alex_leishman>
i understand validation slows propagation as well
<gmaxwell>
Would be slower with this (roughtly) if your connectivity is over a couple mbit/sec.
<alex_leishman>
but that aside
<alex_leishman>
yeah
<alex_leishman>
i'll talk to dan about pairing speed
<alex_leishman>
seems like that's the real issue here
<gmaxwell>
If we were limited to a super naieve implementation of ECC in a poorly selected group or something the difference wouldn't be so huge.
<alex_leishman>
@gmaxwell thanks for the feedback. I will dig deeper and come back with any new ideas
<gmaxwell>
alex_leishman: There may be.... there are things I know of that aren't applicable, e.g. you can precompute the pubkey pairings, but our keys are single use. It might be cheaper to verify a pairing instead of just compute one; but the transfer group elements are huge and would offset the savings.
bit2017 has quit [Ping timeout: 240 seconds]
<gmaxwell>
alex_leishman: feel free to come back with any-- I'm very interested.
<alex_leishman>
I will. Ok glad to hear you think it's worthwhile to pursue digging deeper.
bit2017 has joined #bitcoin-wizards
<alex_leishman>
We actually met a few weeks ago when you were at Stanford. I'm the TA for Dan's class
<alex_leishman>
one of the TAs
<sipa>
alex_leishman: oh, hi, i was there too (i'm pieter wuille)
<alex_leishman>
yeah! hey peter. It was nice meeting you as well
nivah has quit [Ping timeout: 240 seconds]
ThomasV has quit [Ping timeout: 240 seconds]
<gmaxwell>
alex_leishman: there are other tricks I know that also can't be applied here. E.g. if one of the arguments to the pairing is fixed, you can precompute and speed it up a fair bit.
<alex_leishman>
hmm. interesting. I was thinking about precomputing as much as possible
tripleslash has quit [Ping timeout: 240 seconds]
tripleslash_j has joined #bitcoin-wizards
bendavenport has quit [Quit: bendavenport]
LeMiner2 has joined #bitcoin-wizards
CubicEarth has quit [Remote host closed the connection]
CubicEarth has joined #bitcoin-wizards
LeMiner has quit [Ping timeout: 246 seconds]
moa has quit [Ping timeout: 256 seconds]
NewLiberty has joined #bitcoin-wizards
markus-k has joined #bitcoin-wizards
alex_leishman has quit [Ping timeout: 252 seconds]
p15 has quit [Ping timeout: 250 seconds]
p15 has joined #bitcoin-wizards
CubicEarth has quit [Remote host closed the connection]
justanotheruser has quit [Ping timeout: 272 seconds]
justanotheruser has joined #bitcoin-wizards
hashtagg_ has joined #bitcoin-wizards
[Derek] has quit [Ping timeout: 260 seconds]
[Derek] has joined #bitcoin-wizards
BlueMatt has joined #bitcoin-wizards
[Derek] is now known as Guest94083
markus-k has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
markus-k has joined #bitcoin-wizards
markus-k has quit [Read error: Connection reset by peer]
GGuyZ has joined #bitcoin-wizards
atgreen_ has joined #bitcoin-wizards
tromp has joined #bitcoin-wizards
GGuyZ has quit [Quit: GGuyZ]
damethos has quit [Ping timeout: 250 seconds]
tromp has quit [Ping timeout: 256 seconds]
lmatteis has quit [Quit: Connection closed for inactivity]
RoboTeddy has joined #bitcoin-wizards
chmod755 has joined #bitcoin-wizards
RoboTeddy has quit [Ping timeout: 256 seconds]
tripleslash has quit [Ping timeout: 240 seconds]
<bsm117532>
kanzure: "how do you avoid runaway bandwidth on this system without a block size limit?"
<bsm117532>
I didn't say anything about that directly, but I would like to remove the bandwidth limit as a consensus rule. We can find better ways to control it.
<bsm117532>
There is a natural bandwidth limit that is the "number of transactions" times a constant factor. e.g. if you're sending out blocks way faster than the underlying transaction rate, you're just blowing away bandwidth. Since coinbases in my proposal are empty, there's no value in a transaction-less block.
<bsm117532>
Also, it's silly to think that a miner on a Rasperry Pi and a shitty uplink will be able to run a global transaction processing system. Sorry, but eventually they won't be able to keep up. I don't want to artificially restrict bitcoin to be slow.
Quanttek has quit [Remote host closed the connection]
bramc has quit [Quit: This computer has gone to sleep]
<bsm117532>
There's also a general mechanism I call an "Equivalent Bead" where you can bundle a bunch of lower-work beads into a higher-work one. (This is e.g.how a bitcoin block is made from the braid) I actually removed this slide from my talk. At the end of the day it only saves you some headers of lower-work blocks, so doesn't save a large amount of bandwidth anyway.
Quanttek has joined #bitcoin-wizards
<kanzure>
that doesn't address the question at all; "blowing away bandwidth" no- at the moment the way the current system works is that there's a centralizing effect by having hashrate publish full blocks towards high-bandwidth peers. i suspect the same is true in your system.
<bsm117532>
kanzure: I think that is still the case in my system.
<kanzure>
then the scaling problem is unsolved :)
<bsm117532>
In fact I explicitly incentivized it by using the "cohort difficulty". If I hadn't done that, there's an attack where you can just hold on to your block forever and attempt to become everyone's sibling.
JackH has quit [Ping timeout: 240 seconds]
<bsm117532>
kanzure: I would set a floor as the minimum difficulty, which is a constant fraction times bitcoin's bandwidth. The extra bandwidth is only PoW headers for the extra beads and would be less than 2x bitcoin's current bandwidth.
<bsm117532>
So I'm not so concerned about it. Why are you?
<kanzure>
just making sure you don't claim this is some sort of scaling solution. has same inherent problem.
<kanzure>
it has other okay properties, i guess.
ttttemp has quit [Remote host closed the connection]
<bsm117532>
I'm far more concerned about bandwidth usage by free relay of txns in the p2p layer, than mined txns in my proposal.
<bsm117532>
And you can't scale without scaling bandwidth (modulo sipa's talk). My proposal is absolutely a scaling solution, because it removes the problems associated with larger block sizes or faster block rate.
ThomasV has quit [Ping timeout: 240 seconds]
sparetire_ has joined #bitcoin-wizards
<instagibbs>
in what ways? The fairness metric?
<bsm117532>
That and miner utilization too, because work isn't wasted on stale/orphan blocks.
ttttemp has joined #bitcoin-wizards
<instagibbs>
while that would be great, when kanzure is saying "scaling solution" he probably means something that isn't bound by "everyone must know everything forever".
<bsm117532>
kanzure wants sharding. I do too. My proposal doesn't address that at all.
<bsm117532>
I'm disappointed there hasn't been more work on that topic...
adam3us has quit [Quit: Leaving.]
<instagibbs>
LN is another "paradigm"
<bsm117532>
After I finish this paper I'm going to throw some more thought toward sharding.
<instagibbs>
probably not just restricted to LN long-term, only use blockchain as adjudicator with non-cooperative counter-parties.
<kanzure>
"And you can't scale without scaling bandwidth (modulo sipa's talk)." this seems to be false
<gavinandresen>
We seem to get half-baked sharding scheme ideas once every four months or so; my standard response is "go code up a prototype, complete with wallet, and then come back"
<instagibbs>
Make sure to relate any sharding to the SCP/Treechains/EthereumSharding(? no idea what they called it) ideas. If you could even summarize the space in a writeup that may be helpful
<gavinandresen>
Bandwidth is absolutely not the bottleneck to scaling bitcoin right now, even an average home internet connection has plenty of bandwidth assuming just implementing the very lowest-hanging-fruit for optimizing bandwidth usage
<instagibbs>
this is -wizards, and is being used to help clarify claims
<instagibbs>
:)
<gavinandresen>
I was responding to "why hasn't there been more work on sharding solutions"
<instagibbs>
ah
<instagibbs>
+1
<gavinandresen>
Solutions that reduce latency of new block announcements and mitigate selfish mining are big priorities right now, in my humble opinion
<coinoperated_tv>
braiding, sharding, sooner or later the transaction domain will have to be separated into largest practical signal domains
<gavinandresen>
I'm not familiar with the term "largest practical signal domains"
<kanzure>
huh?
<kanzure>
no way to have consensus over bandwidth at a protocol-level, unfortunately.
<kanzure>
(my "huh" is unrelated- was to signal whowhats)
<gavinandresen>
I seem to also dimly remember people saying that something like Google would be impossible way back when... (we'll HAVE to shard the Internet, no possible way anybody could search the entire thing....)
<kanzure>
google is a centralized system, and i think that it makes sense that you can just keep adding hardware and run mapreduce over it.
NewLiberty has joined #bitcoin-wizards
<coinoperated_tv>
google can live with a longer settlement window than 10 minutes though, no?
<kanzure>
what were the actual google concerns?
<kanzure>
to be fair, you still can't search the entire internet. there's a large quantity of pages that are simply inaccessible through the search interface. but why would google advertise that? that's not in their favor.
<coinoperated_tv>
even if internally google is a closed decentralized system, i don't think they have a requirement to have consistency every 10 mins, but i dunno, maybe interally they do
<gavinandresen>
kanzure: but the Internet is still decentralized. And if Google went away tomorrow, we'd still have Bing and duckduckgo and blekko and ... I dunno, a hundred others?
<coinoperated_tv>
seems unlikely
<kanzure>
duckduckgo uses bing, so that doesn't count
<instagibbs>
gavinandresen, pushing the computation/resources to the endpoints allows us closer to google. One reason I really like LN, and even though impractical today, treechains.
DougieBot5000 has joined #bitcoin-wizards
<gavinandresen>
Sure, I like the Poon/Dryja bathtub of too much centralization of what-is-needed-to-be-a-fully-validating-node on one side and too much centralization of number-of-keys-that-can-spend-on-the-main-chain on the other.
<coinoperated_tv>
laregst practical signal domain is the largest graph that can become consistent in a timely manner (timely being however long humans can tolerate waiting for it wrt their practical needs for the graph) given physical boundaries like rtt and speed of light
<kanzure>
gavinandresen: i think the argument for "the internet is decentralized" is "regulators don't have the capacity to interfere with all of the existing peering agreements" or something? i haven't seen anyone elaborating on that, which could be useful. (you and i have discussed internet before, but not in any relevant depth)
<instagibbs>
now we just get to endlessly argue about the shape of bathtubs rather than color of bikeshed :)
<kanzure>
we could do a graph/bikeshed color theorem of some kind, if that would make it seem more topical
<nsh>
sometimes bathtubs result in eurekas
JackH has joined #bitcoin-wizards
<nsh>
(sometimes toes get stuck in taps)
<gavinandresen>
coinoperated_tv: thanks, is that term from academic computer science networking or some other field?
<gavinandresen>
coinoperated_tv: And to save me some googling: what IS the largest practical signal domain for the entire Internet?
<kanzure>
"largest graph"- in this context though it's not the size of the graph that matters.....
<kanzure>
(because sybil)
<coinoperated_tv>
no just from casual reading in complex adaptive systems, i.e. the quark and the jaguar, santa fe institute blog, gmu complexity studies papers. readily admit its little more than a slightly informed hunch
<gavinandresen>
coinoperated_tv: ... to be more specific: if the graph is the entire Internet, with Internet rtt and speed of light, what is the consistency time?
<gavinandresen>
If it's less than a minute or so, it seems to me there's not a whole lot of reason to worry about sharding.
<coinoperated_tv>
gavinandresen: i think consistency time needs to be modeled in graduated degrees from most ideal case (simple consistency assuming everyone plays fair and has equal mean link latency, no cross-validation needed) down through what we actually have to deal with. But ultimately settlement time is a human factor, how long to wait is too long for practical use cases
CubicEarth has joined #bitcoin-wizards
wizkid057 has quit [Ping timeout: 272 seconds]
<bsm117532>
kanzure: I still don't understand your argument WRT bandwidth...
<coinoperated_tv>
10 mins, if i understand properly, is the target right now? And this seems short enough for most uses, i guess LN covers this?
ttttemp has quit [Remote host closed the connection]
ttttemp has joined #bitcoin-wizards
<instagibbs>
coinoperated_tv, you need block propagation to be << average block emission time
<instagibbs>
otherwise bad convergence
<bsm117532>
I don't know if I can make any comparison to treechains -- it seems insufficiently fleshed out. I've spoken to petertodd about it briefly, but it needs more work.
<instagibbs>
even in optimistic cases
<gavinandresen>
Lightning network MIGHT be the solution for instant mostly-trustless transactions... I don't think we know if people will be willing to park money in payment channels to get the benefits.
<instagibbs>
bsm117532, I am speaking at a high level even. For example, in SCP/EthereumSharding you are expecting validators to not lie a sufficient amount
wizkid057 has joined #bitcoin-wizards
<kanzure>
bsm117532: your claim is that you can't scale without scaling bandwidth. my argument is that this is wrong because we have evidence to the contrary. additionally, we have evidence of how both miners and nodes can't download fast enough at some bandwidth limit vs. data size. thus you are simply wrong....
CubicEarth has quit [Remote host closed the connection]
<instagibbs>
treechains is saying "don't trust miners"
<instagibbs>
aside from ordering of data
<coinoperated_tv>
gavinandresen: is LN sharding by another name? I do not mean to stir up controversy here, but some agreement on what terms like sharding means, in the abstract at least, is lacking generally.
<gavinandresen>
treechains are a buzzword, really
<kanzure>
LN is not really sharding; it's more like adding restrictions around otherwise undefined behavior of zero-conf transactions
<instagibbs>
coinoperated_tv, sort of? It's not sharding the chain state though.
<kanzure>
and also, keeping most of those zero-conf transactions to themselves
<instagibbs>
when we say sharding I'm assuming we are sharding the chain's ledger state somehow
<gavinandresen>
instagibbs: +1
<bsm117532>
kanzure: bitcoin's bandwidth usage is extremely modest. The real problem is downloading large blobs quickly and verifying it quickly. A solution is to spread the 10-minute spike of downloading a block over the entire 10 minutes by using a smaller faster layer. Then you're downloading a few kb at a time and verifying it continuously, rather than the spiky 10-minute way. In that sense my proposal *allows* us to s
<kanzure>
bsm117532: your irc client got cutoff at "my proposal *allows us to s".
<bsm117532>
Large 1MB blobs have problems downloading because of latency, routing, and bufferbloat.
<kanzure>
that's a local optimization; i'm confident we can solve those problems.
<bsm117532>
sorry: In that sense my proposal *allows* us to scale to the available bandwidth.
<kanzure>
my disagreement with you is "And you can't scale without scaling bandwidth (modulo sipa's talk)."
<bsm117532>
kanzure: I am too. This is a solution. ;-)
<kanzure>
not about local block propagation optimizations
<kanzure>
gavinandresen: btw have you used rusty's data corpus?
<coinoperated_tv>
instagibbs: ok, fair enough. so off-chain state maintenance is not sharding but something a step closer to decoupling of state?
<gavinandresen>
kanzure: no, not yet-- I keep getting distracted by Drama and Controversy...
<instagibbs>
coinoperated_tv, I think of it as "using blockchain space as adjudication of smart contracts" or something
<instagibbs>
set up smart contract, then do normal negotiation off-chain
<bsm117532>
kanzure: I think that statement is accurate. We're using like 3kb/s (averaged every 10m). We can't go faster because of protocol rules. Doubling the transaction rate means 6kb/s, and there's no way around that, modulo compressing the transaction format a la sipa.
<instagibbs>
only when the contract is violated do you need to tell others, aka close channel
<kanzure>
bsm117532: what does that have to do with "you can't scale without scaling bandwidth"?
<instagibbs>
bsm117532, last time I try to explain this: Your proposal, as well as Bitcoin proper, are bound to the "everyone must know everything" paradigm. Which today can't scale 1000x.
<gavinandresen>
bsm117532: you're poking a pet peeve of mine when you say "protocol rules" . Do you mean p2p network protocol rules, consensus protocol rules ????
<bsm117532>
instagibbs, kanzure: "everyone must know everything" is clearly a scaling limit of bitcoin that must be solved. But even keeping that rule we can't scale because larger blocks generate more orphans. My proposal solves that.
<bsm117532>
gavinandresen: I mean consensus rules.
lmatteis has joined #bitcoin-wizards
<gavinandresen>
bsm117532: ... then it would be much clearer if you said 'the 1MB blocksize limit'
<bsm117532>
gavinandresen: Okay. You're right "protocol rules" was incorrect in the above sentence.
<gavinandresen>
"everyone must know everything" is also a pet peeve of mine....
<gavinandresen>
Today, not everybody using bitcoin knows everything, so on the face it is clearly a false statement....
<kanzure>
i think he means "see all ze transactions"
<instagibbs>
see and validate
<kanzure>
e.g. in the whitepaper "The only way to know is to see everything"
<coinoperated_tv>
@instagibbs: my hunch is that it can never happen, there's a wall ahead where serial Tx convergence above a certain frequency limit just isn't possible on a global scale, no matter what shortcuts are developed.
<coinoperated_tv>
but I'll shup and not pretend to be more than the interested observer that I am
<gavinandresen>
The vast majority of people using bitcoin do not see all the transactions... I agree that all fully-validating nodes need to see all transactions under the consensus rules we're using today.
<kanzure>
coinoperated_tv: shortcuts like centralization should make that possible.
<kanzure>
coinoperated_tv: i also think there are promising long-term proposals that use cryptography magic to make that possible.
<instagibbs>
It's short-hand for: I want to use the full security of Bitcoin, and I'm compelled to know everything.
<gavinandresen>
It is loose talk like "bitcoin can't scale because everybody needs to see all the transactions" that trickles out from -wizards and misleads non-wizards into thinking things that just simply aren't true.
<kanzure>
nobody in here says bitcoin can't scale because of that requirement
<coinoperated_tv>
@instagibbs: some days ago i asked how much security is "enough" security, i think the reply I got was "all of it, or else..."
<kanzure>
perhaps they say "can't scale in the traditional ways", which is valid
<gavinandresen>
Yes, I know... we are all -wizards.
<kanzure>
no that's not what i meant :p
<gavinandresen>
I just get tired of trying to fix misperceptions that I think are made worse when I see experts being kind of sloppy
dEBRUYNE has joined #bitcoin-wizards
<instagibbs>
What language would you prefer
<gavinandresen>
Instead of "everyone" : fully validating nodes
<instagibbs>
Oh, easy fix.
<gavinandresen>
Instead of absolutes like "can't scale" be more precise: "cannot scale without sacrificing some amount of trust" (for example)
<instagibbs>
I was anthropomorphizing myself as a fully validating node
<gavinandresen>
or "cannot scale without sacrificing some amount of centralization"
<kanzure>
"cannot scale in the traditional way by increasing direct throughput without sacrificing decentralization and trustlessness and valuable properties of the bitcoin financial asset"
<kanzure>
whoops let me edit
<kanzure>
"cannot scale in the traditional way by increasing direct throughput without sacrificing some decentralization and trustlessness and valuable properties of the bitcoin financial asset"
<gavinandresen>
instagibbs: yeah, I tend to do that too-- it's a bad habit I need to break, it confuses non-experts who don't think of themselves as computers
<gavinandresen>
("so you validate the transaction then send it to your peers..." "Wait, you mean there are people validating bitcoin transactions somewhere in sweat shops in china?" )
<kanzure>
gavinandresen: one of my thoughts about this is that over time bitcoin is going to increasingly be people's first introductions to: programming, open source software development, p2p, network protocols, security, cryptography
<kanzure>
so over time we will have waves and waves of newbies to some of those fields
<kanzure>
and eventually, in best case scenario, bitcoin might one day be people's first introduction to money...
<coinoperated_tv>
@kanzure ty for link, good one as usual
<instagibbs>
I heard chinese coal miners validate the bitcoin network
<gavinandresen>
I heard it was reindeer in iceland
<kanzure>
instagibbs: that one... might be true.
<kanzure>
(no reason that bitcoind would be incompatible with computers owned by chinese coal miners)
<coinoperated_tv>
it's the planck constant
<gavinandresen>
kanzure: from what I hear, it's mostly excess hydro power. Coal is easy to turn on/off, so there's no excess.....
Quanttek has quit [Ping timeout: 246 seconds]
<kanzure>
i think you are talking about miners
<instagibbs>
I was more poking fun at people who think miners make sure the chain is valid(for others)
<instagibbs>
i guess they do for spv... until fraud proofs. doh
<kanzure>
right... miners != bitcoind for the most part.
<TD-Linux>
coal is actually one of the harder ones to turn on and off. hydro is the easiest (and also has built in energy storage!)
<gavinandresen>
TD-Linux: really? I believe that on short timescales (e.g. diverting water takes a minute, shutting down a coal burner takes... I dunno, an hour?), I was thinking long timescales...
<gavinandresen>
And hydro is free power up to whatever the dam capacity is. You gotta buy coal....
hashtagg_ has quit [Read error: Connection reset by peer]
<TD-Linux>
right, well maybe if they screw up managing the hydro buffer there would be excess (and I think china has been building a lot of hydro recently)
<TD-Linux>
and hydro is really cheap, not only because of free fuel, but also because it's mechanically simple and low maintenance.
<instagibbs>
I wonder how much of the hashing is done on hydro today
<TD-Linux>
so it's likely that they are using hydro power for the cost (the AWS region in oregon is the cheapest for this reason!) but I wouldn't call it *excess*
<bsm117532>
gavinandresen: I understand your frustration, you've been doing this longer than most. It's an inevitable fact that new people are going to come in and try to learn, and get it a little bit wrong, over and and over and over. :-/
<bsm117532>
See also AOL users jumping on the internet in 1994...
<gavinandresen>
instagibbs: a large percentage, I think. The hashing hotspots are china and eastern washington state, which are both hydro. And iceland, which is geothermal.
<TD-Linux>
yeah iceland is pretty neat. drill a hole, get free high pressure steam.
<gavinandresen>
instagibbs: mining is a zero-sum game, of course, so if a miner finds a spot with really cheap electricity they have an incentive not to advertise it
TBI has joined #bitcoin-wizards
TBI_ has quit [Ping timeout: 240 seconds]
ThomasV has joined #bitcoin-wizards
<bsm117532>
If we can decentralize mining by utilizing braids/beads with lower difficulty targets, or validating nodes publishing weak blocks, more mining will happen at the edges where the node operator is subsidizing his node for other reasons.
Erik_dc has joined #bitcoin-wizards
hashtag has joined #bitcoin-wizards
<gavinandresen>
bsm117532: ... I think you're just saying more miners should use p2pool...
_biO_ has quit [Remote host closed the connection]
dave4925 has joined #bitcoin-wizards
bendavenport has joined #bitcoin-wizards
mkarrer has joined #bitcoin-wizards
justanotheruser has quit [Read error: Connection reset by peer]
justanotheruser has joined #bitcoin-wizards
tachys_ has joined #bitcoin-wizards
zooko has joined #bitcoin-wizards
bsm1175321 has joined #bitcoin-wizards
bramc has joined #bitcoin-wizards
matsjj has quit [Remote host closed the connection]
Yoghur114_2 has joined #bitcoin-wizards
<coinoperated_tv>
@gavinandresen speaking of free electricity, many colo facilities sell power in 20A blocks. You pay for the whole 20A whether you use it all or not. I wonder if a significant fraction of hashing is from datacenter tenants monetizing these unused power slices.
tromp has joined #bitcoin-wizards
dEBRUYNE has quit [Ping timeout: 256 seconds]
tromp has quit [Ping timeout: 240 seconds]
liteIRC_ has joined #bitcoin-wizards
liteIRC__ has joined #bitcoin-wizards
zooko has quit [Ping timeout: 240 seconds]
liteIRC___ has joined #bitcoin-wizards
liteIRC__ has quit [Read error: Connection reset by peer]
zooko has joined #bitcoin-wizards
liteIRC_ has quit [Ping timeout: 240 seconds]
mrkent has joined #bitcoin-wizards
liteIRC___ has quit [Ping timeout: 240 seconds]
liteIRC_ has joined #bitcoin-wizards
zooko has quit [Ping timeout: 240 seconds]
liteIRC_ is now known as zooko
liteIRC_ has joined #bitcoin-wizards
NewLiberty_ has joined #bitcoin-wizards
Joseph__ has joined #bitcoin-wizards
zooko has quit [Ping timeout: 256 seconds]
liteIRC_ is now known as zooko
NewLiberty has quit [Ping timeout: 272 seconds]
eudoxia has joined #bitcoin-wizards
NewLiberty_ has quit [Ping timeout: 250 seconds]
zooko has quit [Remote host closed the connection]
bit2017 has joined #bitcoin-wizards
bit2017 has quit [Ping timeout: 240 seconds]
fluffypony has quit [Excess Flood]
Guest71642 has joined #bitcoin-wizards
Guest71642 is now known as fluffypony
fluffypony has quit [Changing host]
fluffypony has joined #bitcoin-wizards
jgarzik has joined #bitcoin-wizards
jgarzik has joined #bitcoin-wizards
bramc has quit [Quit: This computer has gone to sleep]
oldbrew has joined #bitcoin-wizards
justanot1eruser has joined #bitcoin-wizards
justanot1eruser has quit [Client Quit]
justanotheruser has quit [Quit: Lost terminal]
Quanttek has joined #bitcoin-wizards
justanotheruser has joined #bitcoin-wizards
mrkent_ has joined #bitcoin-wizards
RoboTeddy has joined #bitcoin-wizards
<bsm1175321>
gavinandresen: in many ways what I'm proposing absorbs p2pool into bitcoin. I've talked with a couple people about p2pool and am going to pursue testing these ideas there. p2pool users take a 5% hit due to their orphan rate, and braids can make that zero.
<gmaxwell>
P2pool has virtually no orphan rate; it's difficult to measure now because the hashrate was so low, but back when it was about 1% of the network it clearly had the lowest orphan rate of any namable pool.
<gmaxwell>
It has integrated efficient relay similar to the relay network client that causes p2pool nodes to very rapidly announce blocks from all over the network at once.
<bsm1175321>
Yes you can pretend orphaning doesn't happen with a very fast network. In the limit that your network is infinitely fast, there's no orphaning. Braids have no orphans ever though.
<gmaxwell>
My income over on p2pool is about 110% expected.
<gmaxwell>
I think you were talking to someone who was just profoundly ignorant about how p2pool works.
<gmaxwell>
which isn't uncommon, sadly.
<bsm1175321>
Does p2pool not have orphans? Admittedly I haven't looked at p2pool at all yet.
<bsm1175321>
I understood it was just another blockchain with 30s blocks.
<gmaxwell>
bsm1175321: which is used to measure users proportional hashrate over a several day window, nothing else.
<bsm1175321>
So orphans don't affect your payout?
<bsm1175321>
You just get a slightly worse measurement?
<gmaxwell>
relative stale shares compared to other people does; but they also reduce the bitcoin income for the pool.
<gmaxwell>
so if it was completely insensitive you could mine on a huge delay, and your contributions would be worthless (since your work is unlikely to result in a successful block), but you would still get paid.
<bsm1175321>
It sounds like you're verifying my 5% hit statement in bitcoin income. (?)
atgreen_ has quit [Ping timeout: 272 seconds]
<gmaxwell>
bsm1175321: I refuted it above with the strongest possible language I could use without being insulting.
<bsm1175321>
e.g. the pool earns 5% less because 30s/600s=5% and that's how often the pool is mining on a stale block, no?
<bsm1175321>
gmaxwell: let's be civil.
<maaku>
bsm1175321: why would the pool be mining on a stale block for 30s?
<gmaxwell>
bsm1175321: I'm sorry, you have a profound misunderstanding which I don't know how to correct. Someone else will have to take a stab at it.
<bsm1175321>
*sigh* I'll go read about p2pool.
<gmaxwell>
bsm1175321: go talk it through with maaku
<maaku>
eh, sorry, shouldn't have said anything (4am here)
<gmaxwell>
hah
<gmaxwell>
::sigh::
<gmaxwell>
bsm1175321: the processes are completely unrelated. You can't just divide random numbers.
<bsm1175321>
Dunno, some people liked my ideas and said p2pool. That's all I know at this point.
<gmaxwell>
there is no relationship between shares and blocks (well other than every block p2pool finds is also a share)
eudoxia has quit [Quit: Leaving]
<bsm1175321>
So I guess it's impossible that p2pool accurately represents the hashpower in the *last* 10 minutes, and constructs an accurate coinbase for it. At best it would be 5% off, if I started mining at the beginning of the interval.
<bsm1175321>
So it must assume everyone has constant hash rate, and averages over a longer time interval to determine shares....
<maaku>
bsm1175321: your p2pool node is constantly watching the bitcoin network and generating current work. as soon as you hear about a new block, you switch to it
<maaku>
bsm1175321: the interblock interval of p2pool has absolutely no corrolation at all with stale rates...
<gmaxwell>
bsm1175321: yes, it has a several day long rolling window... doesn't need to be accurate.
<bsm1175321>
I see.
<bsm1175321>
And I can't mine my own bitcoin block because then I would be unable to report shares (which require a coinbase provided by p2pool).
<gmaxwell>
well you could, but you wouldn't get paid by p2pool. :)
<bsm1175321>
See gmaxwell only took a few minutes to correct my misunderstanding without being insulting. :-P
<bsm1175321>
BTW I think gmaxwell and sipa deserve awards for engaging with all comers on IRC. I know it must be frustrating.
<Lightsword>
bsm1175321, I think the bigger issue with p2pool is that antminers have problems with it
<katu>
Given the pattern of miners changing pools, I suspect a lot of miners are not fond of p2pool now because it lacks sleek UI with graphs
<katu>
along with its history of being less profitable initially
<bsm1175321>
Lightsword: tough luck. You want a faster network, you have to switch mining hashes faster. I hope Bitmain fixes this. I talked about exactly this issue with several people at Scaling Bitcoin.
<gmaxwell>
katu: it has really nice graphs
<gmaxwell>
better than any other pool, in fact.. I'm not aware of any that will chart your latency, for example.
Joseph__ has quit [Ping timeout: 272 seconds]
<gmaxwell>
katu: it was never less profitable.
<Lightsword>
bsm1175321, problem is the majority of the network runs on antminers, probably over 60%
<katu>
gmaxwell: oh? so that side improved too. so it's all just historical prejudice/ignorance, just like i displayed now?
<bsm1175321>
katu: I had that misunderstanding too...not sure where it came from...
<gmaxwell>
katu: yea. a lot of people think it's less profitable for unfortunate reasons;... also it just doesn't work with some hardware.
<gmaxwell>
Because there is hardware with really embarassing latency that just can't be used with a 30s chain.
<katu>
gmaxwell: it was definitely producinga flurry of orphans when pool operators ddosed powerful p2pool nodes which are obviously quite exposed :>
<Lightsword>
gmaxwell, there is also a discard bug in antminers in the cgminer driver
<gmaxwell>
katu: since before you ever heard of p2pool many of the miners on it have maintained a seperate dark topology.
<gmaxwell>
(specifically to avoid being dos attack vulnerable)
<oldbrew>
very confusing sometimes when rejected blocks are high
<oldbrew>
goes nutz when you find a block
<Lightsword>
share stale rates also depend a lot on your relative latency with other p2pool nodes
<katu>
gmaxwell: are not there limits to it? i thought the progression in variance is linear. ie you need at least 0.1% or so of global hashrate to shoot down variance to some sane number?
<gmaxwell>
There is just a lot of confusion that arises from people who think sharechain performance == blockchain performance. Also, a lot of people have used "p2pool" via third party front ends instead of running it. and these front ends sometimes rip people off.
dEBRUYNE has joined #bitcoin-wizards
<katu>
gmaxwell: meaning, dark f2f networks are not suitable for the small guy
<katu>
but only for couple of industrial mining ops who cooperate?
<gmaxwell>
katu: they're not disconnected from the rest of p2pool the traffic is flooded.
<gmaxwell>
but it means that dos attacking public p2pool nodes doesn't need to impact most of p2pool's hashrate, so there is little to no gain to do it.
<gmaxwell>
and I can only think of two instances where p2pool nodes were being dos attacked and in both cases the attackers quickly gave up.
<katu>
gmaxwell: well, my reasoning is that with tiered, ddos resistant infrastructure you inevitably need someone to have incentive to invest into ddos proof infrastructure
<Lightsword>
gmaxwell, were people actually DoS’ing p2pool nodes a lot, most smaller mining pools rarely get hit AFAIK
<katu>
gmaxwell: with traditional pools, its the pool fee. but with p2pool?
<bsm1175321>
ddosing large centralized pools must be more effective...
* katu
does not believe in altruism
<Lightsword>
katu, a lot of ddos mitigation relies on misdirecting attackers so the techniques aren’t always revealed
<gmaxwell>
katu: it's very hard to dos p2pool, you're attacking a cloud... and indivigual members have self interest to avoid being dosed generally.
<katu>
Lightsword: misdirecting? you mean dns falseflags? :)
<Lightsword>
katu, there are some other tricks
<gmaxwell>
katu: "tiered ddos resistant infrastructure" is a trapping of centeralized systems.
<bsm1175321>
Wouldn't it be even better if every mining bitcoin node was a p2pool node? ;-)
<katu>
gmaxwell: tiered is what it boils down to from external PoV for peer cloud with dark links
<Lightsword>
katu, a lot of the tricks are really specific to stratum protocol
<katu>
gmaxwell: there are obviously simple ways to foil naive attackers (like forced segregation where peer pairs communicate only if hash of their ips share a byte)
<gmaxwell>
katu: there is no special cost to this. you turn off advertisement and manually add-node somewhat trusted peers.
bramc has joined #bitcoin-wizards
<katu>
gmaxwell: that way you get segregated tiers
jgarzik has quit [Quit: This computer has gone to sleep]
<katu>
if its done manually and there is not flux its even more trivial to dos
<katu>
gmaxwell: but agreed, most of attackers do not bother
<gmaxwell>
katu: the attackers never learn about it.
tromp has joined #bitcoin-wizards
<gmaxwell>
it's not visible; I think you're stuck imagining p2pool as a centeralized service or something.
<katu>
gmaxwell: depends. are individual miners self-identifying in the shares they submit, or not?
<gmaxwell>
... what purpose would that serve? advertising so dos attackers could attack them?
<gmaxwell>
No of course not.
Newyorkadam has joined #bitcoin-wizards
Newyorkadam has quit [Client Quit]
NewLiberty has joined #bitcoin-wizards
tromp has quit [Ping timeout: 272 seconds]
<katu>
gmaxwell: well last time i remember (2012) there was a single reward address specified to the proxy.
<katu>
if thats fixed, i cant think of any other way :)
<Lightsword>
katu, people using a proxy in front of p2pool?
<gmaxwell>
katu: what do reward addresses have anything to do with dos attacks?
<gmaxwell>
(though even in 2012 you could provide addresses in the miner url; there is also multiple payout addresses and rotation for monetary privacy)
GGuyZ has joined #bitcoin-wizards
Quanttek has quit [Read error: Connection reset by peer]
DigiDreamer has quit [Remote host closed the connection]
Quanttek has joined #bitcoin-wizards
zooko has joined #bitcoin-wizards
<katu>
gmaxwell: 1. connect to all sharechain nodes 2. look where share submission is propagated to soonest. It's no different from uncloaking classic pool upstreams.
<katu>
*the soonest
<Lightsword>
hmm, it may be easier actually, regular pools are on relay network which makes tracing nodes somewhat more difficult
eudoxia has joined #bitcoin-wizards
<katu>
but as long each share carries no identifying info per miner (for example, use different address per each share)
joecool has joined #bitcoin-wizards
zooko has quit [Ping timeout: 264 seconds]
<katu>
using different address makes the coinbases more spammy, does it not?
<Lightsword>
p2pool has a larger network footprint in general though so it’s harder to attack overall, more nodes you would have to take out
<gmaxwell>
katu: I explained above, that you do not need to be visible at all to the outside world.
<katu>
only those using fixed set of upstreams are easy to take out :)
<katu>
gmaxwell: if you're talking about private share-alt-chain, sure
<gmaxwell>
no. :(
<katu>
but otherwise your shares propagate to all nodes. the sooner observation = the node is closer to you
<gmaxwell>
katu: sure but that isn't very useful for targeting dos attacks.
<gmaxwell>
and hasn't been used.
<katu>
yet thats how some more sophisticated attacks roll :)
<katu>
on mainnet, not sharechain
<gmaxwell>
I am really tired of the relentless fud.
<gmaxwell>
katu: yea you can identify large centeralized miners on bitcoin; mostly because they hardly make more than a superficial effort to hide
zooko has joined #bitcoin-wizards
<Lightsword>
dos isn’t really the issue with p2pool IMO, the issues are that your profitability is dependent on your relative share stale rates in addition to hardware issues
<Lightsword>
most DDoS attackers are just script kiddies renting botnets anyways :P
<gmaxwell>
Lightsword: hardware issues are real; but I think it's just easy to fud and people have.
<katu>
Lightsword: the propdelay attacks are not usually for ddos though
<katu>
gmaxwell calls it fud, yet we've seen a whole /24's connected to virtually all nodes on some occasions
<katu>
they sure werent ddosing, but doing something :)
<Lightsword>
katu transaction traceing probably
<gmaxwell>
katu: thats chainanalysis; they're tracing people's transactions and selling the data.
<gmaxwell>
It takes pulling teeth to convince people of things like the fact that the 30s measurement chain has lots of stale shares doesn't mean p2pool has low profits. Like a half hour explination per person, and then the fudder just repeats the incorrect claim again.
<katu>
gmaxwell: yeah, thats unfortunate people dont realize the stales on the parallel subchain are unrelated.
<Lightsword>
gmaxwell, when the majority of deployed mining hardware however has known issues it’s certainly a problem, also in order to scale up you have to use a stratum proxy to load balance the connections
<gmaxwell>
Lightsword: Why would you need to do that? I've had nearly 1% of the network's hashrate on a single p2pool daemon at one point. Of course, you could just run multiple p2pool daemons.
Taurohtar has joined #bitcoin-wizards
<maaku>
"in order to scale up you have to use a stratum proxy to load balance the connections" is generally true regardless of the transaction selection code used...
atgreen_ has joined #bitcoin-wizards
<katu>
Lightsword: for larger mining ops, using a proxy is a given. i dont see any advantage each antminer doing anything more than being a dumb client to a pool or a sharechain proxy.
<Lightsword>
gmaxwell, from my understanding p2pools networking code has issues dealing with thousands of connections
<gmaxwell>
Lightsword: perhaps, though the only reports along those lines I've seen is from people exposing their downstream interface to the public and getting botnet loads and such.
<Lightsword>
gmaxwell, how many connections did you have when you had 1% of the network?
<gmaxwell>
Lightsword: p2pool scales the difficulty based on hashrate so the actual work it needs to do should be more or less constant; though certantly there could be stupidity with thousands of connections.
<gmaxwell>
Lightsword: not thousands. :)
<maaku>
Lightsword: why would you have thousands of simultaneous connections?
<gmaxwell>
maaku: because of running thousands of little mining devices off of it.
<katu>
dos.
GGuyZ has quit [Quit: GGuyZ]
<katu>
note that you dont need botnet to dos most of bitcoin these days
<maaku>
katu: ... why would you have an open p2pool port?
<gmaxwell>
katu: You might want to see someone about that dos obsession.
<Lightsword>
gmaxwell, it’s not an issue for difficulty so much as pushing out updates as fast as possible
<katu>
about 2000 packets per node to fd starve it momentarily
<Lightsword>
maaku, um that’s pretty normal for any large mining operation
<Lightsword>
katu, yeah, I think most pools have to override those OS limits, they fill up pretty fast
<maaku>
Lightsword: if you have thousands of devices simultaneously fetching work, I'm not sure why this is in particular p2pool's issue .. you run into the same thing with any work server
<gmaxwell>
maaku: well the p2pool one could well perform poorly for that case, it's not a case anyone working on p2pool has cared to address.
<Lightsword>
maaku, depends, stratum servers like ckpool are pretty good at handling very high connection counts
<gmaxwell>
and as far as I know no one ever reported a concern with it.
<Lightsword>
gmaxwell, most that have just put ckproxy in front of it AFAIK
<gmaxwell>
(I mean, p2pool is written in python so networking being stupid is a given)
<Lightsword>
yeah, python is single threaded for the most part
<katu>
Lightsword: indeed. also common sense hashlimit iptables and other. the problem is that the network code is not particularly robust and expects certain degree of expertise wrt networking.
<katu>
Lightsword: similiar to the "slowloris" situation some years back. only after these attacks became commonplace, so did httpd hardening to avoid it.
<gmaxwell>
And I think the comment suggests that they think you can't mine p2pool or will get paid less if you don't always have at least one share in the window.
<gmaxwell>
(maybe not; but even if the commenter doesn't think that-- other people reading it will)
<kanzure>
window means sharechain?
<gmaxwell>
kanzure: the tip of the sharechain used for payout calculations.
<bsm1175321>
Please someone correct them.
<bsm1175321>
There's so much ignorance and noise on reddit and bitcointalk that I rarely read them at all...
<gmaxwell>
If your hashrate is such that you don't always have a share in the sharechain; you'll not get paid for every block. But it's just variance, and not qualitatively different than 1 vs 2 shares. The expected income is still the same, and the variance doesn't have any sudden change or inflection at that point.
<bsm1175321>
I'm not qualified to correct this one...
<gmaxwell>
midnightmagic was fast.
<gmaxwell>
midnightmagic: you could edit to make it not accuse the poster of misunderstanding that. (I dunno if he does or not, but avoiding a dispute is helpful)
<gmaxwell>
might just be that he's defined for himself a "limit of acceptable variance" at that level. Why at that level? who knows.. why even would a 60GH/s miner care about variance at all who knows. :)
<kanzure>
wouldn't a share be required for payout?
<kanzure>
so if there wasn't a share, how could the payout be the same?
<kanzure>
i am clearly missing something.
<katu>
kanzure: if you have bad luck and miss the window, you dont get payout of course. but *overall* (ie when observing several days) you hit the window.
<gmaxwell>
kanzure: say your expected number of shares in the window is 0.5. This means you won't get paid whenever there isn't one, and when you do have one you'll get paid 2x as much as you deserve. The expectation is 0.5. (same story applies for 1 vs 2).
<kanzure>
ok so this is just about variance; having zero shares in the sharechain should always lead to zero payout.
<bsm1175321>
Given that p2pool only finds a block every 4 days, that's a lot of variance.
<gmaxwell>
and, of course, sometimes you'll get lucky and have 2 or 3 in the window and get paid massively more than you 'deserve'.
<gmaxwell>
bsm1175321: most of the time p2pool isn't finding a block, doesn't matter if you were in or out at those times.
<oldbrew>
sounds like wipe out or surf on a tidal wave
<kanzure>
i think midnightmagic should say "On average, the *expected* payout is the same regardless of previous recent shares" ?
<kanzure>
"payout is the same if there's no share in the p2pool sharechain" just still doesn't compute for me.
<bsm1175321>
But if on average I only generate 1/2 share per bitcoin block generated, and p2pool only generates a block every 4 days, I need 8 days on average to see any single payout. The lower my share the longer I have to wait for "average payout" to be a relevant term.
<gmaxwell>
bsm1175321: And, a "lot of variance" means that over the month you expected to make $3 and maybe you make $0.5 or $15. --- who cares? critically the same is true but with a slightly narrower window even if you reliably have 1 share in, vs 2.
<gmaxwell>
bsm1175321: there is no waiting. mining is a posson process.
<kanzure>
how do p2pool shares work? are there partial shares? is it difficulty-weighted?
<gmaxwell>
Every block is found infinitely far ahead of it's expected time, because the expected time is always 4 days from now.
<gmaxwell>
kanzure: share difficulty weighed.
<kanzure>
low-difficulty goes in?
<katu>
does not the p2pool divide solo variance by constant factor of 20?
<gmaxwell>
kanzure: miners set their own share difficulty to try to avoid having more than 1000 shares in the window. minimum share difficulty is adjusted to keep the coinbase transaction from being so large that it breaks stupid hardware.
<kanzure>
hm.
<gmaxwell>
katu: no, more like 600.
<oldbrew>
hard to see the global hash rate drop
<kanzure>
i was hoping the answers to my questions would help me rephrase midnightmagic's answer, but they haven't. :-)
DigiDreamer has joined #bitcoin-wizards
<gmaxwell>
katu: the measurement window is many blocks long, so the variance reduction is much larger than the ratio of share difficulties.
<kanzure>
i understand the concept of variance, and why it works out in the end. but i don't understand why you would expect someone to expect payout from not having a share in the sharechain.
<katu>
gmaxwell: yep, of course forgot about that :)
<bsm1175321>
gmaxwell: payout variance for a small miner (smaller than p2pool share difficulty) is much larger than the average payout (hence often zero) until they mine long enough. The smaller the miner the longer they will mine with zero payout. (That's what I meant by "wait")
<katu>
gmaxwell: it also depends on the total proportion of p2pool hashrate compared to rest. ie it would be much more than 600 if p2pool had large share of the pie, wouldn't it?
<gmaxwell>
bsm1175321: yes but this has no discontinuity. your variance just goes up linearly as your size goes down.
<bsm1175321>
gmaxwell: agreed. Mining with zero payout sucks, perhaps that's what's driving people away?
<gmaxwell>
katu: right now p2pool's variance is limited by its low hashrate. When it is sufficiently large, it's limited by the sharechain.
<katu>
i see
DigiDreamer has quit [Client Quit]
<bsm1175321>
So ultimately the restriction comes from limiting the number of outputs in the coinbase.
moa has joined #bitcoin-wizards
<kanzure>
do you mean "On average, the payout is the same independent of sharechain presence, because you might find the winning share yourself" ?
<gmaxwell>
bsm1175321: ... over $3/month expected? I doubt it. And when people talk about it; they always talk in terms of losing money, which isn't whats happening.
p15 has quit [Ping timeout: 250 seconds]
GGuyZ has joined #bitcoin-wizards
Erik_dc has quit [Remote host closed the connection]
dEBRUYNE_ has quit [Ping timeout: 256 seconds]
MoALTz has quit [Quit: Leaving]
GGuyZ has quit [Client Quit]
<bsm1175321>
gmaxwell p2pool says it keeps track of 8640 shares. That's $1.15 per block won or $8.60 per month (on average) to make it worth mining on p2pool.
<bsm1175321>
Smaller than that and you'll look at several zero p2pool payouts and wonder if something is wrong...
<gmaxwell>
oh jesus.
<bsm1175321>
gmaxwell: Just agreeing with you...
<gmaxwell>
bsm1175321: you're saying "to make it worth"
<gmaxwell>
as if you would be paid less if your expected income from mining was $3/month
<bsm1175321>
Ok rephrase. It's aways "worth it".
<bsm1175321>
But "Smaller than that and you'll look at several zero p2pool payouts and wonder if something is wrong..."
<midnightmagic>
#p2pool
<gmaxwell>
you just won't get paid in every block, if you have less.. you'll get paid in some and other others; when you mine on p2pool you are not paid by f2pools' blocks, but this doesn't make you lose income.
<gmaxwell>
and unlike other pools, p2pool provides great feedback if you're working or not; and I think thats actually worked against it.
<kanzure>
i think that midnightmagic's reply should be rephrased to say that "You don't lose expected income when the sharechain doesn't have your share. You lose only hypothetical income." or something...
liteIRC_ has joined #bitcoin-wizards
<kanzure>
saying that the payout is the same when there's no share in the sharechain, is just absurd... there's zero payout when a share is absent.
<bsm1175321>
I wouldn't use the word "lose" -- It's variance, that's all. Sometimes you'll have 2 shares.
liteIRC__ has joined #bitcoin-wizards
liteIRC___ has joined #bitcoin-wizards
zooko has quit [Ping timeout: 240 seconds]
liteIRC___ is now known as zooko
<kanzure>
haha yep i called it, the dude replied "If the miner does not have a share in the P2Pool share chain when P2Pool finds a block then payout=0"
rusty has quit [Ping timeout: 250 seconds]
liteIRC_ has quit [Ping timeout: 240 seconds]
<midnightmagic>
great, you modelled a moron's misinterpretation of arithmetic accurately
<katu>
so basically, the "unattractiveness" of p2pool is that the PPLNS window is too small?
<kanzure>
he was talking about share inclusion, wasn't he? you should instead tell him that he is talking about the wrong concept.
liteIRC__ has quit [Ping timeout: 256 seconds]
<AdrianG>
how can 21 inc possibly scale p2pool?
<bsm1175321>
21 is already 3%, bigger than p2pool.
<AdrianG>
is it even theoretically possible, what they are trying to do?
<AdrianG>
bsm1175321: yes, but they are saying they want to eventually have their devices mining on p2pool v2 or whatgever.
<AdrianG>
i cant see how can it be possibly scaled like that, in their use case, with tiny embedded devices.
<midnightmagic>
is this really -wizards topic material
<AdrianG>
p2pool scalability for embedded, distributed miners?
<bsm1175321>
midnightmagic: Yeah, I'll take it elsewhere.
<kanzure>
midnightmagic: so, i told you (above) that i understand the concept of variance. i get it. the way you phrased your reddit post was in a way such that someone could read "the payout (when p2pool finds a block) to you is the same even if you don't have a share included", which is simply not true.
<kanzure>
midnightmagic: i think it would be more productive to say that "your expected payout is not determined by p2pool sharechain share inclusion"
dEBRUYNE has joined #bitcoin-wizards
<kanzure>
or, rather, "p2pool finding a block when the sharechain does not include your share, does not determine your actual expected payout (barring some network failure with transmitting any shares you find)"
<bsm1175321>
kanzure: FWIW I can see why anyone who has not had Statistics 101 has difficulty understanding. Things we need to teach in grade school...
rusty has joined #bitcoin-wizards
<bramc>
Bitcoin's approach to payout can be summarized as 'Lotteries do a great job of saving bandwidth on rewards payouts'.
<bsm1175321>
a.k.a. it's not worth it to pay small miners frequently enough to make it worth their time => centralization. :-/
<bramc>
Only a certain amount of centralization though
<bramc>
What we have now is centralization far in excess of what's necessary to damped the variance in payouts.
matsjj has quit [Remote host closed the connection]
meZee has quit [Ping timeout: 256 seconds]
<bramc>
My cold gets one star. Would not have again.