sipa changed the topic of #bitcoin-wizards to: This channel is for discussing theoretical ideas with regard to cryptocurrencies, not about short-term Bitcoin development | http://bitcoin.ninja/ | This channel is logged. | For logs and more information, visit http://bitcoin.ninja
cluckj has quit [Ping timeout: 240 seconds]
cluckj has joined #bitcoin-wizards
kenshi84 has joined #bitcoin-wizards
dkings has quit [Remote host closed the connection]
CrazyLoaf has joined #bitcoin-wizards
dkings has joined #bitcoin-wizards
dkings has quit [Ping timeout: 240 seconds]
rusty2 has quit [Ping timeout: 248 seconds]
]GD has joined #bitcoin-wizards
c0rw1n has quit [Ping timeout: 240 seconds]
c0rw1n has joined #bitcoin-wizards
jtimon has quit [Ping timeout: 252 seconds]
kenshi84 has quit [Remote host closed the connection]
kenshi84 has joined #bitcoin-wizards
kenshi84_ has joined #bitcoin-wizards
kenshi84 has quit [Ping timeout: 240 seconds]
Ylbam has quit [Quit: Connection closed for inactivity]
d9b4bef9 has quit [Remote host closed the connection]
d9b4bef9 has joined #bitcoin-wizards
Noldorin has quit [Ping timeout: 255 seconds]
dkings has joined #bitcoin-wizards
kenshi84_ has quit [Remote host closed the connection]
kenshi84 has joined #bitcoin-wizards
dkings has quit [Ping timeout: 240 seconds]
rusty2 has joined #bitcoin-wizards
]GD has quit [Remote host closed the connection]
]GD has joined #bitcoin-wizards
Dizzle has quit [Remote host closed the connection]
Dizzle has joined #bitcoin-wizards
copumpkin has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
rusty2 has quit [Quit: Leaving.]
rusty2 has joined #bitcoin-wizards
NewLiberty has quit [Ping timeout: 256 seconds]
copumpkin has joined #bitcoin-wizards
c0rw1n_ has joined #bitcoin-wizards
c0rw1n has quit [Ping timeout: 240 seconds]
droark has joined #bitcoin-wizards
Alopex has quit [Remote host closed the connection]
Alopex has joined #bitcoin-wizards
CrazyLoaf has quit [Quit: Connection closed for inactivity]
pro has quit [Quit: Leaving]
NewLiberty has joined #bitcoin-wizards
dkings has joined #bitcoin-wizards
dkings has quit [Ping timeout: 258 seconds]
NewLiberty has quit [Ping timeout: 256 seconds]
Alopex has quit [Remote host closed the connection]
rusty2 has left #bitcoin-wizards [#bitcoin-wizards]
Alopex has joined #bitcoin-wizards
Dizzle has quit [Remote host closed the connection]
Dizzle has joined #bitcoin-wizards
kenshi84 has quit [Remote host closed the connection]
kenshi84 has joined #bitcoin-wizards
Dizzle_ has joined #bitcoin-wizards
Dizzle has quit [Ping timeout: 256 seconds]
AlineGomes has joined #bitcoin-wizards
Dizzle_ is now known as Dizzle
dkings has joined #bitcoin-wizards
dkings has quit [Ping timeout: 240 seconds]
kankles has quit [Quit: Leaving]
kankles has joined #bitcoin-wizards
reBrain has joined #bitcoin-wizards
legogris has quit [Remote host closed the connection]
reBrain has quit [Remote host closed the connection]
legogris has joined #bitcoin-wizards
iddo has quit [Ping timeout: 240 seconds]
iddo has joined #bitcoin-wizards
]GD has quit [Read error: Connection reset by peer]
harrymm has quit [Ping timeout: 245 seconds]
harrymm has joined #bitcoin-wizards
TheSeven has quit [Ping timeout: 258 seconds]
TheSeven has joined #bitcoin-wizards
Dizzle has quit [Remote host closed the connection]
kenshi84_ has joined #bitcoin-wizards
kenshi84_ has quit [Remote host closed the connection]
kenshi84 has quit [Ping timeout: 240 seconds]
Dizzle has joined #bitcoin-wizards
c0rw1n_ is now known as c0rw1n
kenshi84 has joined #bitcoin-wizards
kenshi84 has quit [Client Quit]
cannon-c has joined #bitcoin-wizards
Dizzle has quit [Quit: Leaving...]
bildeamer1 has joined #bitcoin-wizards
bildramer has quit [Ping timeout: 256 seconds]
Ylbam has joined #bitcoin-wizards
dkings has joined #bitcoin-wizards
dkings has quit [Ping timeout: 256 seconds]
forrestv has quit [Excess Flood]
forrestv has joined #bitcoin-wizards
Starduster has quit [Read error: Connection reset by peer]
Starduster has joined #bitcoin-wizards
vidjogamer has quit [Read error: Connection reset by peer]
vidjogamer has joined #bitcoin-wizards
Burrito has joined #bitcoin-wizards
edvorg has joined #bitcoin-wizards
BashCo_ has quit [Remote host closed the connection]
BashCo has joined #bitcoin-wizards
dkings has joined #bitcoin-wizards
BashCo has quit [Ping timeout: 258 seconds]
dkings has quit [Ping timeout: 248 seconds]
BitBully has joined #bitcoin-wizards
jannes has joined #bitcoin-wizards
d9b4bef9 has quit [Remote host closed the connection]
reBrain has quit [Read error: Connection reset by peer]
BitBully has quit [Client Quit]
kenshi84 has quit [Quit: Leaving...]
kenshi84 has joined #bitcoin-wizards
Jeremy_Rand[m] has quit [Remote host closed the connection]
instagibbs has quit [Ping timeout: 240 seconds]
laurentmt has joined #bitcoin-wizards
Chris_Stewart_5 has quit [Ping timeout: 252 seconds]
Jeremy_Rand[m] has joined #bitcoin-wizards
Chris_Stewart_5 has joined #bitcoin-wizards
testnetting has joined #bitcoin-wizards
laurentmt has quit [Quit: laurentmt]
<testnetting>
hey
<testnetting>
anyone here ... ?
<testnetting>
That day Voldemort logged on ..
<testnetting>
he didn't hide his IP adress ...
<testnetting>
Did one hide it on purpose ?
Chris_Stewart_5 has quit [Ping timeout: 255 seconds]
<kanzure>
tor exit node, yo
<kanzure>
no doxxing.
Chris_Stewart_5 has joined #bitcoin-wizards
<testnetting>
access.telenet.be/178.117.69. ...
<testnetting>
i bet - access.telenet.be/178.117.69 .... shows up before he supp. used the Tor exit node? Anyway to check ?
<kanzure>
do you need his identity for some reason?
<testnetting>
No - i know his identity.
<testnetting>
I am merely trying to find out how easy ore hard it is to find that ip address
<testnetting>
Do you remember that day ?
<kanzure>
what do you need
<testnetting>
Did you ask him if he was Satoshi ?
kenshi84 has quit [Quit: Leaving...]
kenshi84 has joined #bitcoin-wizards
<testnetting>
?
<testnetting>
Yes
<testnetting>
His reply ...
<testnetting>
I have to say no ?
<testnetting>
better -
<testnetting>
I have to say no.
<testnetting>
?
<testnetting>
True ore False ?
<testnetting>
Kanzure ?
<gmaxwell>
testnetting: please cut out the subject; it's invasive and weird and making me uncomfortable.
<testnetting>
It is important " to me " so someone can answer that question ?
testnetting was banned on #bitcoin-wizards by gmaxwell [*!*@gateway/web/freenode/ip.178.117.69.206]
testnetting was kicked from #bitcoin-wizards by gmaxwell [testnetting]
Chris_Stewart_5 has quit [Ping timeout: 252 seconds]
Chris_Stewart_5 has joined #bitcoin-wizards
PRab has quit [Read error: Connection reset by peer]
Pr0t3us has quit [Quit: Leaving]
JayDugger has left #bitcoin-wizards [#bitcoin-wizards]
aalex has joined #bitcoin-wizards
mmeijeri has joined #bitcoin-wizards
<mmeijeri>
I've been thinking about weak blocks some more. It would seem that with weak blocks there is no longer any need/point to using something like NG, is that correct?
jannes has quit [Ping timeout: 255 seconds]
AlineGomes has quit [Quit: Connection closed for inactivity]
BashCo has quit [Remote host closed the connection]
anon616 has quit [Remote host closed the connection]
BashCo has joined #bitcoin-wizards
davec has joined #bitcoin-wizards
anon616 has joined #bitcoin-wizards
Davasny has joined #bitcoin-wizards
Uglux has quit [Quit: Leaving]
reBrain has joined #bitcoin-wizards
atgreen has quit [Ping timeout: 260 seconds]
atgreen has joined #bitcoin-wizards
reBrain has quit [Remote host closed the connection]
forrestv has quit [Ping timeout: 252 seconds]
instagibbs has joined #bitcoin-wizards
PRab has joined #bitcoin-wizards
mol has quit [Read error: Connection reset by peer]
Chris_Stewart_5 has quit [Ping timeout: 256 seconds]
Chris_Stewart_5 has joined #bitcoin-wizards
Chris_Stewart_5 has quit [Ping timeout: 240 seconds]
rusty2 has joined #bitcoin-wizards
bitcoin-wizards0 has joined #bitcoin-wizards
<bitcoin-wizards0>
Test
bitcoin-wizards0 has quit [Client Quit]
bitcoin-wizards2 has joined #bitcoin-wizards
bitcoin-wizards2 has quit [Client Quit]
bitcoinbilly has joined #bitcoin-wizards
<bitcoinbilly>
Does everyone agree we will have to alter the bitcoin protocol ?
Chris_Stewart_5 has joined #bitcoin-wizards
bramc has joined #bitcoin-wizards
ipwn has quit [Ping timeout: 255 seconds]
<bramc>
Hey everybody
ipwn has joined #bitcoin-wizards
<kanzure>
what's up
<bitcoinbilly>
MimbleWimble sidechains integrated smart contracts script a new verification systen
<bramc>
On the topic of weak blocks: There are several different approaches one could take. A lot of it depends on whether you're assuming that blocks are full or not. Mostly weak blocks act as a form of compression
<bitcoinbilly>
' lighting network on top of that ..
<bramc>
When a weak block goes out it's safe to propagate it because there's enough work behind it that you don't have to worry about spam. A 'real' block can then be compressed later as a delta to a weak block, which will make it propagate faster
<bitcoinbilly>
a new concept I like to introduce - satoshi group flooder.
<bitcoinbilly>
This can be implemented after one payment channel settles
<mmeijeri>
I was wondering if NG had any remaining advantages over weak blocks.
<bitcoinbilly>
A satoshi group flooder is a automated system wich evenly distributes the in bitcoin transaction terms the flooding output in and from a payment channel. If
<bitcoinbilly>
If any
<bitcoinbilly>
If any in the current payment channel scheme ..
dkings has joined #bitcoin-wizards
dkings has quit [Remote host closed the connection]
<bitcoinbilly>
Is this already discussed ?
dkings has joined #bitcoin-wizards
dkings has quit [Remote host closed the connection]
<bitcoinbilly>
When the MimbleWimble chain enherates the bitcoin from the transaction these coins wil be used in unidirectional payment channels I'm trying to establish if these coins wil be held in a wallet deducted to a end user ore the MimbleWimble payment/fund group
Chris_Stewart_5 has quit [Ping timeout: 255 seconds]
jcluck has joined #bitcoin-wizards
<bitcoinbilly>
When the value of such a group doesn't change -meaning no mimblewimblebitcoin leave this group anyone in the group should and could use the coins ?
<bramc>
I was going to say that weak blocks increase time to commit a bit but, uh, NG does far more of that
<bitcoinbilly>
Sharing our Welt
<bitcoinbilly>
Welth ss
<mmeijeri>
My feeling is that NG, though interesting, is not as good as weak blocks.
dkings has quit [Remote host closed the connection]
cluckj has quit [Ping timeout: 256 seconds]
ratbaneb_ has quit []
<bitcoinbilly>
The only value leaving this system would be tax money witch one still has to pay of course. .
dkings has joined #bitcoin-wizards
<bitcoinbilly>
This must not be done bye the sharing our wealth system
<bitcoinbilly>
Of course :)
rusty2 has quit [Ping timeout: 240 seconds]
<bramc>
mmeijeri: That's my feeling as well, and why I never proposed anything like NG myself.
<bitcoinbilly>
Bitcoin is the way of the future haven't you heard?
<bitcoinbilly>
Smile
<kanzure>
i think bitcoinbilly == testnetting ?
<kanzure>
gonna kick
<bitcoinbilly>
yes baby kick me and be a fool
bitcoinbilly was kicked from #bitcoin-wizards by kanzure [bitcoinbilly]
<kanzure>
gladly.
Chris_Stewart_5 has joined #bitcoin-wizards
bitcoin-wizards9 has joined #bitcoin-wizards
jcluck is now known as cluckj
PaulCapestany has quit [Quit: .]
<bitcoin-wizards9>
We are dead serious! Has anyone thought about what I just described ? Sleep on it - unkick me tomorrow ... Goodnight
CubicEarth has joined #bitcoin-wizards
<bramc>
I didn't follow what was described
<kanzure>
i think it was the same user causing more trouble from earlier today
priidu has quit [Ping timeout: 256 seconds]
<bramc>
Anyone who wants to play along can now watch me make commits all with the comment 'bug fix' for the next month https://github.com/bramcohen/MerkleSet
priidu has joined #bitcoin-wizards
<mmeijeri>
Are weak blocks superior to having smaller but more frequent normal blocks? If so, what are the advantages?
CubicEarth has quit []
<bitcoin-wizards9>
If asking questions is causing trouble maybe we can stop developing bitcoin into what we immagenid it would be.
instagibbs has quit [Ping timeout: 252 seconds]
<kanzure>
what is your question
priidu has quit [Remote host closed the connection]
bildeamer1 is now known as bildramer
<bramc>
mmeijeri: Much lower orphan rate from weak blocks. More frequent smaller blocks makes for faster commit times and more orphans. Weak blocks make fewer orphans with slightly slower commit times
Newyorkadam has joined #bitcoin-wizards
<mmeijeri>
why doesn't the smaller size compensate for the shorter interblock interval?
<bramc>
Because the latencies which cause orphaning are not primarily caused by block sizes. Peter R is lying to you.
instagibbs has joined #bitcoin-wizards
<bitcoin-wizards9>
read the log - and remove the ban !
<Eliel_>
mmeijeri: also, once a large majority of miners are using weak blocks to coordinate their transaction selection, the weak blocks start having some value as weaker confirmations as well.
<mmeijeri>
right. as for orphan rates: I know that Rizun's claim is false, but I thought it was true for the block transmission mechanism as it was before compact blocks. Is that not the case?
<bramc>
Eliel_: They provide some confirmation of the block they're on top of, particularly if you let them be chained, but they don't resurrect zeroconf
<Eliel_>
bramc: I don't expect them to do that
<bramc>
mmeijeri: BlueMatt's backbone thing makes latencies very low as long as you use it verbatim. It's a fairly unpleasant point of centralization though.
<mmeijeri>
sure
<mmeijeri>
but still, it would be nice to try to narrow the gap between the general p2p transmission mechanism and RN/Fibre
<Eliel_>
bramc: although, I'm not sure if you're aware, but there are still services accepting zeroconfs. Even without weak blocks.
<bramc>
Yes weak blocks would be a good thing, particularly if they're soft forked in nicely
<bramc>
Eliel_: Not My Problem
<Eliel_>
bramc: I'm just trying to make the point that zeroconf isn't exactly dead yet.
<bramc>
It's undead
<Eliel_>
I expect lightning to take over for those services once we get the malleability problem dealt with once and for all.
<mmeijeri>
It would be nice if we could go back to 100kb blocks for a while after LN to see what would happen to the full node count
<BlueMatt>
bramc: if someone bothered to audit the thing I'd be perfectly happy with people using it in ip-multicast mode :p
<Eliel_>
mmeijeri: I'm not too hopeful about that, but it's possible.
<BlueMatt>
but, then, everyone would have to be on sprint's backbone
<bramc>
BlueMatt: The internet supports multicast?
<BlueMatt>
bramc: only if you're on sprint
<BlueMatt>
"internet"
* BlueMatt
is still confused why people love weakblocks....they really dont do all that much better than fibre for reasonable block sizes
<BlueMatt>
especially if you soft-forked in a fibre commitment (which is slightly infeasiable without putting it at the root of the merkle tree, but....)
<bramc>
What do you mean by fibre? Is that your backbone?
<BlueMatt>
its the software that powers the backbone, but it could be used p2p-ish
<BlueMatt>
the only reason i dont recommend that is lack of audit for the pile of code I wrote that no one has really ever looked at
<bramc>
Mostly weak blocks are more decentralized
<BlueMatt>
not really, though?
<Eliel_>
BlueMatt: what's the main difference between fibre and weakblocks?
<bramc>
If implemented properly weak blocks could be almost as performant as fibre with no centralized infrastructure at all
<BlueMatt>
weak blocks disadvantage those who cant reliably create weak blocks quickly
<BlueMatt>
eg if you cant create a weak block reliably within a few tens of seconds after new blocks then you're not able to select transactions
<BlueMatt>
and are forced to use whatever selection bigger miners do
<BlueMatt>
Eliel_: they're pretty fundamentally different - fibre is gonna stream everything at block-creation-time, whereas weak blocks are an attempt to frontload that
<bramc>
Uh... That depends on a lot of details. There are a few approaches
<BlueMatt>
Eliel_: for current block sizes streaming at line rate at block-creation-time is really not a problem
<BlueMatt>
if you want 100MB blocks, it would be moreso, but not for 1/2/4/10MB
<bramc>
Also the way you design weak blocks varies a lot depending on whether you're assuming blocks are full all the time and transactions are constantly getting replaced with higher value versions
<Eliel_>
ah right, not when a block is found but when you start attempting to find a block.
<BlueMatt>
bramc: maybe I'm missing some modern variants? last I spoke with gmaxwell on the subject I remained unconvinced
<BlueMatt>
(though, admittedly, I seem to be rather alone in that opinion)
<bramc>
There's also a reasonable argument that blocks should be propagated with just the headers at first so others can start mining off the new one immediately even before validation happens
<BlueMatt>
my concern is that weak blocks do not provide much advantage over fibre with a good fec (which is now being used) - ultimately if you want to select transactions which are not pre-relayed you're gonna have to send them
<kanzure>
what is the reasonable argument
<bramc>
I haven't walked through all the variants with you or gmaxwell yet. It's an interesting engineering problem.
<Eliel_>
BlueMatt: although, would you need to slap some kind of a PoW on to fibre eventually to limit the bandwidth use? That'd seem to remove the difference mostly.
<BlueMatt>
whereas with weakblocks you can only pre-relay transactions which have hashpower behind them, fibre you can pre-forward anything
<BlueMatt>
Eliel_: one idea is to have miners commit to each fibre packet with a merkle tree
<BlueMatt>
Eliel_: the current implementation does not do cut-through, ie it must receive the full block and check its sanity before relaying on
<BlueMatt>
(except in "trusted networks")
<BlueMatt>
sadly the commitment stuff is only realistically doable if you put it at the root of the bitcoin-tx merkle root
<BlueMatt>
otherwise it takes up just barely too many bytes to be practical on internet-sized packets
<BlueMatt>
bramc: I think most any protocol will do that (fibre does so, but really most stuff anyone designs ends up doing that at least by accident because you naturally send the header first)
<Eliel_>
BlueMatt: it seems to me that the only major difference in a p2p versions of each would be that fibre as you described doesn't come with a built-in PoW mechanism to limit network usage.
<BlueMatt>
Eliel_: nothing has a mechanism to limit inbound bandwidth, limiting outbound bandwidth is easy by just checking pow before you relay?
<BlueMatt>
(or are you asking about rate control, ie the thing udp makes you do manually?)
<Eliel_>
BlueMatt: no, just limiting relay bandwidth.
<BlueMatt>
wait, what are you asking about? I mean there's a few issues - not knowing how much data the other side needs, ie how much to send (you could come up with clever protocols here where you pairwise keep track of /roughly/ what your peers have and only send a small amount of fec)
<mmeijeri>
aren't weak blocks synergistic with Fibre?
<mmeijeri>
preconsensus means sketches can be smaller, no?
<mmeijeri>
and of course they do narrow the gap between the p2p network and Fibre
<mmeijeri>
or rather: sketches can remain small even as blocks grow
<BlueMatt>
slightly, yes...I mean weak blocks are another method of pre-forward, except that they're a bit more rigid
<Eliel_>
or, let's put it this way. With weakblocks, if you're receiving more blocks than you have processing power to verify, you can just pick the ones with highest PoW on them and check&relay them and the end result will be that the choices will be reasonably cohesive on the whole network.
<BlueMatt>
I'd be happy to see weak blocks combined with fibre (ie to allow miners to select txn which have low fees for pre-forward)
<BlueMatt>
but only weak blocks I think is shit
<BlueMatt>
Eliel_: oh, you super dont want to do that
<BlueMatt>
Eliel_: that highly encourages matching transaction selection across the network
<BlueMatt>
and forces smaller miners to just use the tx selection from larger ones
<mmeijeri>
precisely why would weak blocks only and no fibre be shit compared to just fibre or both fibre and weak blocks? (I'm all in favour of combining the two, just trying to understand the tradeoffs)
<BlueMatt>
of course I dont think thats neccessary - a) you dont need to validate weak blocks unless you care about latency (ie are a miner), b) you should be able to (mostly) trivially compress them - they're super duplicative
<BlueMatt>
mmeijeri: purely concerns over encouraging matching transaction selection
<mmeijeri>
right
<BlueMatt>
imagine you set your weakblocks difficulty pretty high, now you're kinda forced as a small miner to use the tx selection from larger miners
<BlueMatt>
which is shit
bramc has quit [Ping timeout: 260 seconds]
<Eliel_>
BlueMatt: I don't think there needs to be a set weakblocks difficulty.
<mmeijeri>
got it. so what's the latest thinking on the relative strength of weak blocks?
<BlueMatt>
lol, did I scare bramc off?
<BlueMatt>
mmeijeri: I think I have a unique viewpoint in not being a huge fan of weak blocks, so not sure what "the latest thinking" is
<Eliel_>
nodes just start dropping the weakest ones if they can't handle all of them for whatever reason.
<BlueMatt>
Eliel_: sure, but you want to make sure its pretty low
<mmeijeri>
sorry, I meant how much weaker do we want them to be, 10x, 100x?
<BlueMatt>
(the minium weak blocks requirement, that is)
<BlueMatt>
mmeijeri: look at it this way - if you're censored today on bitcoin, you can spin up some hashrate for relatively cheap that will confirm your transaction by mining a block once a day or every 2 days or once a week
<Eliel_>
BlueMatt: I don't think there really needs to be a minimum difficulty. Rather, node admins should be able to set a bandwidth limit for them and that'd lead to the difficulty.
<BlueMatt>
if we just used weak blocks (and actually had issues with propagation), you couldnt do that
<mmeijeri>
that's an interesting yardstick
<BlueMatt>
I havent run the numbers (someone bored and want to?) but I think even being able to mine your block once a day would be pretty hard on the above yardstick
<kanzure>
Eliel_: it's a consensus parameter, though (difficulty)
<Eliel_>
kanzure: for weak blocks?
<BlueMatt>
(assume you need to get a single weak block and then also, during the same block time, mine a full block)
<BlueMatt>
kanzure: only if you soft-fork in weak blocks
<BlueMatt>
which i think is a shit idea
<kanzure>
ok.
<mmeijeri>
soft fork it in? why isn't this outside the consensus layer?
<BlueMatt>
(again, probably not in the majority in my viewpoint there)
<mmeijeri>
it's just the network layer, right?
<BlueMatt>
well, or maybe am for consensus
<Eliel_>
BlueMatt: anyway, I do see the concern with the censorship resistance weakening if you need lots of PoW to get transactions included in any blocks.
<BlueMatt>
admittedly any reasonable weak blocks impl would have a differential encoding
<BlueMatt>
so it may not be a massive concern
<BlueMatt>
but my point, more, is that we have a great differential encoding in fibre/compact blocks already, and unless we're concerned with with miners being able to include transactions which do not have a high fee (ie are not otherwise propagating) then I dont see a ton of gain
<Eliel_>
although, wouldn't that just mean that you'd need to add bigger fees to get around censorship? Big enough to convince miners to include the transactions despite the increased orphaning potential
<BlueMatt>
potentially
<BlueMatt>
these stats are now out of date due to my being too lazy to fix the script given the fibre cuts in india/singapore, but take a look at http://bitcoinfibre.org/stats.html
<Eliel_>
although, if majority of hashpower colludes to censor transactions, they can just completely block them anyway.
<BlueMatt>
mostly how super flat the prop. times are
bitcoinbilly has joined #bitcoin-wizards
* BlueMatt
concludes propagation is a "solved problem"
<mmeijeri>
would that remain the case if blocks became significantly larger though?
<BlueMatt>
pretty mch
<BlueMatt>
much
<mmeijeri>
how do you know?
<BlueMatt>
would have to find a fatter link out of china, maybe
<bitcoinbilly>
Hello.
<BlueMatt>
because the amount of data usually used to transmit a block over the fibre network is almost always in the 10-20 packet range
<mmeijeri>
wouldn't that be a percentage of the block size?
bitcoin-wizards9 has quit [Ping timeout: 260 seconds]
<mmeijeri>
more or less fixed
<BlueMatt>
and when its not you can almost always divide the size of the differential encoding by the link speed and get the time-to-transmit plus a handful of ms
<BlueMatt>
nope, not at all
<BlueMatt>
its a function of compact block encoding size
<BlueMatt>
which is (mostly) disconnected from block size
<BlueMatt>
there are a few things we can do to better get transactions pre-forwarded across the p2p network
<mmeijeri>
you are basically trying to predict what the other side hasn't heard, no?
<BlueMatt>
nope
<BlueMatt>
you arent predicting anything
<mmeijeri>
ok, then I'm confused
<BlueMatt>
you're sending everything...it just so happens that the other side can reconstruct the instant it has all the transactions
<mmeijeri>
right
<BlueMatt>
and that means, roughly, the size of the differential encoding
<mmeijeri>
ah yes, but that's what I meant, the size of the differential encoding
<BlueMatt>
it turns out on the network today you pretty much never miss more than one or three transactions per block
<mmeijeri>
I'm not talking about packet loss
<BlueMatt>
especially if you include transactions you rejected as double-spends in your mempool
<mmeijeri>
I know you aren't trying to predict packet loss, the fec takes care of that
<BlueMatt>
well I'm saying you dont have to do anything interactive to predict missing txn
<mmeijeri>
but in the differential encoding aren't you sort of predicting what txs the other side doesn't have in its mempool?
<BlueMatt>
you stream everything and even if the other side has no txn in its mempool you still send the same encoding
<BlueMatt>
and it still works just as well
<Eliel_>
mmeijeri: you can send to the data in a form that will get you all the missing txn, no matter which ones you're missing.
<BlueMatt>
the encoding is disconnected from the txn missing on the other side
<BlueMatt>
it just so happens that you will reconstruct after you've received the number of packets which totals in size the same as the txn you're missing
<BlueMatt>
because the fec is good
<mmeijeri>
ok hang on, let me see if I'm getting this right
<Eliel_>
mmeijeri: you can basically just treat the missing txns like they're packet loss.
<Eliel_>
and send the fec data
<mmeijeri>
in fibre today, you are using fec to compensate both for missing txs and lost packets?
<mmeijeri>
ah great, that's what I was asking you a while back and you seemed to be saying no
<mmeijeri>
(already donated)
<BlueMatt>
ok, sorry, irc communication is hard
<mmeijeri>
ok, but that's pretty spectacular!
<mmeijeri>
I thought this was the hard part that had remained unimplemented
<BlueMatt>
yes, see stats page...95th percentile gets blocks around whole network in 18ms + speed of light
Peter__R has joined #bitcoin-wizards
<mmeijeri>
does this mean that if you receive incoming blocks from two peers simultaneously the are synergystic?
<BlueMatt>
so I'd call block propagation a solved problem on the technical level until we have much bigger blocks...needs solving on the social level still (getting peers on the public fibre network, getting audits on the fibre codebase so that we can get public connections using it, etc)
<Eliel_>
mmeijeri: as long as they aren't sending you identical data, yes.
<mmeijeri>
great
<BlueMatt>
mmeijeri: this is where the "trusted" thing come in from before
<BlueMatt>
fibre has two modes, one where that is true, one where it isnt
<mmeijeri>
I knew this was in Greg maxwell's sketch, but I didn't know it had been implemented yet
<BlueMatt>
in "private trusted network" mode, where you trust the data coming from everyone in your network, yes, they are synnergistic
<BlueMatt>
and fibre is designed to relay packets in funny order to help this
<BlueMatt>
sadly, the fec isnt designed to handle malicious or corrupted data
<bitcoinbilly>
image a closed system where we can interact extensively and share our accumulated welth. If al members of the system interact within the system the original value remains equal.
<mmeijeri>
I'm enormously impressed. great work!
<BlueMatt>
so in order to extend it to work in public networks with the same cut-through+synnergistic data properties, miners would need to commit to the fibre chunks
<mmeijeri>
I've been leveling up my old algebra skills to understand the maths behind it
<BlueMatt>
there was some private-chain project where they actually did this
<BlueMatt>
and it worked super well
<mmeijeri>
finite fields and Frobenius automorphisms for the win
<BlueMatt>
sadly to fit in the 1500 byte - overhead limit on the internet, we'd have to have the commitments be at the top of the merkle tree :(
<BlueMatt>
(and have the merkle tree use a reduced-length hash, but thats probably fine)
<mmeijeri>
just to check today's other bit of unexpectedly good news: are you saying that, audits aside, the nontrusted mode is basically ready to be integrated with the p2p layer?
<Eliel_>
BlueMatt: isn't it possible to have the protocol use an efficient encoding in the protocol and just reconstruct the actual structure afterward?
<BlueMatt>
mmeijeri: not quite - it needs flow control
<BlueMatt>
right now you manually commit bandwidth to it
<mmeijeri>
right
<BlueMatt>
and it doesnt have negotiation of said bandwidth
<Peter__R>
mmeijeri: It's easy to see why decreasing the interblock time increases orphan rates. For simplicity, assume that propgation time = t0 + zQ where Q is the number of bytes transmitted and z and t0 are constants.
<Peter__R>
Going to a smaller interblock time makes the "t0" constant portion of block transmission a larger fraction of the interblock time.
<BlueMatt>
but aside from that, yes
<mmeijeri>
we've talked about this before, but maybe quic or some other protocol bramc and I were briefly discussing the other day might help
<Peter__R>
And thus makes orphan races more likely.
<BlueMatt>
mmeijeri: yes, streaming it over something that handles the flow control for you might work
<BlueMatt>
though quic is overkill since it does ordering and serialization
<BlueMatt>
which we dont need/want
<BlueMatt>
plus quic is, itself, adding fec
<BlueMatt>
which would be overkill
<BlueMatt>
Eliel_: E_NO_PARSE
<mmeijeri>
I forget the name of the other protocol. it sounded a bit like PEBKAC
<bitcoinbilly>
For now taxes would need to be paid sepperatly and the intrinsic value is volatile. Miners exchange coins to pay there cost. This doesn't affect the closed system mutch
<Eliel_>
BlueMatt: the commitment size problem.
<BlueMatt>
still confused?
<kanzure>
bitcoinbilly: what is your question
<BlueMatt>
the issue (i hate irc, always hard to tell what was and wasnt effectively communicated and what was misread) is you need to be able to tell, after receiving each packet, that it was faithfully calculated from the given block
<BlueMatt>
otherwise you end up in some hellscape where you're trying to figure out which packet was bad, or if the whole thing was garbage
<BlueMatt>
(there are FECs which handle corruption, so maybe that would be sufficient here, but that is a whole nother level of complexity)
bramc has joined #bitcoin-wizards
<bramc>
Ordering and serialization don't really hurt performance
<BlueMatt>
bramc: they do if you miss enough packets to have to do a rtt
<BlueMatt>
but, otherwise, yes, sure
<mmeijeri>
@bramc: do you remember the name of that protocol we were discussing the other day? It was developed by a guy who worked with you on Bittorrent.
<bitcoinbilly>
Why not ? If we redistribute the value token we spent again into the closed system . The only fluctuation is the mining fee.
<bramc>
If you miss enough packets your congestion control might want to slow down anyway.
<bramc>
mmeijeri: ledbat? uTP?
<BlueMatt>
bramc: the whole point of fibre is to literally never wait for an rtt
<BlueMatt>
which, yea, blows up flow control
<mmeijeri>
ledbat was what I was looking for. The thing that sounded like PEBKAC :-)
<BlueMatt>
oh, yea, that one
<BlueMatt>
i mean decent flow control is kinda at odds with low-latency communication
<BlueMatt>
unless your window size is big enough to get the whole block in it
<bramc>
BlueMatt: It could be that bittorrent-live style flow control would help
<BlueMatt>
have you explained that one to me? i forget?
<bramc>
The basic idea is that there are a bunch of 'clubs' (specifically 6 in the BitTorrent Live case), each peer is a member of a club, and every piece of data is assigned to a club. You blast out data within your own club with no flow control and download everything else with real flow control from someone who is in the club
<BlueMatt>
oh, yes, ok, i remember this
<bramc>
It's a trick for getting the low latency of a screamer protocol while still having efficiency and flow control
<BlueMatt>
yea, that would probably work fine
<BlueMatt>
the annoying thing is fibre is super random - sometimes you need 100kb sometimes (usually) only 5
<BlueMatt>
so with no flow control you throw away a ton of bw
<BlueMatt>
luckily its already separated into chunks - the first is the header + a bit of compact block metadata
<BlueMatt>
so you could scream that out to all your peers all the time for pretty cheap
<BlueMatt>
the second is transaction data, which you could scream out based on a club-like thing
<BlueMatt>
and then request from other clubs if you didnt get enough
<BlueMatt>
yea, club stuff would probably work super well here if you want it to be p2p
<bitcoinbilly>
Imagine 3.5 billion people participating
<bramc>
The main trick is to have 2 or 3 incoming and outgoing connections within your own club so you don't waste too much. There are attacks on peer selection though.
<BlueMatt>
yea, sounds about right
<BlueMatt>
peer selection is always a bitch :(
kenshi84 has quit [Remote host closed the connection]
kenshi84 has joined #bitcoin-wizards
<bitcoinbilly>
There are possibilitys available wich can be used to prevent drainage
* BlueMatt
kanzure now would be an opportune time to ban bitcoinbilly with a fun joke
<BlueMatt>
one day I'll learn the difference between /me and /ms
<BlueMatt>
g
<BlueMatt>
but not today
bildramer1 has joined #bitcoin-wizards
kenshi84 has quit [Ping timeout: 256 seconds]
<kanzure>
bitcoinbilly: perhaps you should go to #bitcoin instead
bildramer has quit [Ping timeout: 256 seconds]
bitcoinbilly has quit [Ping timeout: 260 seconds]
<mmeijeri>
Explicit connection establishment between hosts Selectable congestion control schemes Path MTU discovery to avoid fragmentation Service Codes for identifying applications
<mmeijeri>
but still unreliable and no fec, so less overlap with Fibre
<mmeijeri>
and therefore perhaps a better fit than QUIC?
bitcoin-wizards8 has joined #bitcoin-wizards
<BlueMatt>
probably, though doint something protocol-specific like bramc's club-based suggestion might be better yet
<bramc>
mmeijeri: AUGH!
<bramc>
That shit doesn't work, trust me, I'm the world's leading expert in this space
<mmeijeri>
dccp you mean?
<BlueMatt>
bramc: when you say shit like that I have a knee-jerk reaction to punch you in the face....until I realize its you and you're not wrong
<bramc>
mmeijeri: No it's the preset tree which doesn't work not the particular congestion scheme you use along links
<BlueMatt>
yea, preset trees sound like a bad idea
Newyorkadam has quit [Quit: Newyorkadam]
<mmeijeri>
I've no idea what that means but I'll take your word for it. @bluematt: is the only reason for trusted mode that it would screw up joint reconstruction from two sources if one were malicious or is there more to it?
Peter__R has quit [Ping timeout: 260 seconds]
<BlueMatt>
mmeijeri: i read "preset trees" to be a tree along which you distribute blocks isntead of a blind swarm-y thing
<BlueMatt>
mmeijeri: exactly
Newyorkadam has joined #bitcoin-wizards
<BlueMatt>
well, malicious or generating garbage, but...same thing
Guyver2 has quit [Quit: :)]
<mmeijeri>
I thought DCCP was purely point-to-point
<BlueMatt>
mmeijeri: note that fibre also does cut-through routing in trusted mode, so each packet you receive you forward along to all your (trusted) peers without any validation/reconstruction
<BlueMatt>
(this gives it the wonderful scalability property where nodes in the network tend to receive blocks with the same amount of time needed for receive (ie time from first packet to reconstruction) irrespective of the number of hops passed through on the way
<BlueMatt>
)
kenshi84 has joined #bitcoin-wizards
<mmeijeri>
ah yes, that's great too
<mmeijeri>
another question about the fec and missing txs: does this mean that we're closing to being able to implement p2p mempool synchronisation too?
<BlueMatt>
but makes the malicious peer poblem worse
<BlueMatt>
I'm not sure where we are on p2p mempool sync
<BlueMatt>
I think someone was talking about doing it using the old invertable bloom stuff
<mmeijeri>
why can't you do it with the same fec you are using for fibre?
<BlueMatt>
greg had some ideas, but never got around to actually running numbers on them, iir
<BlueMatt>
c
<BlueMatt>
i suppose you could shove all the txids in your mempool over a certain feerate in a simple fec and send it
<mmeijeri>
and you could negotiate the fee rate
<mmeijeri>
and signal the other side to stop sending info once they have everything
<BlueMatt>
yea, last i heard any discussion of mempool sync you want to eg do "top 2MB of my mempool" after every block, and have more backoff for lower down in your mempool
<BlueMatt>
would have to ask gmaxwell
<BlueMatt>
he had some ideas, and is way more equipped to answer that question than I
<mmeijeri>
anyway, I'm really excited this stuff has progressed so far already
<BlueMatt>
yea, its pretty cool
<BlueMatt>
fibre is pretty awesome in its effectiveness
<BlueMatt>
i mean the stats are like astoundingly reliable
<BlueMatt>
it pretty much never takes any more time than a handful of ms + the actual data you wanted to send
<BlueMatt>
except when my vm in chain pauses randomly
* BlueMatt
hates vms
<BlueMatt>
but all the non-china servers are physical servers so they tend to be rock solid
<BlueMatt>
well, except for fiber cuts in india/singapore right now causing things from southeast asia to europe to be slightly delayed :(
<BlueMatt>
with expected fix times in like....march
bramc has quit [Ping timeout: 260 seconds]
kenshi84 has quit [Ping timeout: 240 seconds]
<mmeijeri>
on flow control: how much of that do you need when you sporadically blast out relatively small amounts of data vs streaming lots of data for a long time as with bittorrent?
<mmeijeri>
wouldn't reasonable estimation of your e2e bandwidth be enough?
<BlueMatt>
depends on your link
<BlueMatt>
if its gonna take 10ms to blast it? probably doesnt matter much
<BlueMatt>
if its gonna take a second, its probably a big deal
<BlueMatt>
esp given buffbloat issues
<BlueMatt>
hence why i like bram's suggestion of using groups
<mmeijeri>
I'm glad we have a bunch of network experts on board
<BlueMatt>
sidesteps the issue somewhat
<BlueMatt>
(because you can limit the groups to relatively small amounts of data - ie many groups)
<mmeijeri>
you, bram, rusty, teitelbaum
<BlueMatt>
and then its always 10ms
<BlueMatt>
as if anyone has time to build this :P
<BlueMatt>
does sound like fun, but man...annoying as fuck to build
<mmeijeri>
if we do it incrementally we could start by reusing something that already exists like quic or more likely dccp
<mmeijeri>
and then later replace it with something that's optimised for our application
kenshi84 has joined #bitcoin-wizards
mountaingoat has quit [Quit: WeeChat 1.6]
mountaingoat has joined #bitcoin-wizards
priidu has joined #bitcoin-wizards
dkings has quit []
priidu has quit [Ping timeout: 258 seconds]
kenshi84 has quit [Read error: Connection reset by peer]
priidu has joined #bitcoin-wizards
ptrx has joined #bitcoin-wizards
Davasny has quit [Remote host closed the connection]
priidu has quit [Ping timeout: 252 seconds]
priidu has joined #bitcoin-wizards
<BlueMatt>
mmeijeri: yea, I mean really I think getting an audit (or even some basic fuzzing on the receive-end) is the first step
<BlueMatt>
problem is you're not gonna get a standard audit firm that will do an audit of this
priidu has quit [Ping timeout: 255 seconds]
<BlueMatt>
too much dense linear algebra even for me to audit
<mmeijeri>
audit for remote exploits you mean?
<BlueMatt>
yea
<BlueMatt>
I'm rather afraid of exposing the fibre ports publicly right now because I'm sure you could find at least some assert crashes
<BlueMatt>
even if not anything rce-ish
priidu has joined #bitcoin-wizards
<mmeijeri>
why would we need it for the p2p layer and not for fibre? or rather, why doesn't fibre need it?
<BlueMatt>
certainly wh256 is wayyy better built than the previous stuff i had been using
<BlueMatt>
because the very first thing it does when receiving a packet is checks the hmac on it
<BlueMatt>
and I'm pretty confident at least that much is fine
<BlueMatt>
so as long as they dont steal your symmetric hmac keys and you trust the other side to not pwn you its probably fine
<mmeijeri>
for trusted mode only?
<BlueMatt>
hmm? no, like I wouldnt be surprised if you found some crashes in sending certain malformed packets
<BlueMatt>
or in bad order
<BlueMatt>
like its a big pile of networking code that is entirely unreviewed
<mmeijeri>
right
Chris_Stewart_5 has quit [Ping timeout: 256 seconds]
<BlueMatt>
its super stable in regular use, but that doesnt mean much if you want to expose it to the internet....
blackwraith has joined #bitcoin-wizards
<mmeijeri>
is the audit the main reason you think implementing this in Core will be a lot of work?
blackwraith has quit [Read error: Connection reset by peer]
<BlueMatt>
yes
<mmeijeri>
ok
<BlueMatt>
i mean the way its written it could pretty much be merged tomorrow, i think....except that no one in their right mind would review it
<BlueMatt>
or think they had sufficiently reviewed it to merge
<BlueMatt>
(of course using it p2p would be more work, but at least the private network stuff is cool on its own)
blackwraith has joined #bitcoin-wizards
<mmeijeri>
and is the fec code the part that would take the most time to audit?
abpa has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<mmeijeri>
that would surprise me a bit
<mmeijeri>
after all, we would be auditing it for exploits, not correctness
<BlueMatt>
probably, though the actual logic is also quite a bear...1500 lines of pure conditional checks and message formatting logic
<BlueMatt>
and of course getting any part of it wrong and youre sending garbage uninitialized memory on the wire
<mmeijeri>
I think I'll have a look at it
<mmeijeri>
I'm wondering if it needs to be refactored first