Merge pull request #2159 from darosior/2505_64b_clarifications_and_rationale

BIP 54: clarify 64-byte transactions item description and rationale

bip-0054.md

@@ -38,10 +38,13 @@ users, increasing the cost to independently fully validate the consensus rules.
 be used by miners to attack their competition, creating perverse incentives, centralization
 pressures and leading to reduced network security.
 
-In computing a block's Merkle root, a 64-byte transaction can be interpreted both as an intermediate
+In computing a block's Merkle root, a transaction with exactly 64 bytes of non-witness data can be
-node in the tree and as a leaf in the tree. This makes it possible to fake inclusion proofs by
+interpreted both as an intermediate node in the tree and as a leaf in the tree. This makes it
-pretending a 64-byte block transaction is an inner node, as well as to pretend the inner nodes on
+possible to trick an SPV verifier into accepting an inclusion proof for a transaction that is not
-one level of the tree are the actual block transactions.
+part of a block, by pretending a 64-byte block transaction is actually an inner node[^9]. Invalidating
+64-byte transactions addresses this vulnerability without requiring users of SPV verifiers, or
+any other user of Merkle proofs, to rely on one of the available workarounds[^13] or even to know one is
+necessary in the first place.
 
 Since [bip-0034][BIP34] activation, explicit [bip-0030][BIP30] validation is not necessary until
 block height 1,983,702[^0].  Mandating new coinbase transactions be different from the early
@@ -102,17 +105,16 @@ increases the preparation cost of an attack, making it uneconomical for a miner[
 2500 was chosen as the tightest value that did not make any non-pathological standard transaction
 invalid[^7].
 
-In the presence of 64-byte transactions a block header's Merkle root may be valid for different sets
+64-byte transactions can only contain a scriptPubKey that lets anyone spend the funds, or one that
-of transactions. This is because, in the Merkle tree construction, a 64-byte value may be
+burns them. They have also been non-standard since 2019 and never been used since 2016.  Several
-interpreted either as a transaction or as the concatenation of two 32-byte hashes. The former allows faking a block inclusion proof and the latter makes
+alternatives to invalidating them were previously proposed. Some believe the improvements for users
-it such that for a valid block the Merkle root in the block header is not a unique identifier for
+of Merkle proofs are too marginal to be worth introducing a discontinuity in the set of valid
-the corresponding list of valid transactions[^8]. 64-byte transactions can only contain a
+witness-stripped transaction sizes. Others have suggested instead committing to the Merkle
-scriptPubKey that lets anyone spend the funds, or one that burns them. 64-byte transactions have
+tree depth in the header's version field[^8], which would make one workaround for a known
-also been non-standard since 2019. It was suggested that the known vulnerabilities could instead be
+vulnerability easier to deploy. The authors believe it is preferable to address the root cause by
-mitigated by committing to the Merkle tree depth in the header's version field[^9]. The authors
+invalidating 64-byte transactions, fixing the vulnerability without Merkle proof users having to
-believe it is preferable to address the root cause by invalidating 64-byte transactions. This
+rely on any workaround or even know one is necessary in the first place. See [this post][64 bytes
-approach also fixes the vulnerability without developers of SPV verifiers having to implement the
+debate] for an attempt at summarizing the arguments for both sides of this debate.
-mitigation or to know it is necessary in the first place.
 
 Several blocks prior to [bip-0034][BIP34] activation contain a coinbase transaction whose scriptSig
 contains a valid [bip-0034][BIP34] commitment to a future block height. This offers an opportunity
@@ -146,7 +148,7 @@ Bitcoin Core version [30.0][Core 30.0] and later will not generate a block templ
 transaction that violates the signature operations limit introduced in this BIP.
 
 Bitcoin Core version [0.16.1][Core 0.16.1] and later will neither relay nor create block templates
-that include 64-byte transactions.
+that include transactions whose witness-stripped serialized size is exactly 64 bytes.
 
 The coinbase transaction is usually crafted by mining pool software. To the best of the authors'
 knowledge, there does not exist an open source reference broadly in use today for such software.
@@ -207,11 +209,14 @@ standard transaction size before running into the newly introduced limit. To run
 introduced limit but not the transaction size a transaction would need to spend P2SH inputs with a
 redeem script similar to `CHECKSIG DROP CHECKSIG DROP ...`. This type of redeem script serves no
 purpose beyond increasing its validation cost, which is exactly what this proposal aims to mitigate.
-[^8]: See [this writeup][Suhas Merkle] by Suhas Daftuar for an explanation as well as a discussion
+[^8]: By Sergio Demian Lerner in a [blog post][Sergio post].
-of the consequences.
+[^9]: Conversely, pretending that the inner nodes on one level of the tree are the actual block
-[^9]: By Sergio Demian Lerner in a [blog post][Sergio post] surfaced [by Eric Voskuil][Eric
+transactions is another source of complexity for full node implementations, which previously
-version]. Eric also pushed back against the importance of fixing this issue. See [this post][64
+resulted in consensus bugs. For instance, Bitcoin Core versions between 0.13.0 and 0.13.2
-bytes debate] for an attempt at summarizing the arguments for both sides of this debate.
+implemented caching that made it vulnerable to this attack. See [this writeup][Suhas Merkle] by
+Suhas Daftuar for a detailed explanation. Invalidating 64-byte transactions may avoid this risk, but
+the issue is largely orthogonal to this proposal: it is fundamentally about caching validation
+status for malleable blocks.
 [^10]: See [here][BIP34 list] for a full list of the heights of historical blocks including a valid
 bip-0034 height commitment and the corresponding future block height.
 [^11]: Technically it could be argued a duplicate could in principle always be possible before block
@@ -223,6 +228,13 @@ notably of Bitcoin Core but also of all other implementations the authors are aw
 where [bip-0034][BIP34] violations have been manually inspected (see [here][Core validation.cpp
 BIP34]). Without the guarantee given by enforcing the timelock on coinbase transactions, this would
 have to be perpetuated for the Consensus Cleanup.
+[^13]: The authors are aware of three workarounds for SPV verifiers. The first is to request a
+Merkle proof for the coinbase transaction in addition to the transaction of interest, to infer the
+depth of the Merkle tree. The second is to reject Merkle proofs in which any inner node is also a
+valid serialisation of a Bitcoin transaction. More details about these are available [here][Sergio
+post]. A third workaround is to change the Merkle proof structure by requiring inner nodes to be
+provided as the single-SHA256 of their preimage, instead of the double-SHA256. See [here][Sergio
+MERKLEBLOCK] for a full description.
 
 [BIP30]: https://github.com/bitcoin/bips/blob/master/bip-0030.mediawiki
 [BIP-XXXX]: https://github.com/TheBlueMatt/bips/blob/7f9670b643b7c943a0cc6d2197d3eabe661050c2/bip-XXXX.mediawiki
@@ -238,7 +250,7 @@ have to be perpetuated for the Consensus Cleanup.
 [ML thread validation time]: https://gnusha.org/pi/bitcoindev/VsltJ2PHqWfzG4BU9YETTXjL7fYBbJhjVXKZQyItemySIA1okvNee9kf0zAOyLMeJ4Nqv1VOrYbWns5nP4TANCWvPJYu1ew_yxQSaudizzk=@protonmail.com
 [Suhas Merkle]: https://gnusha.org/pi/bitcoindev/CAFp6fsGtEm9p-ZQF_XqfqyQGzZK7BS2SNp2z680QBsJiFDraEA@mail.gmail.com
 [Sergio post]: https://bitslog.com/2018/06/09/leaf-node-weakness-in-bitcoin-merkle-tree-design
-[Eric version]: https://delvingbitcoin.org/t/great-consensus-cleanup-revival/710/37
+[Sergio MERKLEBLOCK]: https://bitslog.com/2018/08/21/simple-change-to-the-bitcoin-merkleblock-command-to-protect-from-leaf-node-weakness-in-transaction-merkle-tree/
 [64 bytes debate]: https://delvingbitcoin.org/t/great-consensus-cleanup-revival/710/41
 [BIP34 list]: https://delvingbitcoin.org/t/great-consensus-cleanup-revival/710/4
 [Harding nLockTime]: https://bitcoin.stackexchange.com/questions/90229/nlocktime-in-bitcoin-core