From 7bce321a18dac4a1be114eeb03fa71da9c1469ed Mon Sep 17 00:00:00 2001 From: jl2012 Date: Sat, 19 Dec 2015 03:52:06 +0800 Subject: [PATCH 01/17] Create bip-segwit.mediawiki --- bip-codeshark-jl2012-segwit.mediawiki | 277 ++++++++++++++++++++++++++ 1 file changed, 277 insertions(+) create mode 100644 bip-codeshark-jl2012-segwit.mediawiki diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki new file mode 100644 index 00000000..1fc711d8 --- /dev/null +++ b/bip-codeshark-jl2012-segwit.mediawiki @@ -0,0 +1,277 @@ +
+  BIP: x
+  Title: Segregated Witness (Consensus layer)
+  Author: 
+  Status: Draft
+  Type: Standards Track
+  Created: 2015-12
+
+ +==Abstract== + +This BIP defines a new structure called a "witness" that is committed to blocks separately from the transaction merkle tree. This structure contains data required to check transaction validity but not required to determine transaction effects. In particular, scripts and signatures are moved into this new structure. + +The witness is committed in a tree that is nested into the block's existing merkle root via the coinbase transaction for the purpose of making this BIP soft fork compatible. A future hard fork can place this tree in its own branch. + +==Motivation== + +The entirety of the transaction's effects are determined by output consumption (spends) and new output creation. Other transaction data, and signatures in particular, are only required to validate the blockchain state, not to determine it. + +By removing this data from the transaction structure committed to the transaction merkle tree, several problems are fixed: + +# '''Nonintentional malleability becomes impossible'''. Since signature data is no longer part of the transaction hash, changes to how the transaction was authorized is no longer relevant to transaction identification. As a solution of transaction malleability, this is superior to the canonical signature approach (BIP62): +#* It prevents involuntary transaction malleability for any type of scripts, as long as all inputs are signed (with at least one CHECKSIG or MULTICHECKSIG operation) +#* In the case of a n-of-m MULTICHECKSIG script, a transaction is malleable only with agreement of n private key holders (as opposed to only 1 private key holder with BIP62) +#* It prevents involuntary transaction malleability due to unknown ECDSA signature malleability +#* It allows creation of unconfirmed transaction dependency chains without counterparty risk +# '''Transmission of signature data becomes optional'''. It is needed only if a peer is trying to validate a transaction, instead of just to prove its existence. This also improves prviacy of SPV clients as using the same bandwidth they could request for more transactions for obscuration. +# '''Some constriants could be bypassed with a soft fork''' by moving part of the transaction data to a structure unknown to current protocol, for example: +#* Size of witness could be ignored / discounted when calculating the block size, effectively increasing the block size to some extent +#* Hard coded constants, such as maximum data push size (520 bytes) or sigops limit could be reevaluated or removed +#* New script system could be introduced without any limitation from the existing script semantic + +==Specification== + +Maybe a seperate BIP +
+* CTransaction gets, in addition to vin and vout, a vwit, which
+contains a CTxInWitness object for each input. A CTxInWitness contains a CScriptWitness object
+and can potentially be extended to contain other kinds of witness data.
+A CScriptWitness is a vector of byte vectors (nominally: the input stack to the program, no longer
+encoded as a CScript, but just the resulting stack directly).
+
+* A new serialization for CTransaction is defined: http://blockhawk.net/diagrams/witnesstx.png
+(int32 nVersion,
+0x00 marker, 0x01 flag, vector, vector,
+vector, int32 nLockTime) instead of (int32 nVersion,
+vector, vector, int32 nLockTime). This will never parse
+as a valid transaction (even if parsing succeeds, it means it's
+interpreted as a transaction with no inputs and 1 output). If all
+witnesses are empty, the old serialization format is used.
+- Rationale for not having an independent CWitnessTransaction with
+its own serialization: this would require separate "tx" and "block"
+messages, and all RPC calls operating on raw transactions would need
+to be duplicated, or need inefficinent or nondeterministic guesswork
+to know which type is to be used.
+- Rationale for not using just a single 0x00 byte as marker: that
+would lead to empty transactions (no inputs, no outputs, which are
+used in some tests) to be interpreted as new serialized data.
+- Rationale for the 0x01 flag byte in between: this will allow us to
+easily add more extra non-committed data to transactions (like txouts
+being spent, ...). It can be interpreted as a bitvector.
+
+* A new message 'havewitness' is sent after receiving 'verack' to
+indicate that a node can provide witness if requested (similar to
+'sendheaders')
+
+* New inv types MSG_WITNESS_TX and MSG_WITNESS_BLOCK are added, only
+for use in getdata. Inv itself still use just MSG_TX and MSG_BLOCK,
+similar to MSG_FILTERED_BLOCK.
+- Rationale for not advertizing witnessness in invs: we don't always
+use invs anymore (with 'sendheaders' BIP 130), plus it's not useful:
+implicitly, every transaction and block have a witness, old ones just
+have empty ones.
+
+* Transactions' GetHash is always computed on the old non-witness
+serialization. A new CTransaction::GetWitnessHash is added which is
+computed from the witness-serialization (this means that transactions
+with an empty witness have witness hash equal to normal hash).
+
+ +=== Commitment structure === + +UPDATE NEEDED +A new block rule is added which requires a commitment (a merkle root +computed similarly to the normal transaction one) to the witness +hashes to be present as the last 32 bytes of +block.vtx[0].vin[0].scriptSig (it doesn't need to be a push). This +hopefully does not conflict with any other existing commitment +schemes. To make it extensible, an extra merkle path can be provided +(in the coinbase's "witness" field) so that coinbase commitment can be +used for multiple things. +
+    // Validation for witness commitments.
+    // * We compute the witness hash (which is the hash including witnesses) of all the block's transactions, except the
+    //   coinbase (where 0x0000....0000 is used instead).
+    // * We build a merkle tree with all those witness hashes as leaves (similar to the hashMerkleRoot in the block header).
+    // * The first coinbase scriptSig minimal push of 41 bytes for which the first 4 bytes are {0xaa, 0x21, 0xa9, 0xed} is
+    //   treated as a commitment header. If no such push is present, the block is invalid. If multiple are present, the first
+    //   is used.
+    //   * The first 4 bytes of the commitment header are just magic identifier bytes, and have no further meaning.
+    //   * The next 4 bytes describe a nonce.
+    //   * The next 1 byte describes the number of levels in a Merkle tree.
+    //   * locator = SHA256('WitnessV1\x00\x00\x00\x00\x00\x00\x00' || nonce). The first levels bits of locator, interpreted
+    //     in little endian, are assumed to be the position in the leaves of this Merkle tree where the witness commitment
+    //     goes.
+    //   * The last 32 bytes of the commitment header are its root hash.
+    //   * The coinbase's input's witness must consist of a single byte array of 32 * levels bytes, and are assumed to be
+    //     the Merkle path to connect the witness root hash to the commitment root hash.
+    
+    // No witness data is allowed in blocks that don't commit to witness data, as this would otherwise leave room from spam.
+
+ +=== Block size limit === +2-4-8? Discount for witness data? + +=== Witness program === + +* A scriptPubKey (or redeemScript as defined in BIP16/P2SH) that consists of a single push of 2 to 41 bytes gets a new special meaning. The byte vector pushed by it is called the "witness program". +** In case the scriptPubKey pushes a witness program directly, the scriptSig must be exactly empty. +** In case the redeemScript pushes a witness program, the scriptSig must be exactly the single push of the redeemScript. + +* The first byte of a witness program is the "version byte", an unsigned integer. +** If the version byte is 0, the rest of the witness program is the actual script. +*** The script is executed after normal script evaluation but with data from the witness rather than the scriptSig. +*** The program must not fail, and result in exactly a single TRUE on the stack. +** If the version byte is 1, the rest of the witness program must be 32 bytes, as a SHA256 hash of the actual script. +*** The witness must consist of an input stack to feed to the program, followed by the serialized program. +*** The serialized program is popped off the initial witness stack. Hash of the serialized program must match the hash pushed in the witness program. +*** The serialized program is deserialized, and executed after normal script evaluation with the remaining witness stack. +*** The script must not fail, and result in exactly a single TRUE on the stack. +** If the witness version byte is 2 or above, no further interpretation of the witness program or witness happens. + +=== Other consensus critical constraints === + +== Block size analysis == +Definitions: +
+ Core block size (CBS): The block size as seen by a non-upgrading full node
+ Witness size (WS): The total size of witness in a block
+ Total block size (TBS): CBS + WS
+ Witness discount (WD): A discount factor for witness for calculationg of VBS (1 = no discount)
+ Virtual block size (VBS): CBS + (WS * WD)
+ Witness adoption (WA): Proportion of new format transactions among all transactions
+ Prunable ratio (PR): Proportion of signature data size in a transaction
+
+ +With some transformation it could be shown that: +
+ TBS = CBS / (1 - WA * PR)
+     = VBS / (1 - WA * PR * (1 - WD))
+
+ +In order to keep the proposal as a soft fork, the CBS must not have a upper limit higher than 1MB. + +TBS is a function of only CBS, PR, and WA. + +The PR heavily depends on the transaction script type and input-output ratio. For example, the PR of 1-in 2-out P2PKH and 1-in 1-out 2-of-2 multisig P2SH are about 47% and 72% respectively. According to the data presented by Pieter Wuille on 7 December 2015, the current average PR on the blockchain is about 60%. + + + +== Examples == + +=== Version 0 witness program === + +The following example is a version 0 witness program, equivalent to the existing Pay-to-Pubkey-Hash (P2PKH) output. + + witness: + scriptSig: (empty) + scriptPubKey: <0x0076A914{20-byte-hash-value}88AC> + +The version byte 0x00 is removed. The rest of the witness program is deserialized and becomes: + + DUP HASH160 <20byte-hash-value> EQUALVERIFY CHECKSIG + +The script is executed with the data from witness + + DUP HASH160 <20byte-hash-value> EQUALVERIFY CHECKSIG + +Comparing with a P2PKH output, the witness program equivalent occupies 2 more bytes in the scriptPubKey, while moving the signature and public key from scriptSig to witness. + +=== Version 1 witness program === + +The following example is an 1-of-2 multi-signature version 1 witness program. + + witness: 0 <0x5121{33-byte-pubkey1}21{33-byte-pubkey2}52AE> + scriptSig: (empty) + scriptPubKey: <0x01{32-byte-hash-value}> + +The last item in the witness is popped off, hashed with SHA256, compared against the 32-byte-hash-value in scriptPubKey, and deserialized: + + 1 <33-byte-pubkey1> <33-byte-pubkey2> 2 CHECKMULTISIG + +The script is executed with the remaining data from witness: + + 0 1 <33-byte-pubkey1> <33-byte-pubkey2> 2 CHECKMULTISIG + +Since the actual program is larger than 40 bytes, it cannot be accommodated in a version 0 witness program. A version 1 witness program allows arbitrarily large script as the 520-byte push limit is bypassed. + +The scriptPubKey occupies 34 bytes, as opposed to 23 bytes of P2SH. The increased size improves security against possible collision attacks, as 2^80 work is not infeasible anymore (By the end of 2015, 2^84 hashes have been calculated in Bitcoin mining since the creation of Bitcoin). The spending script is same as the one for an equivalent P2SH output but is moved to witness. + +=== Witness program nested in Pay-to-Script-Hash === + +The following example is the same 1-of-2 multi-signature version 1 witness program, but nested in a P2SH output. + + witness: 0 <0x5121{33-byte-pubkey1}21{33-byte-pubkey2}52AE> + scriptSig: <0x2101{32-byte-hash-value}> + scriptPubKey: HASH160 <20-byte-hash-value> EQUAL + +The only item in scriptSig is hashed with HASH160, compared against the 20-byte-hash-value in scriptPubKey, and interpreted as a single push of: + + <0x01{32-byte-hash-value}> + +The version 1 witness program is then executed as described in the last example + +Comparing with the last example, the scriptPubKey is 11 bytes smaller (with reduced security) while witness is the same. However, it also requires 35 bytes in scriptSig, which is not prunable in transmission. Although a nested witness program is less efficient in many ways, its payment address is fully transparent and backward compatible for all Bitcoin reference client since version 0.6.0. + +=== Trust-free unconfirmed transaction dependency chain === +Segregated witness fixes the problem of transaction malleability fundamentally, which enables the building of unconfirmed transaction dependency chains in a trust-free manner. + +Two parties, Alice and Bob, may agree to send certain amount of Bitcoin to a 2-of-2 multisig output (the "funding transaction"). Without signing the funding transaction, they may create another transaction, time-locked in the future, spending the 2-of-2 multisig output to third account(s) (the "spending transaction"). Alice and Bob will sign the spending transaction and exchange the signatures. After examining the signatures, they will sign and commit the funding transaction to the blockchain. Without further action, the spending transaction will be confirmed after the lock-time and release the funding according to the original contract. It also retains the flexibility of revoking the original contract before the lock-time, by another spending transaction with shorter lock-time, but only with mutual-agreement of both parties. + +Such setups is not possible with BIP62 as the malleability fix, since the spending transaction could not be created without both parties first signing the funding transaction. If Alice reveals the funding transaction signature before Bob does, Bob is able to lock up the funding indefinitely without ever signing the spending transaction. + +Unconfirmed transaction dependency chain is a fundamental building block of more sophisticated payment networks, such as duplex micropayment channel and the Lightning Network, which have the potential to greatly improve the scalability and efficiency of the Bitcoin system. + +== Future extensions == +=== Compact fraud proof for SPV nodes === +Bitcoin right now only has two real security models. A user either runs a full-node which validates every block with all rules in the system, or a SPV (Simple Payment Verification) client which only validates the headers as a proof of publication of some transactions. The Bitcoin whitepaper suggested that SPV nodes may accept alerts from full nodes when they detect an invalid block, prompting the SPV node to download the questioned blocks and transactions for validation. This approach, however, could become a DoS attack vector as there is virtually no cost to generate a false alarm. An alarm must come with a compact, yet deterministic fraud proof. + +In the current Bitcoin protocol, it is possible to generate compact fraud proof for almost all rules except a few: + +# It is not possible to proof a miner has introduced too many Bitcoins in the coinbase transaction outputs without showing the whole block itself and all input transactions. +# It is not possible to prove the violation of any block specific constraints, such as size and sigop limits, without showing the whole block (and all input transactions in the case of sigop limit) +# It is not possible to prove the spending of a non-existing input without showing all transaction IDs in the blockchain way back to the genesis block. + +It is possible to proof the first 2 types of fraud if a block is committed to a Merkle-sum-tree of the fee, size, and sigop count of each transaction. It is also possible to proof the last type of fraud if a block is committed to a Merkle tree with the originating block height and transaction index of all inputs. These commitments could be included in the extensible witness commitment through a soft fork and will be transparent to nodes that do not understand such new rules. + +=== New script system === +Since all witness programs begin with a version byte, and programs with unknown versions are always considered as anyone-can-spend script, it is possible to introduce any new script system with a soft fork. The witness as a structure is not restricted by any existing script semantics and constraints, the 520-byte push limit in particular, and therefore allows arbitrarily large scripts and signatures. + +Examples of new script system include Schnorr signatures which reduce the size of multisig transactions dramatically, Lamport signature which is quantum computing resistance, and Merklized abstract syntax trees which allow very compact witness for conditional scripts with extreme complexity. + +The 41-byte limitation for witness programme could be easily extended through a soft fork in case a stronger hash function is needed in the future. The version byte is also expandable by introducing a secondary version byte for some specific primary version values. + +== Backward compatibility == +Without lifting the core block size limit of 1MB at the beginning, this proposal is a soft fork which all existing full nodes and SPV nodes are compatible. Non-upgrading nodes, however, will not see nor validate the witness data, and will consider all witness programs as anyone-can-spend scripts (except a few edge cases in version 0 witness programs which are provably unspendable with original script semantics). Non-upgrading nodes are also unable to generate or sign for scriptPubKey templates (payment address). Nonetheless, they may still pay to a witness program if it is nested in a P2SH address, which has been defined since 2012. + +When the core block size limit is increased to over 1MB as scheduled, this proposal becomes a hard fork. Anybody running code that fully validates blocks must upgrade before the activation time or they will reject a chain containing blocks with core block size above 1MB. SPV software is not affected, unless it makes assumptions about the maximum depth of the Merkle tree based on the minimum size of a transaction and the maximum block size. + +== Deployment == + +We reuse the double-threshold IsSuperMajority() switchover mechanism used in +BIP65 with the same thresholds, but for nVersion = 5. The new rules are +in effect for every block (at height H) with nVersion = 5 and at least +750 out of 1000 blocks preceding it (with heights H-1000..H-1) also +have nVersion >= 5. Furthermore, when 950 out of the 1000 blocks +preceding a block do have nVersion >= 5, nVersion < 5 blocks become +invalid, and all further blocks enforce the new rules. + +It should be noted that BIP9 involves permanently setting a high-order bit to +1 which results in nVersion >= all prior IsSuperMajority() soft-forks and thus +no bits in nVersion are permanently lost. + + +=== SPV Clients === + +While SPV clients are (currently) unable to validate blocks in general, +trusting miners to do validation for them, they are able to validate block +headers and thus can validate a subset of the deployment rules. SPV clients +should reject nVersion < 5 blocks if 950 out of 1000 preceding blocks have +nVersion >= 5 to prevent false confirmations from the remaining 5% of +non-upgraded miners when the 95% threshold has been reached. + +== Credits == + +== Reference Implementation == +https://github.com/sipa/bitcoin/commits/segwit From 1817685aa94fdc5a58823e7e0a407463bea55ae0 Mon Sep 17 00:00:00 2001 From: Eric Lombrozo Date: Sat, 19 Dec 2015 21:55:26 -0500 Subject: [PATCH 02/17] Added fraud proof motivation. --- bip-codeshark-jl2012-segwit.mediawiki | 3 +++ 1 file changed, 3 insertions(+) diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki index 1fc711d8..703954c0 100644 --- a/bip-codeshark-jl2012-segwit.mediawiki +++ b/bip-codeshark-jl2012-segwit.mediawiki @@ -29,6 +29,9 @@ By removing this data from the transaction structure committed to the transactio #* Size of witness could be ignored / discounted when calculating the block size, effectively increasing the block size to some extent #* Hard coded constants, such as maximum data push size (520 bytes) or sigops limit could be reevaluated or removed #* New script system could be introduced without any limitation from the existing script semantic +# '''Additional data required for fraud proofs can be added to witness'''. Extra data can be committed that allows short proofs of block invalidity that SPV nodes can quickly verify. +#* Backlinks for the outputs spent by the transaction's inputs can be provided. These backlinks consist of a block hash and an offset that thin clients can easily query and check to verify that the outputs exist. +#* Sum trees for transaction inputs and outputs can be committed making it possible to construct short proofs that no new coins are created in any noncoinbase transaction and that the miner does not add excessive fees to the coinbase transaction. ==Specification== From 1cdf145b9b74f465b514b19067c6a781769fdc77 Mon Sep 17 00:00:00 2001 From: jl2012 Date: Mon, 21 Dec 2015 13:27:25 +0800 Subject: [PATCH 03/17] Update bip-segwit.mediawiki --- bip-codeshark-jl2012-segwit.mediawiki | 93 ++++++++++++++++----------- 1 file changed, 56 insertions(+), 37 deletions(-) diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki index 703954c0..cc87de3c 100644 --- a/bip-codeshark-jl2012-segwit.mediawiki +++ b/bip-codeshark-jl2012-segwit.mediawiki @@ -34,7 +34,6 @@ By removing this data from the transaction structure committed to the transactio #* Sum trees for transaction inputs and outputs can be committed making it possible to construct short proofs that no new coins are created in any noncoinbase transaction and that the miner does not add excessive fees to the coinbase transaction. ==Specification== - Maybe a seperate BIP
 * CTransaction gets, in addition to vin and vout, a vwit, which
@@ -42,7 +41,6 @@ contains a CTxInWitness object for each input. A CTxInWitness contains a CScript
 and can potentially be extended to contain other kinds of witness data.
 A CScriptWitness is a vector of byte vectors (nominally: the input stack to the program, no longer
 encoded as a CScript, but just the resulting stack directly).
-
 * A new serialization for CTransaction is defined: http://blockhawk.net/diagrams/witnesstx.png
 (int32 nVersion,
 0x00 marker, 0x01 flag, vector, vector,
@@ -83,38 +81,54 @@ with an empty witness have witness hash equal to normal hash).
 
 === Commitment structure ===
 
-UPDATE NEEDED
-A new block rule is added which requires a commitment (a merkle root
-computed similarly to the normal transaction one) to the witness
-hashes to be present as the last 32 bytes of
-block.vtx[0].vin[0].scriptSig (it doesn't need to be a push). This
-hopefully does not conflict with any other existing commitment
-schemes. To make it extensible, an extra merkle path can be provided
-(in the coinbase's "witness" field) so that coinbase commitment can be
-used for multiple things.
-
-    // Validation for witness commitments.
-    // * We compute the witness hash (which is the hash including witnesses) of all the block's transactions, except the
-    //   coinbase (where 0x0000....0000 is used instead).
-    // * We build a merkle tree with all those witness hashes as leaves (similar to the hashMerkleRoot in the block header).
-    // * The first coinbase scriptSig minimal push of 41 bytes for which the first 4 bytes are {0xaa, 0x21, 0xa9, 0xed} is
-    //   treated as a commitment header. If no such push is present, the block is invalid. If multiple are present, the first
-    //   is used.
-    //   * The first 4 bytes of the commitment header are just magic identifier bytes, and have no further meaning.
-    //   * The next 4 bytes describe a nonce.
-    //   * The next 1 byte describes the number of levels in a Merkle tree.
-    //   * locator = SHA256('WitnessV1\x00\x00\x00\x00\x00\x00\x00' || nonce). The first levels bits of locator, interpreted
-    //     in little endian, are assumed to be the position in the leaves of this Merkle tree where the witness commitment
-    //     goes.
-    //   * The last 32 bytes of the commitment header are its root hash.
-    //   * The coinbase's input's witness must consist of a single byte array of 32 * levels bytes, and are assumed to be
-    //     the Merkle path to connect the witness root hash to the commitment root hash.
-    
-    // No witness data is allowed in blocks that don't commit to witness data, as this would otherwise leave room from spam.
-
+!!!DRAFT ONLY!!! + +A new block rule is added which requires a commitment to the witness hashes in the coinbase (or an OP_RETURN output of a specific transaction, or somewhere else) + +(Hardfork after x year to move the commitment to block header?) + +Witness hash is the double SHA256 of a transaction including witnesses: http://blockhawk.net/diagrams/witnesstx.png. For the coinbase transaction, its witness hash is assumed to be 0x0000....0000. + +A witness root hash is calculated with all those witness hashes as leaves, in a way similar to the hashMerkleRoot in the block header. + +No witness data is allowed in blocks that don't commit to witness data, as this would otherwise leave room from spam. + +==== Proposal 1: balance tree (the current implementation) ==== +The commitment is a push of exactly 41 bytes: +*The first 4 bytes are commitment header: 0xaa21a9ed +*The next 4 bytes describe a nonce +*The next 1 byte is the "levels bytes", describes the number of levels in a Merkle tree (0 to 32) +*The next 32 bytes is a commitment root hash (not the witness root hash) + +Locator of the witness root hash is defined as: + +
SHA256('WitnessV1\x00\x00\x00\x00\x00\x00\x00' || nonce)
+ +The first levels bits of locator, interpreted in little endian, are assumed to be the position in the leaves of the Merkle tree where the witness root hash goes. + +The coinbase's input's witness must consist of a single byte array of 32 * levels bytes, and are assumed to be the Merkle path to connect the witness root hash to the commitment root hash. + +====Proposal 2: imbalance tree==== +The commitment is a push of exactly 40 bytes: +*The first 4 bytes are commitment header: 0xaa21a9ed +*The next 4 bytes describe a nonce +*The next 32 bytes is a commitment root hash (not the witness root hash) + +Locator of the witness root hash is defined as: + +
SHA256('WitnessV1\x00\x00\x00\x00\x00\x00\x00' || nonce)
+ +The first levels bits of locator, interpreted in little endian, are assumed to be the position in the leaves of the Merkle tree where the witness root hash goes. + +The coinbase's input's witness must consist of a single byte array, which the size must be a multiple of 32 bytes but not bigger than 32*32 bytes. These are assumed to be the Merkle path to connect the witness root hash to the commitment root hash. Depth of the commitment is determined by the length of the path. If the depth of the tree is 0, the path should be represented by 0x00, and the witness root hash will be same as the commitment root hash. === Block size limit === -2-4-8? Discount for witness data? +A seperate BIP? +
+2-4-8?
+Discount for witness data?
+Start at softfork, moving to hardfork?
+
=== Witness program === @@ -136,14 +150,16 @@ used for multiple things. === Other consensus critical constraints === == Block size analysis == +WORK IN PROGRESS + Definitions:
  Core block size (CBS): The block size as seen by a non-upgrading full node
- Witness size (WS): The total size of witness in a block
+ Witness size (WS): The total size of witness data in a block
  Total block size (TBS): CBS + WS
- Witness discount (WD): A discount factor for witness for calculationg of VBS (1 = no discount)
+ Witness discount (WD): A discount factor for witness data in VBS calculation (1 = no discount)
  Virtual block size (VBS): CBS + (WS * WD)
- Witness adoption (WA): Proportion of new format transactions among all transactions
+ Witness adoption (WA): Proportion (in size) of new format transactions among all transactions
  Prunable ratio (PR): Proportion of signature data size in a transaction
 
@@ -155,8 +171,6 @@ With some transformation it could be shown that: In order to keep the proposal as a soft fork, the CBS must not have a upper limit higher than 1MB. -TBS is a function of only CBS, PR, and WA. - The PR heavily depends on the transaction script type and input-output ratio. For example, the PR of 1-in 2-out P2PKH and 1-in 1-out 2-of-2 multisig P2SH are about 47% and 72% respectively. According to the data presented by Pieter Wuille on 7 December 2015, the current average PR on the blockchain is about 60%. @@ -245,6 +259,11 @@ Examples of new script system include Schnorr signatures which reduce the size o The 41-byte limitation for witness programme could be easily extended through a soft fork in case a stronger hash function is needed in the future. The version byte is also expandable by introducing a secondary version byte for some specific primary version values. +=== Per-input lock-time and relative-lock-time === +Currently there is only one nLockTime field in a transaction and all inputs must share the same value. BIP68 enables per-input relative-lock-time using the nSequence field, however, with a limited lock-time period and resolution. + +With a soft fork, it is possible to introduce a separate witness structure to allow per-input lock-time and relative-lock-time, and a new script system that could sign and manipulate the new data (like BIP65 and BIP112). + == Backward compatibility == Without lifting the core block size limit of 1MB at the beginning, this proposal is a soft fork which all existing full nodes and SPV nodes are compatible. Non-upgrading nodes, however, will not see nor validate the witness data, and will consider all witness programs as anyone-can-spend scripts (except a few edge cases in version 0 witness programs which are provably unspendable with original script semantics). Non-upgrading nodes are also unable to generate or sign for scriptPubKey templates (payment address). Nonetheless, they may still pay to a witness program if it is nested in a P2SH address, which has been defined since 2012. From f026c28da5d32489c0c2346c33664d2b8a2f7d16 Mon Sep 17 00:00:00 2001 From: Eric Lombrozo Date: Tue, 22 Dec 2015 06:15:05 -0500 Subject: [PATCH 04/17] MULICHECKSIG -> CHECKMULTISIG n-of-m -> m-of-n LN motivation smaller SPV proof motivation --- bip-codeshark-jl2012-segwit.mediawiki | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki index cc87de3c..f7026d6c 100644 --- a/bip-codeshark-jl2012-segwit.mediawiki +++ b/bip-codeshark-jl2012-segwit.mediawiki @@ -20,11 +20,11 @@ The entirety of the transaction's effects are determined by output consumption ( By removing this data from the transaction structure committed to the transaction merkle tree, several problems are fixed: # '''Nonintentional malleability becomes impossible'''. Since signature data is no longer part of the transaction hash, changes to how the transaction was authorized is no longer relevant to transaction identification. As a solution of transaction malleability, this is superior to the canonical signature approach (BIP62): -#* It prevents involuntary transaction malleability for any type of scripts, as long as all inputs are signed (with at least one CHECKSIG or MULTICHECKSIG operation) -#* In the case of a n-of-m MULTICHECKSIG script, a transaction is malleable only with agreement of n private key holders (as opposed to only 1 private key holder with BIP62) +#* It prevents involuntary transaction malleability for any type of scripts, as long as all inputs are signed (with at least one CHECKSIG or CHECKMULTISIG operation) +#* In the case of an m-of-n CHECKMULTISIG script, a transaction is malleable only with agreement of m private key holders (as opposed to only 1 private key holder with BIP62) #* It prevents involuntary transaction malleability due to unknown ECDSA signature malleability -#* It allows creation of unconfirmed transaction dependency chains without counterparty risk -# '''Transmission of signature data becomes optional'''. It is needed only if a peer is trying to validate a transaction, instead of just to prove its existence. This also improves prviacy of SPV clients as using the same bandwidth they could request for more transactions for obscuration. +#* It allows creation of unconfirmed transaction dependency chains without counterparty risk, an important feature for offchain protocols such as the Lightning Network +# '''Transmission of signature data becomes optional'''. It is needed only if a peer is trying to validate a transaction instead of just checking its existence. This reduces the size of SPV proofs and potentially improves the privacy of SPV clients as they can download more transactions using the same bandwidth. # '''Some constriants could be bypassed with a soft fork''' by moving part of the transaction data to a structure unknown to current protocol, for example: #* Size of witness could be ignored / discounted when calculating the block size, effectively increasing the block size to some extent #* Hard coded constants, such as maximum data push size (520 bytes) or sigops limit could be reevaluated or removed From 26a7293e67740b6bba854d0feb2ed25875c9037d Mon Sep 17 00:00:00 2001 From: Johnson Lau Date: Tue, 22 Dec 2015 22:54:40 +0800 Subject: [PATCH 05/17] Update bip-segwit.mediawiki --- bip-codeshark-jl2012-segwit.mediawiki | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki index f7026d6c..5c0d28ea 100644 --- a/bip-codeshark-jl2012-segwit.mediawiki +++ b/bip-codeshark-jl2012-segwit.mediawiki @@ -123,12 +123,8 @@ The first levels bits of locator, interpreted in little endian, are assumed to b The coinbase's input's witness must consist of a single byte array, which the size must be a multiple of 32 bytes but not bigger than 32*32 bytes. These are assumed to be the Merkle path to connect the witness root hash to the commitment root hash. Depth of the commitment is determined by the length of the path. If the depth of the tree is 0, the path should be represented by 0x00, and the witness root hash will be same as the commitment root hash. === Block size limit === -A seperate BIP? -
-2-4-8?
 Discount for witness data?
-Start at softfork, moving to hardfork?
-
+Virtual block size limit? === Witness program === @@ -265,9 +261,7 @@ Currently there is only one nLockTime field in a transaction and all inputs must With a soft fork, it is possible to introduce a separate witness structure to allow per-input lock-time and relative-lock-time, and a new script system that could sign and manipulate the new data (like BIP65 and BIP112). == Backward compatibility == -Without lifting the core block size limit of 1MB at the beginning, this proposal is a soft fork which all existing full nodes and SPV nodes are compatible. Non-upgrading nodes, however, will not see nor validate the witness data, and will consider all witness programs as anyone-can-spend scripts (except a few edge cases in version 0 witness programs which are provably unspendable with original script semantics). Non-upgrading nodes are also unable to generate or sign for scriptPubKey templates (payment address). Nonetheless, they may still pay to a witness program if it is nested in a P2SH address, which has been defined since 2012. - -When the core block size limit is increased to over 1MB as scheduled, this proposal becomes a hard fork. Anybody running code that fully validates blocks must upgrade before the activation time or they will reject a chain containing blocks with core block size above 1MB. SPV software is not affected, unless it makes assumptions about the maximum depth of the Merkle tree based on the minimum size of a transaction and the maximum block size. +Without lifting the core block size limit of 1MB, this proposal is a soft fork which all existing full nodes and SPV nodes are compatible. Non-upgrading nodes, however, will not see nor validate the witness data, and will consider all witness programs as anyone-can-spend scripts (except a few edge cases in version 0 witness programs which are provably unspendable with original script semantics). Non-upgrading nodes are also unable to generate or sign for scriptPubKey templates (payment address). Nonetheless, they may still pay to a witness program if it is nested in a P2SH address, which has been defined since 2012. == Deployment == From 7f196396d34d5cf1ad36491f87880ff6ef25ff90 Mon Sep 17 00:00:00 2001 From: Eric Lombrozo Date: Tue, 22 Dec 2015 19:45:26 -0500 Subject: [PATCH 06/17] Updates to backward compatibility section --- bip-codeshark-jl2012-segwit.mediawiki | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki index 5c0d28ea..efed2b27 100644 --- a/bip-codeshark-jl2012-segwit.mediawiki +++ b/bip-codeshark-jl2012-segwit.mediawiki @@ -261,7 +261,15 @@ Currently there is only one nLockTime field in a transaction and all inputs must With a soft fork, it is possible to introduce a separate witness structure to allow per-input lock-time and relative-lock-time, and a new script system that could sign and manipulate the new data (like BIP65 and BIP112). == Backward compatibility == -Without lifting the core block size limit of 1MB, this proposal is a soft fork which all existing full nodes and SPV nodes are compatible. Non-upgrading nodes, however, will not see nor validate the witness data, and will consider all witness programs as anyone-can-spend scripts (except a few edge cases in version 0 witness programs which are provably unspendable with original script semantics). Non-upgrading nodes are also unable to generate or sign for scriptPubKey templates (payment address). Nonetheless, they may still pay to a witness program if it is nested in a P2SH address, which has been defined since 2012. +As a soft fork, older software will continue to operate without modification. Non-upgraded nodes, however, will not see nor validate the witness data and will consider all witness programs as anyone-can-spend scripts (except a few edge cases in version 0 witness programs which are provably unspendable with original script semantics). Wallets should always be wary of anyone-can-spend scripts and treat them with suspicion. Non-upgraded nodes are strongly encouraged to upgrade in order to take advantage of the new features. + +Wallets will be able to migrate in two phases: + +=== P2SH compatibility === +Witness programs are hashed to 256 bit "redeemscripts" which are then hashed to 160 bit P2SH. This format is fully compatible with currently existing wallets that support P2SH. Upgraded wallets will be able to send and receive to and from older wallets without any problems. + +=== Native outputs === +Witness programs are hashed to a 256 bit output. This format will not be compatible with older wallets but will require less block space and will have better security due to increased collision resistance. == Deployment == @@ -273,17 +281,16 @@ have nVersion >= 5. Furthermore, when 950 out of the 1000 blocks preceding a block do have nVersion >= 5, nVersion < 5 blocks become invalid, and all further blocks enforce the new rules. -It should be noted that BIP9 involves permanently setting a high-order bit to +(It should be noted that BIP9 involves permanently setting a high-order bit to 1 which results in nVersion >= all prior IsSuperMajority() soft-forks and thus -no bits in nVersion are permanently lost. +no bits in nVersion are permanently lost.) === SPV Clients === -While SPV clients are (currently) unable to validate blocks in general, -trusting miners to do validation for them, they are able to validate block -headers and thus can validate a subset of the deployment rules. SPV clients -should reject nVersion < 5 blocks if 950 out of 1000 preceding blocks have +While SPV clients are unable to fully validate blocks, +they are able to validate block headers and, thus, can check block version and proof-of-work. +SPV clients should reject nVersion < 5 blocks if 950 out of 1000 preceding blocks have nVersion >= 5 to prevent false confirmations from the remaining 5% of non-upgraded miners when the 95% threshold has been reached. From a208f3de00a7a79cf5cdc1731b6f7d166e799e37 Mon Sep 17 00:00:00 2001 From: Eric Lombrozo Date: Tue, 22 Dec 2015 21:06:10 -0500 Subject: [PATCH 07/17] Copyright statement --- bip-codeshark-jl2012-segwit.mediawiki | 3 +++ 1 file changed, 3 insertions(+) diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki index efed2b27..da624122 100644 --- a/bip-codeshark-jl2012-segwit.mediawiki +++ b/bip-codeshark-jl2012-segwit.mediawiki @@ -298,3 +298,6 @@ non-upgraded miners when the 95% threshold has been reached. == Reference Implementation == https://github.com/sipa/bitcoin/commits/segwit + +== Copyright == +This document is placed in the public domain. From 656b72695dc0a6d63cb691b02befc15039f7e5bc Mon Sep 17 00:00:00 2001 From: Eric Lombrozo Date: Tue, 22 Dec 2015 21:15:11 -0500 Subject: [PATCH 08/17] Credits --- bip-codeshark-jl2012-segwit.mediawiki | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki index da624122..df43c273 100644 --- a/bip-codeshark-jl2012-segwit.mediawiki +++ b/bip-codeshark-jl2012-segwit.mediawiki @@ -1,7 +1,7 @@
   BIP: x
   Title: Segregated Witness (Consensus layer)
-  Author: 
+  Authors: Eric Lombrozo , Johnson Lau , Pieter Wuille 
   Status: Draft
   Type: Standards Track
   Created: 2015-12
@@ -295,6 +295,7 @@ nVersion >= 5 to prevent false confirmations from the remaining 5% of
 non-upgraded miners when the 95% threshold has been reached.
 
 == Credits ==
+Special thanks to Gregory Maxwell for originating many of the ideas in this BIP and Luke-Jr for figuring out how to deploy this as a soft fork.
 
 == Reference Implementation ==
 https://github.com/sipa/bitcoin/commits/segwit

From ec059d15822228d11a2c21a14fe6b91d2307888b Mon Sep 17 00:00:00 2001
From: Eric Lombrozo 
Date: Tue, 22 Dec 2015 21:24:47 -0500
Subject: [PATCH 09/17] Wallet migration phase update

---
 bip-codeshark-jl2012-segwit.mediawiki | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki
index df43c273..72bb9806 100644
--- a/bip-codeshark-jl2012-segwit.mediawiki
+++ b/bip-codeshark-jl2012-segwit.mediawiki
@@ -263,12 +263,10 @@ With a soft fork, it is possible to introduce a separate witness structure to al
 == Backward compatibility ==
 As a soft fork, older software will continue to operate without modification.  Non-upgraded nodes, however, will not see nor validate the witness data and will consider all witness programs as anyone-can-spend scripts (except a few edge cases in version 0 witness programs which are provably unspendable with original script semantics). Wallets should always be wary of anyone-can-spend scripts and treat them with suspicion. Non-upgraded nodes are strongly encouraged to upgrade in order to take advantage of the new features.
 
-Wallets will be able to migrate in two phases:
-
-=== P2SH compatibility ===
+=== Version 0 witness programs ===
 Witness programs are hashed to 256 bit "redeemscripts" which are then hashed to 160 bit P2SH. This format is fully compatible with currently existing wallets that support P2SH. Upgraded wallets will be able to send and receive to and from older wallets without any problems.
 
-=== Native outputs ===
+=== Version 1 witness programs ===
 Witness programs are hashed to a 256 bit output. This format will not be compatible with older wallets but will require less block space and will have better security due to increased collision resistance.
 
 == Deployment ==

From 4c0863a2e406c7706b8ee44e6074d3bdf080a458 Mon Sep 17 00:00:00 2001
From: Eric Lombrozo 
Date: Tue, 22 Dec 2015 21:43:05 -0500
Subject: [PATCH 10/17] typo fix

---
 bip-codeshark-jl2012-segwit.mediawiki | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki
index 72bb9806..f44c6f1e 100644
--- a/bip-codeshark-jl2012-segwit.mediawiki
+++ b/bip-codeshark-jl2012-segwit.mediawiki
@@ -25,7 +25,7 @@ By removing this data from the transaction structure committed to the transactio
 #* It prevents involuntary transaction malleability due to unknown ECDSA signature malleability
 #* It allows creation of unconfirmed transaction dependency chains without counterparty risk, an important feature for offchain protocols such as the Lightning Network
 # '''Transmission of signature data becomes optional'''. It is needed only if a peer is trying to validate a transaction instead of just checking its existence. This reduces the size of SPV proofs and potentially improves the privacy of SPV clients as they can download more transactions using the same bandwidth.
-# '''Some constriants could be bypassed with a soft fork''' by moving part of the transaction data to a structure unknown to current protocol, for example:
+# '''Some constraints could be bypassed with a soft fork''' by moving part of the transaction data to a structure unknown to current protocol, for example:
 #* Size of witness could be ignored / discounted when calculating the block size, effectively increasing the block size to some extent
 #* Hard coded constants, such as maximum data push size (520 bytes) or sigops limit could be reevaluated or removed
 #* New script system could be introduced without any limitation from the existing script semantic

From 181280bed71acbee34ea5c83671b240ed28545dd Mon Sep 17 00:00:00 2001
From: Eric Lombrozo 
Date: Tue, 22 Dec 2015 21:52:55 -0500
Subject: [PATCH 11/17] Added block size limit

---
 bip-codeshark-jl2012-segwit.mediawiki | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki
index f44c6f1e..61f341e7 100644
--- a/bip-codeshark-jl2012-segwit.mediawiki
+++ b/bip-codeshark-jl2012-segwit.mediawiki
@@ -123,8 +123,11 @@ The first levels bits of locator, interpreted in little endian, are assumed to b
 The coinbase's input's witness must consist of a single byte array, which the size must be a multiple of 32 bytes but not bigger than 32*32 bytes. These are assumed to be the Merkle path to connect the witness root hash to the commitment root hash. Depth of the commitment is determined by the length of the path. If the depth of the tree is 0, the path should be represented by 0x00, and the witness root hash will be same as the commitment root hash.
 
 === Block size limit ===
-Discount for witness data?
-Virtual block size limit?
+Blocks are currently limited to 1 MB total size. We change this restriction as follows:
+
+We define a base block size sb consisting of the existing header and transactions, a witness size sw consisting of only the size of the witness data, and a virtual block size sv = sb + sw/4.
+
+The new rule is sv <= 1 MB.
 
 === Witness program ===
 

From 8fd3a42a954683268d158639aa7d602309950720 Mon Sep 17 00:00:00 2001
From: Eric Lombrozo 
Date: Wed, 23 Dec 2015 09:57:37 -0500
Subject: [PATCH 12/17] Removed peer services specification.

---
 bip-codeshark-jl2012-segwit.mediawiki | 44 ---------------------------
 1 file changed, 44 deletions(-)

diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki
index 61f341e7..bbbf58cd 100644
--- a/bip-codeshark-jl2012-segwit.mediawiki
+++ b/bip-codeshark-jl2012-segwit.mediawiki
@@ -34,50 +34,6 @@ By removing this data from the transaction structure committed to the transactio
 #* Sum trees for transaction inputs and outputs can be committed making it possible to construct short proofs that no new coins are created in any noncoinbase transaction and that the miner does not add excessive fees to the coinbase transaction. 
 
 ==Specification==
-Maybe a seperate BIP
-
-* CTransaction gets, in addition to vin and vout, a vwit, which
-contains a CTxInWitness object for each input. A CTxInWitness contains a CScriptWitness object
-and can potentially be extended to contain other kinds of witness data.
-A CScriptWitness is a vector of byte vectors (nominally: the input stack to the program, no longer
-encoded as a CScript, but just the resulting stack directly).
-* A new serialization for CTransaction is defined: http://blockhawk.net/diagrams/witnesstx.png
-(int32 nVersion,
-0x00 marker, 0x01 flag, vector, vector,
-vector, int32 nLockTime) instead of (int32 nVersion,
-vector, vector, int32 nLockTime). This will never parse
-as a valid transaction (even if parsing succeeds, it means it's
-interpreted as a transaction with no inputs and 1 output). If all
-witnesses are empty, the old serialization format is used.
-- Rationale for not having an independent CWitnessTransaction with
-its own serialization: this would require separate "tx" and "block"
-messages, and all RPC calls operating on raw transactions would need
-to be duplicated, or need inefficinent or nondeterministic guesswork
-to know which type is to be used.
-- Rationale for not using just a single 0x00 byte as marker: that
-would lead to empty transactions (no inputs, no outputs, which are
-used in some tests) to be interpreted as new serialized data.
-- Rationale for the 0x01 flag byte in between: this will allow us to
-easily add more extra non-committed data to transactions (like txouts
-being spent, ...). It can be interpreted as a bitvector.
-
-* A new message 'havewitness' is sent after receiving 'verack' to
-indicate that a node can provide witness if requested (similar to
-'sendheaders')
-
-* New inv types MSG_WITNESS_TX and MSG_WITNESS_BLOCK are added, only
-for use in getdata. Inv itself still use just MSG_TX and MSG_BLOCK,
-similar to MSG_FILTERED_BLOCK.
-- Rationale for not advertizing witnessness in invs: we don't always
-use invs anymore (with 'sendheaders' BIP 130), plus it's not useful:
-implicitly, every transaction and block have a witness, old ones just
-have empty ones.
-
-* Transactions' GetHash is always computed on the old non-witness
-serialization. A new CTransaction::GetWitnessHash is added which is
-computed from the witness-serialization (this means that transactions
-with an empty witness have witness hash equal to normal hash).
-
=== Commitment structure === From 894d8ac1cb8d2717b865d36b342f58fe7253a2c0 Mon Sep 17 00:00:00 2001 From: Eric Lombrozo Date: Wed, 23 Dec 2015 09:58:58 -0500 Subject: [PATCH 13/17] Moved authors to separate lines. --- bip-codeshark-jl2012-segwit.mediawiki | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki index bbbf58cd..3bddf8a6 100644 --- a/bip-codeshark-jl2012-segwit.mediawiki +++ b/bip-codeshark-jl2012-segwit.mediawiki @@ -1,10 +1,12 @@
   BIP: x
   Title: Segregated Witness (Consensus layer)
-  Authors: Eric Lombrozo , Johnson Lau , Pieter Wuille 
+  Author: Eric Lombrozo 
+          Johnson Lau 
+          Pieter Wuille 
   Status: Draft
   Type: Standards Track
-  Created: 2015-12
+  Created: 2015-12-21
 
==Abstract== From 3e3006127e7d9d6a4f172d692df5195cf740cf71 Mon Sep 17 00:00:00 2001 From: Johnson Lau Date: Tue, 29 Dec 2015 01:32:35 +0800 Subject: [PATCH 14/17] Clarify Backward compatibility --- bip-codeshark-jl2012-segwit.mediawiki | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki index 3bddf8a6..643b9d13 100644 --- a/bip-codeshark-jl2012-segwit.mediawiki +++ b/bip-codeshark-jl2012-segwit.mediawiki @@ -224,11 +224,14 @@ With a soft fork, it is possible to introduce a separate witness structure to al == Backward compatibility == As a soft fork, older software will continue to operate without modification. Non-upgraded nodes, however, will not see nor validate the witness data and will consider all witness programs as anyone-can-spend scripts (except a few edge cases in version 0 witness programs which are provably unspendable with original script semantics). Wallets should always be wary of anyone-can-spend scripts and treat them with suspicion. Non-upgraded nodes are strongly encouraged to upgrade in order to take advantage of the new features. -=== Version 0 witness programs === -Witness programs are hashed to 256 bit "redeemscripts" which are then hashed to 160 bit P2SH. This format is fully compatible with currently existing wallets that support P2SH. Upgraded wallets will be able to send and receive to and from older wallets without any problems. +'''What a non-upgraded wallet can do''' +* Receiving bitcoin from non-upgraded and upgraded wallets +* Sending bitcoin to non-upgraded wallets +* Sending bitcoin to upgraded wallets using a P2SH address (a less efficient way to use segregated witness) -=== Version 1 witness programs === -Witness programs are hashed to a 256 bit output. This format will not be compatible with older wallets but will require less block space and will have better security due to increased collision resistance. +'''What a non-upgraded wallet cannot do''' +* Validating segregated witness transaction. It assumes such a transaction is always valid +* Sending bitcoin to upgraded wallets using a native witness program (a more efficient way to use segregated witness) == Deployment == From 66ba4e6fcf6b8752ec38b78a85c7f2120148d856 Mon Sep 17 00:00:00 2001 From: Johnson Lau Date: Tue, 29 Dec 2015 11:11:16 +0800 Subject: [PATCH 15/17] Separating version byte and witness program --- bip-codeshark-jl2012-segwit.mediawiki | 35 ++++++++++++++------------- 1 file changed, 18 insertions(+), 17 deletions(-) diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki index 643b9d13..ce85c534 100644 --- a/bip-codeshark-jl2012-segwit.mediawiki +++ b/bip-codeshark-jl2012-segwit.mediawiki @@ -89,20 +89,19 @@ The new rule is sv <= 1 MB. === Witness program === -* A scriptPubKey (or redeemScript as defined in BIP16/P2SH) that consists of a single push of 2 to 41 bytes gets a new special meaning. The byte vector pushed by it is called the "witness program". -** In case the scriptPubKey pushes a witness program directly, the scriptSig must be exactly empty. -** In case the redeemScript pushes a witness program, the scriptSig must be exactly the single push of the redeemScript. +* A scriptPubKey (or redeemScript as defined in BIP16/P2SH) that consists of a 1-byte push opcode (for 0 to 16) followed by a data push between 2 and 32 bytes gets a new special meaning. The value of the first push is called the "version byte". The following byte vector pushed is called the "witness program". +** In case the scriptPubKey pushes a version byte and witness program directly, the scriptSig must be exactly empty. +** In case the redeemScript pushes a version byte and witness program, the scriptSig must be exactly the single push of the redeemScript. -* The first byte of a witness program is the "version byte", an unsigned integer. -** If the version byte is 0, the rest of the witness program is the actual script. +** If the version byte is 0, the WP is the actual script. *** The script is executed after normal script evaluation but with data from the witness rather than the scriptSig. *** The program must not fail, and result in exactly a single TRUE on the stack. -** If the version byte is 1, the rest of the witness program must be 32 bytes, as a SHA256 hash of the actual script. +** If the version byte is 1, the WP must be 32 bytes, as a SHA256 hash of the actual script. *** The witness must consist of an input stack to feed to the program, followed by the serialized program. *** The serialized program is popped off the initial witness stack. Hash of the serialized program must match the hash pushed in the witness program. *** The serialized program is deserialized, and executed after normal script evaluation with the remaining witness stack. *** The script must not fail, and result in exactly a single TRUE on the stack. -** If the witness version byte is 2 or above, no further interpretation of the witness program or witness happens. +** If the version byte is 2 to 16, no further interpretation of the witness program or witness happens. === Other consensus critical constraints === @@ -140,9 +139,9 @@ The following example is a version 0 witness program, equivalent to the existing witness: scriptSig: (empty) - scriptPubKey: <0x0076A914{20-byte-hash-value}88AC> + scriptPubKey: OP_0 <0x76A914{20-byte-hash-value}88AC> -The version byte 0x00 is removed. The rest of the witness program is deserialized and becomes: +The OP_0 indicates the following push is a version 0 witness program. The witness program is deserialized and becomes: DUP HASH160 <20byte-hash-value> EQUALVERIFY CHECKSIG @@ -158,9 +157,9 @@ The following example is an 1-of-2 multi-signature version 1 witness program. witness: 0 <0x5121{33-byte-pubkey1}21{33-byte-pubkey2}52AE> scriptSig: (empty) - scriptPubKey: <0x01{32-byte-hash-value}> + scriptPubKey: OP_1 <0x{32-byte-hash-value}> -The last item in the witness is popped off, hashed with SHA256, compared against the 32-byte-hash-value in scriptPubKey, and deserialized: +The OP_1 in scriptPubKey indicates the following push is a version 1 witness program. The last item in the witness is popped off, hashed with SHA256, compared against the 32-byte-hash-value in scriptPubKey, and deserialized: 1 <33-byte-pubkey1> <33-byte-pubkey2> 2 CHECKMULTISIG @@ -168,7 +167,7 @@ The script is executed with the remaining data from witness: 0 1 <33-byte-pubkey1> <33-byte-pubkey2> 2 CHECKMULTISIG -Since the actual program is larger than 40 bytes, it cannot be accommodated in a version 0 witness program. A version 1 witness program allows arbitrarily large script as the 520-byte push limit is bypassed. +Since the actual program is larger than 32 bytes, it cannot be accommodated in a version 0 witness program. A version 1 witness program allows arbitrarily large script as the 520-byte push limit is bypassed. The scriptPubKey occupies 34 bytes, as opposed to 23 bytes of P2SH. The increased size improves security against possible collision attacks, as 2^80 work is not infeasible anymore (By the end of 2015, 2^84 hashes have been calculated in Bitcoin mining since the creation of Bitcoin). The spending script is same as the one for an equivalent P2SH output but is moved to witness. @@ -177,12 +176,12 @@ The scriptPubKey occupies 34 bytes, as opposed to 23 bytes of P2SH. The increase The following example is the same 1-of-2 multi-signature version 1 witness program, but nested in a P2SH output. witness: 0 <0x5121{33-byte-pubkey1}21{33-byte-pubkey2}52AE> - scriptSig: <0x2101{32-byte-hash-value}> + scriptSig: <0x5120{32-byte-hash-value}> scriptPubKey: HASH160 <20-byte-hash-value> EQUAL -The only item in scriptSig is hashed with HASH160, compared against the 20-byte-hash-value in scriptPubKey, and interpreted as a single push of: +The only item in scriptSig is hashed with HASH160, compared against the 20-byte-hash-value in scriptPubKey, and interpreted as: - <0x01{32-byte-hash-value}> + OP_1 <0x{32-byte-hash-value}> The version 1 witness program is then executed as described in the last example @@ -210,11 +209,11 @@ In the current Bitcoin protocol, it is possible to generate compact fraud proof It is possible to proof the first 2 types of fraud if a block is committed to a Merkle-sum-tree of the fee, size, and sigop count of each transaction. It is also possible to proof the last type of fraud if a block is committed to a Merkle tree with the originating block height and transaction index of all inputs. These commitments could be included in the extensible witness commitment through a soft fork and will be transparent to nodes that do not understand such new rules. === New script system === -Since all witness programs begin with a version byte, and programs with unknown versions are always considered as anyone-can-spend script, it is possible to introduce any new script system with a soft fork. The witness as a structure is not restricted by any existing script semantics and constraints, the 520-byte push limit in particular, and therefore allows arbitrarily large scripts and signatures. +Since a version byte is pushed before a witness program, and programs with unknown versions are always considered as anyone-can-spend script, it is possible to introduce any new script system with a soft fork. The witness as a structure is not restricted by any existing script semantics and constraints, the 520-byte push limit in particular, and therefore allows arbitrarily large scripts and signatures. Examples of new script system include Schnorr signatures which reduce the size of multisig transactions dramatically, Lamport signature which is quantum computing resistance, and Merklized abstract syntax trees which allow very compact witness for conditional scripts with extreme complexity. -The 41-byte limitation for witness programme could be easily extended through a soft fork in case a stronger hash function is needed in the future. The version byte is also expandable by introducing a secondary version byte for some specific primary version values. +The 32-byte limitation for witness program could be easily extended through a soft fork in case a stronger hash function is needed in the future. The version byte is also expandable through a softfork. === Per-input lock-time and relative-lock-time === Currently there is only one nLockTime field in a transaction and all inputs must share the same value. BIP68 enables per-input relative-lock-time using the nSequence field, however, with a limited lock-time period and resolution. @@ -225,11 +224,13 @@ With a soft fork, it is possible to introduce a separate witness structure to al As a soft fork, older software will continue to operate without modification. Non-upgraded nodes, however, will not see nor validate the witness data and will consider all witness programs as anyone-can-spend scripts (except a few edge cases in version 0 witness programs which are provably unspendable with original script semantics). Wallets should always be wary of anyone-can-spend scripts and treat them with suspicion. Non-upgraded nodes are strongly encouraged to upgrade in order to take advantage of the new features. '''What a non-upgraded wallet can do''' + * Receiving bitcoin from non-upgraded and upgraded wallets * Sending bitcoin to non-upgraded wallets * Sending bitcoin to upgraded wallets using a P2SH address (a less efficient way to use segregated witness) '''What a non-upgraded wallet cannot do''' + * Validating segregated witness transaction. It assumes such a transaction is always valid * Sending bitcoin to upgraded wallets using a native witness program (a more efficient way to use segregated witness) From 2f06ee5006e5fa69d52f0d7c9d60ad510101ab38 Mon Sep 17 00:00:00 2001 From: alp-bitcoin Date: Thu, 31 Dec 2015 15:01:27 -0600 Subject: [PATCH 16/17] Fixed typos. --- bip-codeshark-jl2012-segwit.mediawiki | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-codeshark-jl2012-segwit.mediawiki index 3bddf8a6..8bdade79 100644 --- a/bip-codeshark-jl2012-segwit.mediawiki +++ b/bip-codeshark-jl2012-segwit.mediawiki @@ -203,11 +203,11 @@ Bitcoin right now only has two real security models. A user either runs a full-n In the current Bitcoin protocol, it is possible to generate compact fraud proof for almost all rules except a few: -# It is not possible to proof a miner has introduced too many Bitcoins in the coinbase transaction outputs without showing the whole block itself and all input transactions. +# It is not possible to prove a miner has introduced too many Bitcoins in the coinbase transaction outputs without showing the whole block itself and all input transactions. # It is not possible to prove the violation of any block specific constraints, such as size and sigop limits, without showing the whole block (and all input transactions in the case of sigop limit) # It is not possible to prove the spending of a non-existing input without showing all transaction IDs in the blockchain way back to the genesis block. -It is possible to proof the first 2 types of fraud if a block is committed to a Merkle-sum-tree of the fee, size, and sigop count of each transaction. It is also possible to proof the last type of fraud if a block is committed to a Merkle tree with the originating block height and transaction index of all inputs. These commitments could be included in the extensible witness commitment through a soft fork and will be transparent to nodes that do not understand such new rules. +It is possible to prove the first 2 types of fraud if a block is committed to a Merkle-sum-tree of the fee, size, and sigop count of each transaction. It is also possible to prove the last type of fraud if a block is committed to a Merkle tree with the originating block height and transaction index of all inputs. These commitments could be included in the extensible witness commitment through a soft fork and will be transparent to nodes that do not understand such new rules. === New script system === Since all witness programs begin with a version byte, and programs with unknown versions are always considered as anyone-can-spend script, it is possible to introduce any new script system with a soft fork. The witness as a structure is not restricted by any existing script semantics and constraints, the 520-byte push limit in particular, and therefore allows arbitrarily large scripts and signatures. From b16a75072d11b367d97d2676d260a5cd6a907adf Mon Sep 17 00:00:00 2001 From: Luke Dashjr Date: Fri, 8 Jan 2016 17:53:59 +0000 Subject: [PATCH 17/17] Assign BIP 141 --- README.mediawiki | 6 ++++++ bip-codeshark-jl2012-segwit.mediawiki => bip-0141.mediawiki | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) rename bip-codeshark-jl2012-segwit.mediawiki => bip-0141.mediawiki (99%) diff --git a/README.mediawiki b/README.mediawiki index b45c17a0..af982a5e 100644 --- a/README.mediawiki +++ b/README.mediawiki @@ -379,6 +379,12 @@ Those proposing changes should consider that ultimately consent may rest with th | Suhas Daftuar | Standard | Draft +|- +| [[bip-0141.mediawiki|141]] +| Segregated Witness (Consensus layer) +| Eric Lombrozo, Johnson Lau, and Pieter Wuille +| Standard +| Draft |} diff --git a/bip-codeshark-jl2012-segwit.mediawiki b/bip-0141.mediawiki similarity index 99% rename from bip-codeshark-jl2012-segwit.mediawiki rename to bip-0141.mediawiki index dd81c1d0..ac44d870 100644 --- a/bip-codeshark-jl2012-segwit.mediawiki +++ b/bip-0141.mediawiki @@ -1,5 +1,5 @@
-  BIP: x
+  BIP: 141
   Title: Segregated Witness (Consensus layer)
   Author: Eric Lombrozo 
           Johnson Lau