bips/bip-0322.mediawiki

<pre>
  BIP: 322
  Layer: Applications
  Title: Generic Signed Message Format
  Authors: Karl-Johan Alm <karljohan-alm@garage.co.jp>
  Comments-Summary: No comments yet.
  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0322
  Status: Draft
  Type: Specification
  Assigned: 2018-09-10
  License: CC0-1.0
</pre>

== Abstract ==

A standard for interoperable signed messages based on the Bitcoin Script format, either for proving
fund availability, or committing to a message as the intended recipient of funds sent to the invoice
address.

== Motivation ==

The current message signing standard only works for P2PKH (1...) invoice addresses. We propose to
extend and generalize the standard by using a Bitcoin Script based approach. This ensures that any
coins, no matter what script they are controlled by, can in-principle be signed for. For easy
interoperability with existing signing hardware, we also define a signature message format which
resembles a Bitcoin transaction (except that it contains an invalid input, so it cannot be spent on
any real network).

The Proof of Funds variant allows demonstrating control of a set of UTXOs.
The list of UTXOs may or may not be related to the address being signed with (the
<code>message_challenge</code>).
But in any case, the UTXO list does not aim to prove completeness (e.g. it does NOT mean:
"these are all UTXOs that exist for an address") nor that they are unspent (e.g. a
validator must consult the blockchain to verify that).

Additionally, the current message signature format uses ECDSA signatures which do not commit to the
public key, meaning that they do not actually prove knowledge of any secret keys. (Indeed, valid
signatures can be tweaked by 3rd parties to become valid signatures on certain related keys.)

Ultimately no message signing protocol can actually prove control of funds, both because a signature
is obsolete as soon as it is created, and because the possessor of a secret key may be willing to
sign messages on others' behalf even if it would not sign actual transactions. No message signing
protocol can fix these limitations.

Finally, this BIP only addresses the use case where a signer shows they will be able to control
funds sent to the invoice address. Proving that a signer sent a prior transaction is not possible
using this BIP.

== Terminology ==

In the context of this BIP, whenever the word "signature" or similar is used, it refers to the
output of the signing process described below and, depending on the script type of the
<code>message_challenge</code>, is either a full transaction input witness stack, a full
transaction, or a PSBT packet that can be validated against a Bitcoin Script Interpreter. Such a
"signature" may or may not contain an actual cryptographic (ECDSA or Schnorr) signature, depending
on what is required to satisfy the script corresponding to the <code>message_challenge</code>.

== Types of Signatures ==

This BIP specifies three formats for signing messages: ''legacy'', ''simple'' and ''full''.
Additionally, a variant of the ''full'' format can be used to demonstrate control over a set of
UTXOs.

{| class="wikitable"
|- style="font-weight:bold;"
!
! Compatible script types
! Signature format
|-
| Legacy
| <code>P2PKH</code>, <code>P2SH-P2WPKH</code><sup>1</sup>, <code>P2WPKH</code><sup>1</sup>
| compact, public key recoverable ECDSA signature, base64-encoded
|-
| Simple
| <code>P2WPKH</code>, <code>P2WSH</code><sup>2</sup>, <code>P2TR</code><sup>2</sup> <br/>
| witness stack, consensus encoded and base64-encoded
|-
| Full
| <code>all</code>
| full <code>to_sign</code> transaction, consensus and base64-encoded
|-
| Full (PoF)
| <code>all</code>
| full <code>to_sign</code> transaction, consensus and base64-encoded
|}

<sup>1</sup>: Possible on a technical level but should NOT be used anymore in the context of this
BIP.<br/>
<sup>2</sup>: Excluding time lock scripts.

=== Legacy ===

New proofs should use the new format for all invoice address formats, including P2PKH.

The legacy format MAY be used, but must be restricted to the legacy P2PKH invoice address format.

=== Simple ===

A ''simple'' signature consists of a witness stack, consensus encoded as a vector of vectors of
bytes, and base64-encoded. Validators should construct <code>to_spend</code> and
<code>to_sign</code> as defined below, with default values for all fields except that

<ul>
<li>
  <code>message_hash</code> is a BIP340-tagged hash of the message, as specified below
</li>
<li>
  <code>message_challenge</code> in <code>to_spend</code> is set to the scriptPubKey being signed
  with
</li>
<li>
  <code>message_signature</code> in <code>to_sign</code> is set to the provided simple signature.
</li>
</ul>

and then proceed as they would for a full signature.

=== Full ===

Full signatures follow an analogous specification to the BIP-325 challenges and solutions used by
Signet.

Let there be two virtual transactions <code>to_spend</code> and <code>to_sign</code>.

The <code>to_spend</code> transaction is:

    nVersion = 0
    nLockTime = 0
    vin[0].prevout.hash = 0000...000
    vin[0].prevout.n = 0xFFFFFFFF
    vin[0].nSequence = 0
    vin[0].scriptSig = OP_0 PUSH32[ message_hash ]
    vin[0].scriptWitness = []
    vout[0].nValue = 0
    vout[0].scriptPubKey = message_challenge

where <code>message_hash</code> is a BIP340-tagged hash of the message, i.e. sha256_tag(m), where
tag = <code>BIP0322-signed-message</code> and <code>m</code> is the message as is without length
prefix or null terminator, and <code>message_challenge</code> is the to be proven (public) key
script.

The <code>to_sign</code> transaction is:

    nVersion = 0 or (FULL format only) as appropriate (e.g. 2, for time locks)
    nLockTime = 0 or (FULL format only) as appropriate (for time locks)
    vin[0].prevout.hash = to_spend.txid
    vin[0].prevout.n = 0
    vin[0].nSequence = 0 or (FULL format only) as appropriate (for time locks)
    vin[0].scriptSig = [] or (FULL format only) as appropriate (for non segwit-native transactions)
    vin[0].scriptWitness = message_signature
    vout[0].nValue = 0
    vout[0].scriptPubKey = OP_RETURN

A full signature consists of the base64-encoding of the <code>to_sign</code> transaction in standard
network serialisation once it has been signed.

=== Full (Proof of Funds) ===

A signer may construct a proof of funds, demonstrating control of a set of UTXOs, by constructing a
full signature as above, with the following modifications.

<ul>
<li>
  All outputs that the signer wishes to demonstrate control of are included as additional inputs
  of <code>to_sign</code>, and their witness and scriptSig data should be set as though these
  outputs were actually being spent.
</li>
</ul>

Unlike an ordinary signature, validators of a proof of funds need access to the current UTXO set, to
learn that the claimed inputs exist on the blockchain and remain unspent.
An offline validator therefore can only attest to the cryptographic validity of the additional
inputs' witness stack, but not its blockchain state.
An attested list of UTXOs can also never prove that there don't exist more UTXOs for a certain
address.

== Detailed Specification ==

For all signature types, except legacy, the <code>to_spend</code> and <code>to_sign</code>
transactions must be valid transactions which pass all consensus checks, except of course that the
output with prevout <code>000...000:FFFFFFFF</code> does not exist.

=== Verification ===

A validator is given as input an address ''A'' (which may be omitted in a proof-of-funds), signature
''s'' and message ''m'', and outputs one of three states

<ul>
<li>
  ''valid at time T and age S'' indicates that the signature has set timelocks but is otherwise
  valid
</li>
<li>
  ''inconclusive'' means the validator was unable to check the scripts
</li>
<li>
  ''invalid'' means that some check failed
</li>
</ul>

==== Verification Process ====

Validation consists of the following steps:

# Basic validation
## Compute the transaction <code>to_spend</code> from ''m'' and ''A''
## Decode ''s'' as the transaction <code>to_sign</code>
## If ''s'' was a full transaction, confirm all fields are set as specified above; in particular that
##* <code>to_sign</code> has at least one input and its first input spends the output of </code>to_spend</code>
##* <code>to_sign</code> has exactly one output, as specified above
## Confirm that the two transactions together satisfy all consensus rules, except for <code>to_spend</code>'s missing input, and except that ''nSequence'' of <code>to_sign</code>'s first input and ''nLockTime'' of <code>to_sign</code> are not checked.
# (Optional) If the validator does not have a full script interpreter, it should check that it understands all scripts being satisfied. If not, it should stop here and output ''inconclusive''.
# Check the '''required rules''':
## All signatures must use the SIGHASH_ALL flag.
## The use of <code>CODESEPARATOR</code> or <code>FindAndDelete</code> is forbidden.
## <code>LOW_S</code>, <code>STRICTENC</code> and <code>NULLFAIL</code>: valid ECDSA signatures must be strictly DER-encoded and have a low-S value; invalid ECDSA signature must be the empty push
## <code>MINIMALDATA</code>: all pushes must be minimally encoded
## <code>CLEANSTACK</code>: require that only a single stack element remains after evaluation
## <code>MINIMALIF</code>: the argument of <code>IF</code>/<code>NOTIF</code> must be exactly 0x01 or empty push
## If any of the above steps failed, the validator should stop and output the ''invalid'' state.
# Check the '''upgradeable rules'''
## The version of <code>to_sign</code> must be 0 or 2.
## The use of NOPs reserved for upgrades is forbidden.
## The use of Segwit versions greater than 1 are forbidden.
## If any of the above steps failed, the validator should stop and output the ''inconclusive'' state.
# Let ''T'' by the nLockTime of <code>to_sign</code> and ''S'' be the nSequence of the first input of <code>to_sign</code>. Output the state ''valid at time T and age S''.

=== Signing ===

Signers who control an address ''A'' who wish to sign a message ''m'' act as follows:

<ol>
<li>
  They construct <code>to_spend</code> and <code>to_sign</code> as specified above, using the
  scriptPubKey of ''A'' for <code>message_challenge</code> and tagged hash of ''m'' as
  <code>message_hash</code>.
</li>
<li>
  Optionally, they may set nVersion/nLockTime of <code>to_sign</code> or nSequence of its first
  input.
</li>
<li>
  Optionally, they may add any additional inputs to <code>to_sign</code> that they wish to prove
  control of.
</li>
<li>
  They satisfy <code>to_sign</code> as they would any other transaction.
</li>
</ol>

They then encode their signature, choosing either ''simple'' or ''full'' as follows:

<ul>
<li>
  If they added no inputs to <code>to_sign</code>, left nVersion, nSequence and nLockTime at 0, and
  ''A'' is a "native" Segwit address (P2WPKH, P2WSH, P2TR), then they may base64-encode
  <code>message_signature</code>
</li>
<li>
  Otherwise they must base64-encode <code>to_sign</code>.
</li>
</ul>

== Compatibility ==

This specification is backwards compatible with the legacy signmessage/verifymessage specification
through the special case as described above.

== Reference implementation ==

<ul>
<li>
  Bitcoin Core pull request (basic support) at: https://github.com/bitcoin/bitcoin/pull/24058
</li>
<li>
  btcd pull request (complete support, source of test vectors) at:
  https://github.com/btcsuite/btcd/pull/2521
</li>
</ul>

== Acknowledgements ==

Thanks to David Harding, Jim Posen, Kalle Rosenbaum, Pieter Wuille, Andrew Poelstra, Luke Dashjr,
and many others for their feedback on the specification.

== References ==

<ol>
<li>
  Original mailing list thread:
  https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-March/015818.html
</li>
</ol>

== Copyright ==

This document is licensed under the Creative Commons CC0 1.0 Universal license.

== Test vectors ==

Basic test vectors for message hashing, transaction hashes and "simple" variant test cases can be
found in [[bip-0322/basic-test-vectors.json|<code>basic-test-vectors.json</code>]].

Generated test vectors for more "simple" and "full" variant test cases can be found in
[[bip-0322/generated-test-vectors.json|<code>generated-test-vectors.json</code>]].

They were generated using
[https://github.com/guggero/btcd/blob/f0d87198/btcutil/bip322/bip322_test.go#L910 this code].