Switch to 32 byte public keys in bip-schnorr

bip-schnorr.mediawiki

@@ -35,6 +35,7 @@ propose a new standard, a number of improvements not specific to Schnorr signatu
 made:
 
 * '''Signature encoding''': Instead of [https://en.wikipedia.org/wiki/X.690#DER_encoding DER]-encoding for signatures (which are variable size, and up to 72 bytes), we can use a simple fixed 64-byte format.
+* '''Public key encoding''': Instead of ''compressed'' 33-byte encoding of elliptic curve points which are common in Bitcoin, public keys in this proposal are encoded as 32 bytes.
 * '''Batch verification''': The specific formulation of ECDSA signatures that is standardized cannot be verified more efficiently in batch compared to individually, unless additional witness data is added. Changing the signature scheme offers an opportunity to avoid this.
 
 [[File:bip-schnorr/speedup-batch.png|frame|This graph shows the ratio between the time it takes to verify ''n'' signatures individually and to verify a batch of ''n'' signatures. This ratio goes up logarithmically with the number of signatures, or in other words: the total time to verify ''n'' signatures grows with ''O(n / log n)''.]]
@@ -57,24 +58,35 @@ We choose the ''R''-option to support batch verification.
 
 '''Key prefixing''' When using the verification rule above directly, it is possible for a third party to convert a signature ''(R,s)'' for key ''P'' into a signature ''(R,s + aH(R || m))'' for key ''P + aG'' and the same message, for any integer ''a''. This is not a concern for Bitcoin currently, as all signature hashes indirectly commit to the public keys. However, this may change with proposals such as SIGHASH_NOINPUT ([https://github.com/bitcoin/bips/blob/master/bip-0118.mediawiki BIP 118]), or when the signature scheme is used for other purposes&mdash;especially in combination with schemes like [https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki BIP32]'s unhardened derivation. To combat this, we choose ''key prefixed''<ref>A limitation of committing to the public key (rather than to a short hash of it, or not at all) is that it removes the ability for public key recovery or verifying signatures against a short public key hash. These constructions are generally incompatible with batch verification.</ref> Schnorr signatures; changing the equation to ''sG = R + H(R || P || m)P''.
 
-'''Encoding the sign of R''' As we chose the ''R''-option above, we're required to encode the point ''R'' into the signature. Several possibilities exist:
+'''Encoding the sign of P and the sign of R''' There exist several possibilities for encoding the sign of the public key point:
+# Encoding the full X and Y coordinate of P, resulting in a 64-byte public key.
+# Encoding the full X coordinate, but only whether Y is even or odd (like compressed public keys). This would result in 33-byte public keys.
+# Encoding only the X coordinate, resulting in 32-byte public keys.
+
+As we chose the ''R''-option above, we're required to encode the point ''R'' into the signature. The possibilities are similar to the encoding of ''P'':
 # Encoding the full X and Y coordinate of R, resulting in a 96-byte signature.
 # Encoding the full X coordinate, but only whether Y is even or odd (like compressed public keys). This would result in 65-byte signatures.
 # Encoding only the X coordinate, leaving us with 64-byte signature.
 
-Using the first option would be slightly more efficient for verification (around 5%), but we prioritize compactness, and therefore choose option 3.
+Using the first option for both ''P'' and ''R'' would be slightly more efficient for verification (around 5%), but we prioritize compactness, and therefore choose option 3.
 
-'''Implicit Y coordinate''' In order to support batch verification, the Y coordinate of ''R'' cannot be ambiguous (every valid X coordinate has two possible Y coordinates). We have a choice between several options for symmetry breaking:
+'''Implicit Y coordinates''' In order to support batch verification, the Y coordinate of ''P'' and of ''R'' cannot be ambiguous (every valid X coordinate has two possible Y coordinates). We have a choice between several options for symmetry breaking:
 # Implicitly choosing the Y coordinate that is in the lower half.
 # Implicitly choosing the Y coordinate that is even<ref>Since ''p'' is odd, negation modulo ''p'' will map even numbers to odd numbers and the other way around. This means that for a valid X coordinate, one of the corresponding Y coordinates will be even, and the other will be odd.</ref>.
 # Implicitly choosing the Y coordinate that is a quadratic residue (has a square root modulo the field size)<ref>A product of two numbers is a quadratic residue when either both or none of the factors are quadratic residues. As ''-1'' is not a quadratic residue, and the two Y coordinates corresponding to a given X coordinate are each other's negation, this means exactly one of the two must be a quadratic residue.</ref>.
 
-The third option is slower at signing time but a bit faster to verify, as the quadratic residue of the Y coordinate can be computed directly for points represented in
+In the case of ''R'' the third option is slower at signing time but a bit faster to verify, as the quadratic residue of the Y coordinate can be computed directly for points represented in
 [https://en.wikibooks.org/wiki/Cryptography/Prime_Curve/Jacobian_Coordinates Jacobian coordinates] (a common optimization to avoid modular inverses
 for elliptic curve operations). The two other options require a possibly
 expensive conversion to affine coordinates first. This would even be the case if the sign or oddness were explicitly coded (option 2 in the previous design choice). We therefore choose option 3.
 
-'''Final scheme''' As a result, our final scheme ends up using signatures ''(r,s)'' where ''r'' is the X coordinate of a point ''R'' on the curve whose Y coordinate is a quadratic residue, and which satisfies ''sG = R + H(r || P || m)P''.
+For ''P'' the speed of signing and verification is not significantly different between any of the three options because affine coordinates of the point have to computed anyway. We therefore choose the same option as for ''R''. The signing algorithm ensures that the signature is valid under the correct public key by negating the secret key if necessary.
+
+It is important to not mix up the 32 byte public key format and other existing public key formats. Concretely, a verifier should only accept 32 byte public keys and not, for example, convert a 33 byte public key by throwing away the first byte. Otherwise, two public keys would be valid for a single signature which can result in subtle malleability issues.
+
+Implicit Y coordinates are not a reduction in security when expressed as the number of elliptic curve operations an attacker is expected to perform to compute the secret key. An attacker can normalize any given public key to a point whose Y coordinate is a quadratic residue by negating the point if necessary. This is just a subtraction of field elements and not an elliptic curve operation.
+
+'''Final scheme''' As a result, our final scheme ends up using public keys ''p'' where ''p'' is the X coordinate of a point ''P'' on the curve whose Y coordinate is a quadratic residue and signatures ''(r,s)'' where ''r'' is the X coordinate of a point ''R'' whose Y coordinate is a quadratic residue. The signature satisfies ''sG = R + H(r || p || m)P''.
 
 === Specification ===
 
@@ -94,26 +106,27 @@ The following convention is used, with constants as defined for secp256k1:
 ** ''||'' refers to byte array concatenation.
 ** The function ''x[i:j]'', where ''x'' is a byte array, returns a ''(j - i)''-byte array with a copy of the ''i''-th byte (inclusive) to the ''j''-th byte (exclusive) of ''x''.
 ** The function ''bytes(x)'', where ''x'' is an integer, returns the 32-byte encoding of ''x'', most significant byte first.
-** The function ''bytes(P)'', where ''P'' is a point, returns ''bytes(0x02 + (y(P) & 1)) || bytes(x(P))''<ref>This matches the ''compressed'' encoding for elliptic curve points used in Bitcoin already, following section 2.3.3 of the [http://www.secg.org/sec1-v2.pdf SEC 1] standard.</ref>.
+** The function ''bytes(P)'', where ''P'' is a point, returns ''bytes(x(P))'.
 ** The function ''int(x)'', where ''x'' is a 32-byte array, returns the 256-bit unsigned integer whose most significant byte encoding is ''x''.
 ** The function ''lift_x(x)'', where ''x'' is an integer in range ''0..p-1'', returns the point ''P'' for which ''x(P) = x'' and ''y(P)'' is a quadratic residue modulo ''p'', or fails if no such point exists<ref>Given an candidate X coordinate ''x'' in the range ''0..p-1'', there exist either exactly two or exactly zero valid Y coordinates. If no valid Y coordinate exists, then ''x'' is not a valid X coordinate either, i.e., no point ''P'' exists for which ''x(P) = x''.  Given a candidate ''x'', the valid Y coordinates are the square roots of ''c = x<sup>3</sup> + 7 mod p'' and they can be computed as ''y = &plusmn;c<sup>(p+1)/4</sup> mod p'' (see [https://en.wikipedia.org/wiki/Quadratic_residue#Prime_or_prime_power_modulus Quadratic residue]) if they exist, which can be checked by squaring and comparing with ''c''. Due to [https://en.wikipedia.org/wiki/Euler%27s_criterion Euler's criterion] it then holds that ''c<sup>(p-1)/2</sup> = 1 mod p''. The same criterion applied to ''y'' results in ''y<sup>(p-1)/2</sup> mod p = &plusmn;c<sup>((p+1)/4)((p-1)/2)</sup> mod p = &plusmn;1 mod p''. Therefore ''y = +c<sup>(p+1)/4</sup> mod p'' is a quadratic residue and ''-y mod p'' is not.</ref>. The function ''lift_x(x)'' is equivalent to the following pseudocode:
 *** Let ''y = c<sup>(p+1)/4</sup> mod p''.
 *** Fail if ''c &ne; y<sup>2</sup> mod p''.
 *** Return ''(r, y)''.
-** The function ''point(x)'', where ''x'' is a 33-byte array, returns the point ''P'' for which ''x(P) = int(x[1:33])'' and ''y(P) & 1 = int(x[0:1]) - 0x02)'', or fails if no such point exists. The function  ''point(x)'' is equivalent to the following pseudocode:
-*** Fail if (''x[0:1] ≠ 0x02'' and ''x[0:1] ≠ 0x03'').
-*** Set flag ''odd'' if ''x[0:1] = 0x03''.
-*** Let ''(r, y) = lift_x(x)''; fail if ''lift_x(x)'' fails.
-*** If (flag ''odd'' is set and ''y'' is an even integer) or (flag ''odd'' is not set and ''y'' is an odd integer):
-**** Let ''y = p - y''.
-*** Return ''(r, y)''.
+** The function ''point(x)'', where ''x'' is a 32-byte array, returns the point ''P = lift_x(int(x))''.
 ** The function ''hash(x)'', where ''x'' is a byte array, returns the 32-byte SHA256 hash of ''x''.
 ** The function ''jacobi(x)'', where ''x'' is an integer, returns the [https://en.wikipedia.org/wiki/Jacobi_symbol Jacobi symbol] of ''x / p''. It is equal to ''x<sup>(p-1)/2</sup> mod p'' ([https://en.wikipedia.org/wiki/Euler%27s_criterion Euler's criterion])<ref>For points ''P'' on the secp256k1 curve it holds that ''jacobi(y(P)) &ne; 0''.</ref>.
 
+=== Public Key Generation ===
+
+Input:
+* The secret key ''d'': an integer in the range ''1..n-1'' chosen uniformly at random.
+
+The public key corresponding to secret key ''d'' is ''bytes(dG)''.
+
 ==== Verification ====
 
 Input:
-* The public key ''pk'': a 33-byte array
+* The public key ''pk'': a 32-byte array
 * The message ''m'': a 32-byte array
 * A signature ''sig'': a 64-byte array
 
@@ -130,7 +143,7 @@ The signature is valid if and only if the algorithm below does not fail.
 
 Input:
 * The number ''u'' of signatures
-* The public keys ''pk<sub>1..u</sub>'': ''u'' 33-byte arrays
+* The public keys ''pk<sub>1..u</sub>'': ''u'' 32-byte arrays
 * The messages ''m<sub>1..u</sub>'': ''u'' 32-byte arrays
 * The signatures ''sig<sub>1..u</sub>'': ''u'' 64-byte arrays
 
@@ -147,16 +160,18 @@ All provided signatures are valid with overwhelming probability if and only if t
 ==== Signing ====
 
 Input:
-* The secret key ''d'': an integer in the range ''1..n-1''.
+* The secret key ''d' '': an integer in the range ''1..n-1'' chosen uniformly at random.
 * The message ''m'': a 32-byte array
 
-To sign ''m'' for public key ''dG'':
+To sign ''m'' for public key ''bytes(dG)'':
+* Let ''P = dG''
+* Let ''d = d' '' if ''jacobi(y(P)) = 1'', otherwise let ''d = n - d' ''.
 * Let ''k' = int(hash(bytes(d) || m)) mod n''<ref>Note that in general, taking the output of a hash function modulo the curve order will produce an unacceptably biased result. However, for the secp256k1 curve, the order is sufficiently close to ''2<sup>256</sup>'' that this bias is not observable (''1 - n / 2<sup>256</sup>'' is around ''1.27 * 2<sup>-128</sup>'').</ref>.
 * Fail if ''k' = 0''.
 * Let ''R = k'G''.
 * Let ''k = k' '' if ''jacobi(y(R)) = 1'', otherwise let ''k = n - k' ''.
-* Let ''e = int(hash(bytes(x(R)) || bytes(dG) || m)) mod n''.
-* The signature is ''bytes(x(R)) || bytes((k + ed) mod n)''.
+* Let ''e = int(hash(bytes(R) || bytes(P) || m)) mod n''.
+* The signature is ''bytes(R) || bytes((k + ed) mod n)''.
 
 '''Above deterministic derivation of ''R'' is designed specifically for this signing algorithm and may not be secure when used in other signature schemes.'''
 For example, using the same derivation in the MuSig multi-signature scheme leaks the secret key (see the [https://eprint.iacr.org/2018/068 MuSig paper] for details).