frost: document deterministic_sign's nonce derivation domain

BIP 445's det_nonce_hash commits to the secret share, my_id, u, the
sorted ids, the aggothernonce, the x-only tweaked threshold public key
and the message. It does not commit to the pubshares, to the untweaked
threshold public key, or to the accumulated tweaks. Because Q and -Q
share an x-coordinate, two tweak caches can agree on everything the
derivation hashes and still disagree on the sign g*gacc that multiplies
the secret share -- a cache initialized from thresh_pk and one
initialized from its negation being the smallest example.

Two calls differing only in that emit the same pubnonce and partial
signatures s = k + e*lambda*d and s' = k - e*lambda*d over the identical
k = k1 + b*k2, so subtracting them yields the secret share. Demonstrated
on a sole signer (u = 1, ids = {0}, pubshares = NULL) with thresh_sk =
0x11.. and msg = 0x42..:

  tweaked pk (cache A)  4f355bdc...075871aa
  tweaked pk (cache B)  4f355bdc...075871aa   same x-only key
  pubnonce A == pubnonce B                    nonce reused
  sA - sB               0d7d9c4e...aa748ffa
  -2*e*d                0d7d9c4e...aa748ffa   d recovered

Nothing inside a single call can catch this. The self-verification that
Sign performs passes in both cases, because each partial signature is
individually valid under the cache it was produced with;
validate_session_params likewise only ties the pubshares to the cache's
own Q0, which both caches satisfy by construction. Note also that
pubshares is optional, so there need not be a second value to disagree
with.

No code change: this is the specified derivation, and committing to Q0
or to gacc here would diverge from BIP 445 and invalidate the
det_sign test vectors. The obligation is the caller's, so state it where
the caller will meet it -- in the function's own documentation and
alongside the existing secnonce and session_secrand32 rules in frost.md.
The rule is that the tweak cache and the pubshares are fixed key
material settled at key generation, never per-session parameters taken
from a coordinator or a peer; under that discipline a repeated call is
byte-identical and harmless, which is the point of a deterministic
nonce.

Worth raising against the BIP: the spec could close this by hashing the
untweaked threshold public key, at the cost of new test vectors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

include/secp256k1_frost.h

@@ -520,6 +520,25 @@ SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_frost_sign(
  *  through NonceAgg as a pubnonce contribution, and a pubnonce's components
  *  are never the point at infinity); if it does, this function fails.
  *
+ *  WARNING: the derivation above is the whole of what the nonce depends on. It
+ *  does NOT commit to the pubshares, to the untweaked threshold public key, or
+ *  to which tweaks the cache accumulated -- only to the x-only encoding of the
+ *  _tweaked_ threshold public key (this is BIP 445's det_nonce_hash, not a
+ *  deviation). Two tweak caches can therefore agree on that x-only key and
+ *  still disagree on the sign g*gacc that multiplies the secret share, because
+ *  Q and -Q have the same x-coordinate: a cache initialized from the threshold
+ *  public key and one initialized from its negation are the simplest example.
+ *  Two calls that differ only in that way emit the SAME pubnonce and two
+ *  partial signatures that differ only in the sign of the secret-share term,
+ *  which is two equations in the nonce and the secret share -- the secret
+ *  share falls out of the pair.
+ *
+ *  The caller must therefore treat the tweak cache and the pubshares as fixed
+ *  key material belonging to the group, established once at key generation,
+ *  and never as per-session parameters accepted from a coordinator or any
+ *  other peer. Given that, repeating a call reproduces a byte-identical result
+ *  and is harmless, which is the point of a deterministic nonce.
+ *
  *  Returns: 0 if the arguments are invalid or signing fails, 1 otherwise
  *  Args:             ctx: pointer to a context object
  *  Out:      partial_sig: pointer to a partial_sig object