Port the experimental Iceberg module from the benchmark-iceberg tree
(github.com/furszy/benchmark-iceberg, sources/secp256k1-kmp/native/
secp256k1) into this repo.
Iceberg is a threshold scheme that lets a group of parties stand in
for a single MuSig2 (BIP 327) participant: the group produces one
ordinary MuSig2 public nonce and one ordinary MuSig2 partial
signature, so cosigners cannot tell a group is involved and need no
changes. Nonces are derived from a caller-chosen per-session label
(sid32) rather than stored, so no signer holds a secret nonce between
rounds; labels are public but must never be reused. A quorum of 2t-1
members (of whom up to t-1 may be corrupt) is needed in each round,
so the threshold is at most half the group rounded up; combined with
the scheme's other constraints the smallest usable group is 2-of-4.
See doc/iceberg.md and the module header for the full usage notes.
Module layout (src/modules/iceberg/, layered bottom-up, each layer
may only use the ones above it -- that ordering is also the
constant-time story):
- scalar_poly.{h,_impl.h}: secret-carrying polynomial arithmetic,
keeping secrets away from inversions (documented in the header).
- rss.{h,_impl.h}: replicated secret sharing evaluation.
- vpss.{h,_impl.h}: verifiable public shares; variable-time by
design, sees only participant indices and published points.
- keygen_impl.h: distributed key generation producing one share per
member.
- session_impl.h: nonce_gen/nonce_agg and partial_sign/
partial_sig_agg producing plain MuSig2 objects.
- tests_impl.h: 28 tests including the shipped vectors.h vector
suite and dealer known-answer tests.
- bench_impl.h: benchmark definitions (wired in a follow-up commit).
Public headers: include/secp256k1_iceberg.h (installed) and
include/secp256k1_iceberg_dealer.h (in-tree only: a trusted dealer is
not part of the shipped API, but tests, benchmarks and the example
need to deal shares).
Content adaptations relative to the source tree (the only changes to
the ported code): three secp256k1_musig_nonce_process call sites in
tests_impl.h gained a NULL adaptor argument, because this repo's
musig is the zkp variant whose public nonce_process takes an optional
adaptor point. All musig internals the module uses (ge_parse_ext,
ge_serialize_ext, keyaggcoef, aggnonce_load, pubnonce_save,
partial_sig_save, nonce_process_internal) are identical in both
trees, as are all core headers the module touches; nothing else
needed adaptation.
Build wiring mirrors the chilldkg module:
- configure.ac: --enable-module-iceberg (default no, experimental
gate), hard dependency on the musig module with a configure error
if musig is explicitly disabled (musig itself pulls in schnorrsig),
AM_CONDITIONAL(ENABLE_MODULE_ICEBERG), summary line.
- Makefile.am: include src/modules/iceberg/Makefile.am.include under
the conditional.
- src/secp256k1.c: guarded include of modules/iceberg/main_impl.h
after the chilldkg block (musig is included earlier, so its
internals are in scope).
- src/tests.c: module test registration via MAKE_TEST_MODULE(iceberg).
- CMakeLists.txt / src/CMakeLists.txt: SECP256K1_ENABLE_MODULE_ICEBERG
option (OFF) with a dependency check on SECP256K1_ENABLE_MODULE_MUSIG
(placed before the musig block so the force-enable takes effect),
ENABLE_MODULE_ICEBERG=1 compile definition, public header export,
summary line.
Verified: ./configure --enable-experimental --enable-module-iceberg
&& make check passes; ./tests --target=iceberg runs the full module
suite (28/28); CMake build + ctest pass; the musig dependency error
fires correctly in both build systems.
Add an empty, experimental `chilldkg` module as the foundation for a
ChillDKG implementation (distributed key generation for FROST) per the
bip-frost-dkg BIP draft (v0.3.0-dev):
https://github.com/BlockstreamResearch/bip-frost-dkg
The module lives in src/modules/chilldkg/ (separate from the frost
module, per the implementation plan in .idea/docs/
chilldkg-implementation-plan.md: FROST signing (BIP 445) and ChillDKG
are separate BIPs with separate reference repos, test vectors and
review cycles; the dependency between them is one-way bytes).
New files:
- include/secp256k1_chilldkg.h: public header skeleton with the same
"EXTREMELY DANGEROUS / work in progress" warning style as
secp256k1_frost.h, plus a note that the BIP is a draft and tagged
hashes/wire formats may change. No API yet (Phase 3+).
- src/modules/chilldkg/main_impl.h: implementation skeleton including
the public header.
- src/modules/chilldkg/tests_impl.h: trivial scaffolding unit test
(chilldkg_scaffolding_test) registered via the tests_chilldkg[]
CASE1 array used by this repo's unit-test framework.
- src/modules/chilldkg/Makefile.am.include: autotools file list,
mirroring the frost module's.
- src/modules/chilldkg/chilldkg.md: module doc stub (purpose, draft
status, dependency on the schnorrsig and ecdh modules).
Build wiring (mirrors the frost module exactly):
- configure.ac: --enable-module-chilldkg (default no, experimental
gate), dependency errors when schnorrsig or ecdh are explicitly
disabled, AM_CONDITIONAL(ENABLE_MODULE_CHILLDKG), summary line.
- Makefile.am: include src/modules/chilldkg/Makefile.am.include under
ENABLE_MODULE_CHILLDKG.
- src/secp256k1.c: guarded include of modules/chilldkg/main_impl.h
after the frost module.
- src/tests.c: guarded include of tests_impl.h and
MAKE_TEST_MODULE(chilldkg) registration.
- CMakeLists.txt: SECP256K1_ENABLE_MODULE_CHILLDKG option (OFF) +
summary line.
- src/CMakeLists.txt: dependency checks on
SECP256K1_ENABLE_MODULE_SCHNORRSIG and SECP256K1_ENABLE_MODULE_ECDH,
ENABLE_MODULE_CHILLDKG=1 compile definition, public header export.
Verified:
- ./autogen.sh && ./configure --enable-experimental
--enable-module-chilldkg --enable-module-schnorrsig
--enable-module-ecdh && make check: PASS 3/3 (tests, noverify_tests,
exhaustive_tests).
- configure fails with a clear error when schnorrsig or ecdh are
disabled, or when experimental is not enabled.
- CMake build with SECP256K1_ENABLE_MODULE_CHILLDKG=ON: ctest 345/345
passed; dependency errors fire correctly when schnorrsig/ecdh OFF.
CI / x86_64: Linux (Debian stable) (clang, map[env_vars:map[CFLAGS:-O1 ECDH:yes ELLSWIFT:yes EXTRAKEYS:yes MUSIG:yes RECOVERY:yes SCHNORRSIG:yes]]) (push) Has been cancelled
CI / x86_64: Linux (Debian stable) (clang, map[env_vars:map[ELLSWIFT:yes EXTRAKEYS:yes MUSIG:yes RECOVERY:yes SCHNORRSIG:yes WIDEMUL:int128]]) (push) Has been cancelled
CI / x86_64: Linux (Debian stable) (clang-snapshot, map[env_vars:map[BENCH:no BUILD:distcheck CTIMETESTS:no WITH_VALGRIND:no]]) (push) Has been cancelled
CI / x86_64: Linux (Debian stable) (clang-snapshot, map[env_vars:map[CFLAGS:-O1 ECDH:yes ELLSWIFT:yes EXTRAKEYS:yes MUSIG:yes RECOVERY:yes SCHNORRSIG:yes]]) (push) Has been cancelled
CI / x86_64: Linux (Debian stable) (clang-snapshot, map[env_vars:map[ELLSWIFT:yes EXTRAKEYS:yes MUSIG:yes RECOVERY:yes SCHNORRSIG:yes WIDEMUL:int128]]) (push) Has been cancelled
CI / x86_64: Linux (Debian stable) (gcc, map[env_vars:map[CFLAGS:-O1 ECDH:yes ELLSWIFT:yes EXTRAKEYS:yes MUSIG:yes RECOVERY:yes SCHNORRSIG:yes]]) (push) Has been cancelled
CI / x86_64: Linux (Debian stable) (gcc, map[env_vars:map[ELLSWIFT:yes EXTRAKEYS:yes MUSIG:yes RECOVERY:yes SCHNORRSIG:yes WIDEMUL:int128]]) (push) Has been cancelled
CI / x86_64: Linux (Debian stable) (gcc-snapshot, map[env_vars:map[BENCH:no BUILD:distcheck CTIMETESTS:no WITH_VALGRIND:no]]) (push) Has been cancelled
CI / x86_64: Linux (Debian stable) (gcc-snapshot, map[env_vars:map[CFLAGS:-O1 ECDH:yes ELLSWIFT:yes EXTRAKEYS:yes MUSIG:yes RECOVERY:yes SCHNORRSIG:yes]]) (push) Has been cancelled
CI / x86_64: Linux (Debian stable) (gcc-snapshot, map[env_vars:map[ELLSWIFT:yes EXTRAKEYS:yes MUSIG:yes RECOVERY:yes SCHNORRSIG:yes WIDEMUL:int128]]) (push) Has been cancelled
CI / i686: Linux (Debian stable) (clang --target=i686-pc-linux-gnu -isystem /usr/i686-linux-gnu/include, map[env_vars:map[]]) (push) Has been cancelled
CI / MSan (clang, map[env_vars:map[CFLAGS:-fsanitize=memory -fsanitize-recover=memory -fsanitize-memory-param-retval -g CTIMETESTS:no]]) (push) Has been cancelled
CI / MSan (clang, map[env_vars:map[CFLAGS:-fsanitize=memory -fsanitize-recover=memory -g -O3 CTIMETESTS:yes ECMULTGENKB:2 ECMULTWINDOW:2]]) (push) Has been cancelled
CI / MSan (clang-snapshot, map[env_vars:map[CFLAGS:-fsanitize=memory -fsanitize-recover=memory -fsanitize-memory-param-retval -g CTIMETESTS:no]]) (push) Has been cancelled
CI / MSan (clang-snapshot, map[env_vars:map[CFLAGS:-fsanitize=memory -fsanitize-recover=memory -g -O3 CTIMETESTS:yes ECMULTGENKB:2 ECMULTWINDOW:2]]) (push) Has been cancelled
`secp256k1_fe_equal` negates `a` before adding `b`.
That gives the temporary value magnitude 2, and the following field addition requires the input magnitudes to sum to at most 32.
So the largest `b` magnitude the implementation can accept is 30, not 31.
Lower the documented and checked bound for `b` to 30.
Adjust the focused test to use random field elements with randomized magnitudes within the accepted `a <= 1` and `b <= 30` bounds.
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
Co-authored-by: Tim Ruffing <me@real-or-random.org>
Passing a non-malloc pointer to free() would be UB. In this case, the
free() line is never actually reached (and GCC 17 fails to prove this)
in a correct implementation of secp256k1_scratch_space_destroy(), but
the test shouldn't rely on the correctness of the tested function.
If the generator point multiplication result in Jacobian coordinates is
immediately converted to affine coordinates after and is not needed for
anything else, we can deduplicate by using the helper introduced in #1861.
Note that in a very strict sense this is not a refactor, as the Jacobian
object is now cleared out which was not done on master, but for the logic
in the tests this shouldn't matter at all.
Now that we have a function `_ecmult_gen_ge`, it makes sense to rename
the existing function `_ecmult_gen` to `_ecmult_gen_gej` for
consistency, to signal that the result is a Jacobian group element.
This diff was created by applying
```
$ sed -i s/secp256k1_ecmult_gen\(/secp256k1_ecmult_gen_gej\(/g $(git ls-files)
```
3a403639dc eckey: Call ecmult with NULL instead of zero scalar (Tim Ruffing)
7e68c0c88b ecmult: Document and test ng=NULL in ecmult (Tim Ruffing)
Pull request description:
ACKs for top commit:
theStack:
re-ACK 3a403639dc
Tree-SHA512: 954928d4dfa120845c6e899c1a69ad0408072809551d42735eac491b8bc41249eb25d7c57cfa4f44763167620b5cb78639b5c396c0a342c47b0afc48a088c755
4d92a083bc sha256: speed up writes using multi-block compression (furszy)
0753f8b909 Add API to override SHA256 compression at runtime (furszy)
fdb6a91a5e Introduce hash context to support pluggable SHA256 compression (furszy)
Pull request description:
Tackling the long-standing request #702.
Right now we ship our own SHA256 implementation, a standard baseline version that does not take advantage of any hardware-optimized instruction, and it cannot be accessed by the embedding application - it is for internal usage only.
This means embedding applications often have to implement or include a different version for their use cases, wasting space on constrained environments, and in performance-sensitive setups it forces them to use a slower path than what the platform provides. Many projects already rely on tuned SHA-NI / ARMv8 / or other hardware-optimized code, so always using the baseline implementation we ship within the library is not ideal.
These changes allow users to supply their own SHA256 compression function at runtime, while preserving the existing default behavior for everyone else. This is primarily intended for environments where the available SHA256 implementation is detected dynamically and recompiling the library with a different implementation is not feasible (equivalent build-time functionality will come in a follow-up PR).
It introduces a new API:
```C89
secp256k1_context_set_sha256_transform_callback(ctx, fn_transform)
```
This function installs the optimized SHA256 compression into the `secp256k1_context`, which is then used by all internal computations. Important: The provided function is verified to be output-equivalent to the original one.
As a quick example, using this functionality in Bitcoin-Core will be very straightforward: f68bef06d9
ACKs for top commit:
real-or-random:
ACK 4d92a083bc
w0xlt:
ACK 4d92a083bc
theStack:
ACK 4d92a083bc
Tree-SHA512: 058e2e82071f1ca77254b684458292c621e60d65bbcc5500574429717e7db75bc9f3221129fafd11eb5d33e666a5efec5e9844460d3b194ef3b6b16f2df28fb9
921b9711ea util: introduce and use `ARRAY_SIZE` macro (Sebastian Falbesoner)
Pull request description:
This PR is another tiny improvement found while working on #1765, with the goal to avoid code repetition.
The `ARRAY_SIZE` macro definition is pretty wide-spread in C projects and e.g. matches the one [used in the Linux Kernel](9702969978/include/linux/array_size.h (L11)) (without the additional check to reject pointers, as we would need GNU C for that, see e.g. https://stackoverflow.com/a/19455169; not sure if a useful counterpart exists that only relies on C89). Replacement instances were identified via `$ git grep sizeof.*/.*sizeof`.
ACKs for top commit:
w0xlt:
ACK 921b9711ea
real-or-random:
utACK 921b9711ea
Tree-SHA512: 44b6bf0132cf00fade526a3fc04e03dc896d04874123614c032206b61f97c81f94d139b6cc0c108eceaa699251580c19420d230b3150607303ca2cb7ab9a0bcb
This introduces `secp256k1_context_set_sha256_compression()`,
which allows users to provide their own SHA256 block-compression
function at runtime.
This is useful in setups where the fastest implementation can only
be determined dynamically based on the available CPU features, and
rebuilding the library is not possible.
The callback is installed on the `secp256k1_context` and is then used
by all operations that compute SHA256 hashes. As part of the setup,
the library performs sanity checks to ensure that the supplied
function is equivalent to the default transform.
Passing NULL to the callback setter restores the built-in
implementation.
This is purely a mechanical change with no behavior change.
It introduces a secp256k1_hash_ctx struct inside secp256k1_context
and propagates it to all SHA256-related operations.
This sets up the ability to provide a hardware-optimized SHA256
compression function at runtime in a follow-up commit.
f48b1bfa5d hash: add midstate initializer and use it for tagged hashes (w0xlt)
Pull request description:
Each tagged hash midstate function (e.g., `secp256k1_schnorrsig_sha256_tagged`) calls `secp256k1_sha256_initialize` before immediately overwriting every field it sets: `s[0]` through `s[7]` and `bytes`. The `buf[64]` member does not need initialization either, because `bytes` is set to 64, which means the buffer position (`bytes & 0x3F`) (`= bytes % 64`) is 0, so buf is always written before being read.
Remove the 11 redundant `secp256k1_sha256_initialize` calls across the `schnorrsig`, `ellswift`, and `musig` modules.
ACKs for top commit:
real-or-random:
utACK f48b1bfa5d
theStack:
Code-review ACK f48b1bfa5d
Tree-SHA512: 769beb96f3921cc3c180ed0d17484ffa0dc78041c889a8e56603679d8eaca5fe13e63759ada78f83d8e0ff7aae392e6bcbc1a9fe8b959105ea4a3d8ef51abf15
Introduce secp256k1_sha256_initialize_midstate() in the hash layer and use it at all tagged-hash midstate call sites across schnorrsig, musig, and ellswift.
Document the byte-counter contract at the declaration site in hash.h and add run_sha256_initialize_midstate_tests() to directly verify helper behavior against initialize_tagged.
Also switch the helper to take const uint32_t state[8] to reduce argument-order risk at call sites.
f47bbc07f0 test: add unit tests for secp256k1_scalar_check_overflow (Rohit Yadav)
Pull request description:
This Pull Request improves the tests for `secp256k1_scalar_check_overflow` as requested in #1812.
### Changes:
- Removed the redundant "all ones" check from `run_scalar_tests`.
- Added a new dedicated test function `test_scalar_check_overflow`.
- Added static checks for edge cases: `0`, `N-1`, `N`, `N+1`, and `MAX`.
- Added random input tests that verify `check_overflow` against a manual byte comparison.
Fixes#1812.
ACKs for top commit:
theStack:
re-ACK f47bbc07f0
real-or-random:
utACK f47bbc07f0
Tree-SHA512: dad3aa31ecf3f296843c907ac3d9aa5a9b9cb839b36aa3b59e49c853c60c58291412e70dff37dc15f8e14023a8f1e1aba87395065607612d5f6cfa92e14e73b5
It seems that there is no good reason to do this and it's even
considered bad practice, see e.g. https://stackoverflow.com/a/605858
This commit touches mostly test code, the only two functions used
in production are `secp256k1_context_{create,clone}`.
Instances were found manually via `$ git grep "malloc("`
2f73e5281d group: Avoid using infinity field directly in other modules (Tim Ruffing)
Pull request description:
Minor refactoring to make the abstraction cleaner
ACKs for top commit:
hebasto:
ACK 2f73e5281d, I have reviewed the code and it looks OK.
theStack:
ACK 2f73e5281d
Tree-SHA512: eae5ad1ce81f491adb48ab1cbf04211f8d43e41255abcacc958fa3dcb1de5021707d56ed1b009a6f3f6c45cd8f20c1f2677891690a3c0a467fc7e064af2512a8
2f4546ce56 test: add --log option to display tests execution (furszy)
95b9953ea4 test: Add option to display all available tests (furszy)
953f7b0088 test: support running specific tests/modules targets (furszy)
0302c1a3d7 test: add --help for command-line options (furszy)
9ec3bfe22d test: adapt modules to the new test infrastructure (furszy)
48789dafc2 test: introduce (mini) unit test framework (furszy)
9cce703863 refactor: move 'gettime_i64()' to tests_common.h (furszy)
Pull request description:
Early Note:
Don’t be scared by the PR’s line changes count — most of it’s just doc or part of the test framework API.
Context:
Currently, all tests run single-threaded sequentially and the library lacks the ability to specify which test (or group of tests) you would like to run. This is not only inconvenient as more tests are added but also time consuming during development and affects downstream projects that may want to parallelize the workload (such as Bitcoin-Core CI).
PR Goal:
Introduce a lightweight, extensible C89 unit test framework with no dynamic memory allocations, providing a structured way to register, execute, and report tests. The framework supports named command-line arguments in `-key=value` form, parallel test execution across multiple worker processes, granular test selection (selecting tests either by name or by module name), and time accumulation reports.
The introduced framework supports:
* `-help` or `-h`: display list of available commands along with their descriptions.
* `-jobs=<num>`: distribute tests across multiple worker processes (default: sequential if 0).
* `-target=<name>` or `-t=<name>`: run only specific tests by name; can be repeated to select multiple tests.
* `-target=<module name>`, `-t=<module>` Run all tests within a specific module (can be provided multiple times)
* `-seed=<hex>`: set a specific RNG seed (defaults to random if unspecified).
* `-iterations=<n>`: specify the number of iterations.
* `-list_tests`: display list of available tests and modules you can run.
* `-log=<0|1>`: enable or disable test execution logging (default: 0 = disabled).
Beyond these features, the idea is to also make future developments smoother, as adding new tests require only a single entry in the central test registry, and new command-line options can be introduced easily by extending the framework’s `parse_arg()` function.
Compatibility Note:
The framework continues accepting the two positional arguments previously supported (iterations and seed), ensuring existing workflows remain intact.
Testing Notes:
Have fun. You can quickly try it through `./tests -j=<workers_num>` for parallel execution or `./tests -t=<test_name>` to run a specific test (call `./tests -print_tests` to display all available tests and modules).
Extra Note:
I haven't checked the exhaustive tests file so far, but I will soon. For now, this only runs all tests declared in the `tests` binary.
Testing Results: (Current master branch vs PR in seconds)
* Raspberry Pi 5: master \~100 s → PR \~38 s (5 jobs)
* MacBook Pro M1: master \~30 s → PR \~10 s (6 jobs)
ACKs for top commit:
theStack:
re-ACK 2f4546ce56
real-or-random:
ACK 2f4546ce56
hebasto:
ACK 2f4546ce56.
Tree-SHA512: 85ca2cbb620b84b35b353d5d4cf093c388fc3851ca405eeb0e458f8fa72b60534bccd357c7edabf8fc9aa93d9ad0a6fbac3dd5c4d5f9dfdf4d8701a9834755b9
Lightweight unit testing framework, providing a structured way to define,
execute, and report tests. It includes a central test registry, a flexible
command-line argument parser of the form "--key=value" / "-k=value" /
"-key=value" (facilitating future framework extensions), ability to run
tests in parallel and accumulated test time logging reports.
So far the supported command-line args are:
- "--jobs=<num>" or "-j=<num>" to specify the number of parallel workers.
- "--seed=<hex>" to specify the RNG seed (random if not set).
- "--iterations=<num>" or "-i=<num>" to specify the number of iterations.
Compatibility Note:
To stay compatible with previous versions, the framework also supports
the two original positional arguments: the iterations count and the
RNG seed (in that order).
