Files
secp256k1-zkp/ci/ci.sh

166 lines
4.8 KiB
Bash
Raw Normal View History

#!/bin/sh
set -eux
2020-12-22 16:42:08 +01:00
export LC_ALL=C
# Print commit and relevant CI environment to allow reproducing the job outside of CI.
git show --no-patch
print_environment() {
# Turn off -x because it messes up the output
set +x
# There are many ways to print variable names and their content. This one
# does not rely on bash.
for var in WERROR_CFLAGS MAKEFLAGS BUILD \
ECMULTWINDOW ECMULTGENKB ASM WIDEMUL WITH_VALGRIND EXTRAFLAGS \
EXPERIMENTAL ECDH RECOVERY EXTRAKEYS SCHNORRSIG MUSIG SCHNORRSIG_HALFAGG ELLSWIFT \
ECDSA_S2C GENERATOR RANGEPROOF SURJECTIONPROOF WHITELIST ECDSAADAPTOR BPPP \
build: wire the frost_enrollment module into both build systems Second of six commits adding the frost_enrollment module. This one is scaffolding only: the five entry points are stubs that validate their pointer arguments, zero their outputs and return 0. What is being verified here is that the module configures, compiles, links, exports its symbols and registers its test module in both build systems -- so that the next commit changes nothing but arithmetic. Ordering is the one thing in this commit that can go silently wrong, and it goes wrong in opposite directions in the two build systems: - configure.ac executes its `if` blocks in file order, and enable_module_frost defaults to no (configure.ac:243). A block placed after the frost block at :601 that sets enable_module_frost=yes flips the variable too late: AM_CONDITIONAL goes true, so the header is installed and the Makefile fragment is pulled in, but -DENABLE_MODULE_FROST=1 is never appended, so src/secp256k1.c never includes frost's implementation and every secp256k1_frost_* symbol fails to link. The new block therefore goes ahead of both the frost block and prefractal's, which documents the same trap. - src/CMakeLists.txt processes dependents FIRST, so the same block goes above the FROST block there, beside prefractal's. Verified rather than assumed: configuring with ONLY --enable-module-frost-enrollment emits -DENABLE_MODULE_FROST=1 alongside -DENABLE_MODULE_FROST_ENROLLMENT=1, and the CMake summary prints "frost ON" for the same configuration -- the latter is what the PARENT_SCOPE lift buys, since the summary runs after add_subdirectory(src) and would otherwise report a module it is compiling in as OFF. The dependency guard is prefractal's implies-frost idiom, copied verbatim along with its reasoning. frost is default-OFF, so the `test x"$enable_module_frost" = x"no"` / `DEFINED X AND NOT X` guard every other module uses -- which reads as "the user disabled it explicitly" for a default-ON dependency -- is true by default here and cannot tell an explicit --disable-module-frost from the default once both are in the cache. Enabling frost-enrollment simply implies frost, with no error. The one frost-module change in the whole series is in this commit: src/modules/frost/session.h gains a declaration for secp256k1_frost_sort_ids, which is defined at session_impl.h:517 and declared nowhere. The params hash needs it to canonicalize identifier order. Prefractal reaches frost's statics through translation-unit ordering alone; rather than inherit reuse-by-link-order, this declares the function where keygen.h:48 already declares derive_pubshare_at, so the reuse goes through an interface. No behavior change: it is a declaration for an existing static definition in the same TU. CI wiring is two files, and skipping either half fails quietly: - ci/ci.sh gets FROST_ENROLLMENT in the reproduction header's variable list and --enable-module-frost-enrollment="$FROST_ENROLLMENT" after the prefractal line. - .github/workflows/ci.yml gets FROST_ENROLLMENT at every PREFRACTAL site: the global default, 11 inline matrix entries and 10 job-level env blocks. Without the default, ci.sh runs under set -eux with an empty $FROST_ENROLLMENT, passes --enable-module-frost-enrollment="", `test x"" = x"yes"` is false, and the module is off in all of CI while ci.sh visibly has the plumbing. Verified programmatically over the parsed workflow: across the 106 effective job contexts, PREFRACTAL and FROST_ENROLLMENT now agree in every single one (45 set to yes, no mismatches), no context sets FROST_ENROLLMENT without FROST or without EXPERIMENTAL, and no context leaves it undefined. ci.sh passes sh -n. The stub test is not a placeholder that has to be deleted later: every entry point must reject an empty helper set and leave its output zeroed, which is true of the stubs and stays true of the finished implementation, so it doubles as the check that all five symbols are reachable from the test binary. Verification. Autotools: ./autogen.sh, then a frost-enrollment-only configure and a full configure with frost, chilldkg, iceberg, prefractal and frost-enrollment all on -- both build with zero warnings under the project's -Werror-grade flag set, ./tests and ./exhaustive_tests exit 0, and `./tests -l` lists the frost_enrollment module. CMake: configure with -DSECP256K1_EXPERIMENTAL=ON -DSECP256K1_ENABLE_MODULE_FROST_ENROLLMENT=ON builds clean and ctest passes 391 tests. nm shows the five new symbols exported from libsecp256k1.so; tools/symbol-check.py could not be run here because python3-lief is not installed in this environment, but all five carry the required secp256k1_ prefix. make dist succeeds and the tarball carries src/modules/frost_enrollment/frost_enrollment.md alongside the other module documents. One unrelated observation from this build: a stale src/ctime_tests-ctime_tests.o left over from an earlier configure with a different module set will fail to link, because automake does not track CPPFLAGS changes across reconfigures. make clean between configurations with different module sets, not a fault in this change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 03:53:03 +02:00
FROST CHILLDKG ICEBERG PREFRACTAL FROST_ENROLLMENT SECP256K1_TEST_ITERS BENCH SECP256K1_BENCH_ITERS CTIMETESTS SYMBOL_CHECK \
EXAMPLES \
HOST WRAPPER_CMD \
CC CFLAGS CPPFLAGS AR NM \
UBSAN_OPTIONS ASAN_OPTIONS LSAN_OPTIONS
do
eval "isset=\${$var+x}"
if [ -n "$isset" ]; then
eval "val=\${$var}"
# shellcheck disable=SC2154
printf '%s="%s" ' "$var" "$val"
fi
done
echo "$0"
set -x
}
print_environment
2020-12-22 16:42:08 +01:00
env >> test_env.log
# If gcc is requested, assert that it's in fact gcc (and not some symlinked Apple clang).
case "${CC:-undefined}" in
*gcc*)
$CC -v 2>&1 | grep -q "gcc version" || exit 1;
;;
esac
if [ -n "${CC+x}" ]; then
# The MSVC compiler "cl" doesn't understand "-v"
$CC -v || true
fi
if [ "$WITH_VALGRIND" = "yes" ]; then
valgrind --version
fi
if [ -n "$WRAPPER_CMD" ]; then
$WRAPPER_CMD --version
fi
2020-12-22 16:42:08 +01:00
./autogen.sh
./configure \
2020-09-25 20:06:36 -07:00
--enable-experimental="$EXPERIMENTAL" \
--with-test-override-wide-multiply="$WIDEMUL" --with-asm="$ASM" \
2021-12-15 14:10:46 +01:00
--with-ecmult-window="$ECMULTWINDOW" \
--with-ecmult-gen-kb="$ECMULTGENKB" \
--enable-module-ecdh="$ECDH" --enable-module-recovery="$RECOVERY" \
2022-11-04 16:21:25 -04:00
--enable-module-ellswift="$ELLSWIFT" \
--enable-module-extrakeys="$EXTRAKEYS" \
--enable-module-ecdsa-s2c="$ECDSA_S2C" \
2023-02-06 13:53:02 -08:00
--enable-module-bppp="$BPPP" \
--enable-module-rangeproof="$RANGEPROOF" --enable-module-surjectionproof="$SURJECTIONPROOF" --enable-module-whitelist="$WHITELIST" --enable-module-generator="$GENERATOR" \
--enable-module-schnorrsig="$SCHNORRSIG" --enable-module-ecdsa-adaptor="$ECDSAADAPTOR" \
--enable-module-musig="$MUSIG" \
--enable-module-schnorrsig-halfagg="$SCHNORRSIG_HALFAGG" \
2026-08-31 00:05:16 +02:00
--enable-module-frost="$FROST" \
chilldkg: CI wiring, ctime_tests coverage, declassify fixes CI: - ci/ci.sh: new CHILLDKG environment variable, passed to configure as --enable-module-chilldkg (mirroring FROST). - .github/workflows/ci.yml: default CHILLDKG: 'no' and CHILLDKG: 'yes' in every job that enables FROST, except the x86_64 matrix entry that deliberately builds without the ecdh module (chilldkg requires schnorrsig + ecdh; the configure-time dependency error would fire there). YAML validity and per-job dependency presence checked programmatically. ctime_tests: - src/ctime_tests.c: run a full ChillDKG session (n = 2, t = 2) through the public API under the memory checker: hostpubkey_gen, params_hash, participant_step1, coordinator_step1, participant_step2, coordinator_finalize, participant_finalize, participant_recover and recovery_ack_sign. Host secret keys, session randomness, aux randomness and the resulting secret shares are undefined (secret); all protocol messages, the certificate, threshold public key, public shares, recovery data, ack signature and the secret-free state1 objects are defined (public). state2 stays secret (contains the secret share). Constant-time fixes found by running the new block under MemorySanitizer (valgrind unavailable locally; MSan build via clang + CMake). All are missing declassifications of secret-derived but public (or public-outcome) values, following the frost module's secp256k1_declassify pattern with justification comments; no real constant-time bugs were found: - hostpubkey_gen: declassify the computed host public key before serialization (public output). - participant_step1: declassify the zero-randomness check result (only reveals "the RNG returned 32 zero bytes", which aborts the session). - encpedpop participant_step1: declassify the pubnonce point before serialization (public, part of pmsg1). - chilldkg_schnorrsig_sign: declassify the signer public key before normalization/parity branch, and declassify the return value (a failure only reveals a zero derived nonce, negligible probability). - vss_commit: declassify the VSS commitments before serialization (public, part of pmsg1). - vss_verify_secshare: declassify secshare*G before the infinity/eq checks (equals the public pubshare in honest runs; the discrete log is not revealed). - simplpedpop_participant_investigate (proactive audit; not reached by ctime_tests): declassify the secshare-sum comparison result (the public fault code reveals it anyway). Verified: MSan ctime_tests exits 0; autotools make check 10/10 (the local tree is configured without --enable-ctime-tests because neither valgrind nor an MSan-instrumented gcc build is available; CI runs ctime_tests under valgrind as before); CMake ctest 428/428; ./tests --target=chilldkg and ./chilldkg_example pass.
2026-08-31 10:25:14 +02:00
--enable-module-chilldkg="$CHILLDKG" \
--enable-module-iceberg="$ICEBERG" \
build: wire prefractal into CI and the README feature list The prefractal module landed on main in parallel with the branches that established the current module conventions, so it never picked up the integration points every other experimental module has. 01379624 covered the CMake experimental gate and EXTRA_DIST; these two are what remained. The CI gap is the more consequential of the pair. ci/ci.sh had no PREFRACTAL plumbing at all, so no CI job has ever passed --enable-module-prefractal -- the flag was reachable only by configuring by hand, and the module has been built and tested exclusively outside CI since it landed. - ci/ci.sh: PREFRACTAL joins FROST, CHILLDKG and ICEBERG in the reproduction header's variable list, and configure gains --enable-module-prefractal="$PREFRACTAL" after the iceberg line. Unlike 2473c768 there is no bench-step counterpart: prefractal ships neither a bench binary nor an example program, so the BENCH block needs nothing. - .github/workflows/ci.yml: a PREFRACTAL: 'no' default alongside the other module defaults, then PREFRACTAL: 'yes' in the 21 places that already enable FROST (11 inline matrix entries and 10 job-level env blocks). The default is not cosmetic -- ci.sh runs under set -eux, so an undefined PREFRACTAL would abort every job at the configure step, including the ones that build no modules at all. FROST-enabled jobs are the right target because enabling prefractal implies frost (configure.ac:539, mirrored in src/CMakeLists.txt:90): adding PREFRACTAL to a job that sets FROST: 'no' would silently turn frost on and change what that job covers. Every job that already builds frost also enables musig, schnorrsig, extrakeys and experimental, which is the remainder of prefractal's dependency chain, so no job needed any other variable adjusted. This is a slightly wider set than CHILLDKG and ICEBERG cover. The x86_64-debian matrix entry at line 117 sets FROST: 'yes' without CHILLDKG or ICEBERG, and 2473c768 deliberately left it alone; it gets PREFRACTAL here, on the rule above. The consequence is that no job now builds frost without prefractal. If that standalone-frost combination is worth preserving, that one entry is the place to drop it. README.md gains the doc/prefractal.md link beside the frost, chilldkg and iceberg entries. This is the exact inverse of the bug 42f827a7 fixed: there the documents were linked from README.md but missing from EXTRA_DIST, so the tarball's README pointed at files it did not carry. Here doc/prefractal.md has been in EXTRA_DIST since 01379624 but nothing referenced it, so it shipped unreferenced. All four module documents this fork adds are now both linked and distributed. Verified with ./autogen.sh, ./configure --enable-experimental --enable-module-prefractal and make -j: all exit 0 with no warnings, and the configure summary shows the implication chain resolving, with prefractal = yes pulling in frost = yes and musig = yes. nm confirms run_prefractal_api_test is linked into ./tests, and the suite passes. make dist succeeds and the tarball carries doc/prefractal.md next to doc/iceberg.md, src/modules/frost/frost.md and src/modules/chilldkg/chilldkg.md. ci/ci.sh passes sh -n. Verified programmatically for the workflow, as in 2473c768: the YAML parses, 33 effective job contexts set FROST: 'yes', all 33 of them now also set PREFRACTAL: 'yes', every one has MUSIG, SCHNORRSIG, EXTRAKEYS and EXPERIMENTAL set to 'yes', and no context sets PREFRACTAL without FROST. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 03:28:22 +02:00
--enable-module-prefractal="$PREFRACTAL" \
build: wire the frost_enrollment module into both build systems Second of six commits adding the frost_enrollment module. This one is scaffolding only: the five entry points are stubs that validate their pointer arguments, zero their outputs and return 0. What is being verified here is that the module configures, compiles, links, exports its symbols and registers its test module in both build systems -- so that the next commit changes nothing but arithmetic. Ordering is the one thing in this commit that can go silently wrong, and it goes wrong in opposite directions in the two build systems: - configure.ac executes its `if` blocks in file order, and enable_module_frost defaults to no (configure.ac:243). A block placed after the frost block at :601 that sets enable_module_frost=yes flips the variable too late: AM_CONDITIONAL goes true, so the header is installed and the Makefile fragment is pulled in, but -DENABLE_MODULE_FROST=1 is never appended, so src/secp256k1.c never includes frost's implementation and every secp256k1_frost_* symbol fails to link. The new block therefore goes ahead of both the frost block and prefractal's, which documents the same trap. - src/CMakeLists.txt processes dependents FIRST, so the same block goes above the FROST block there, beside prefractal's. Verified rather than assumed: configuring with ONLY --enable-module-frost-enrollment emits -DENABLE_MODULE_FROST=1 alongside -DENABLE_MODULE_FROST_ENROLLMENT=1, and the CMake summary prints "frost ON" for the same configuration -- the latter is what the PARENT_SCOPE lift buys, since the summary runs after add_subdirectory(src) and would otherwise report a module it is compiling in as OFF. The dependency guard is prefractal's implies-frost idiom, copied verbatim along with its reasoning. frost is default-OFF, so the `test x"$enable_module_frost" = x"no"` / `DEFINED X AND NOT X` guard every other module uses -- which reads as "the user disabled it explicitly" for a default-ON dependency -- is true by default here and cannot tell an explicit --disable-module-frost from the default once both are in the cache. Enabling frost-enrollment simply implies frost, with no error. The one frost-module change in the whole series is in this commit: src/modules/frost/session.h gains a declaration for secp256k1_frost_sort_ids, which is defined at session_impl.h:517 and declared nowhere. The params hash needs it to canonicalize identifier order. Prefractal reaches frost's statics through translation-unit ordering alone; rather than inherit reuse-by-link-order, this declares the function where keygen.h:48 already declares derive_pubshare_at, so the reuse goes through an interface. No behavior change: it is a declaration for an existing static definition in the same TU. CI wiring is two files, and skipping either half fails quietly: - ci/ci.sh gets FROST_ENROLLMENT in the reproduction header's variable list and --enable-module-frost-enrollment="$FROST_ENROLLMENT" after the prefractal line. - .github/workflows/ci.yml gets FROST_ENROLLMENT at every PREFRACTAL site: the global default, 11 inline matrix entries and 10 job-level env blocks. Without the default, ci.sh runs under set -eux with an empty $FROST_ENROLLMENT, passes --enable-module-frost-enrollment="", `test x"" = x"yes"` is false, and the module is off in all of CI while ci.sh visibly has the plumbing. Verified programmatically over the parsed workflow: across the 106 effective job contexts, PREFRACTAL and FROST_ENROLLMENT now agree in every single one (45 set to yes, no mismatches), no context sets FROST_ENROLLMENT without FROST or without EXPERIMENTAL, and no context leaves it undefined. ci.sh passes sh -n. The stub test is not a placeholder that has to be deleted later: every entry point must reject an empty helper set and leave its output zeroed, which is true of the stubs and stays true of the finished implementation, so it doubles as the check that all five symbols are reachable from the test binary. Verification. Autotools: ./autogen.sh, then a frost-enrollment-only configure and a full configure with frost, chilldkg, iceberg, prefractal and frost-enrollment all on -- both build with zero warnings under the project's -Werror-grade flag set, ./tests and ./exhaustive_tests exit 0, and `./tests -l` lists the frost_enrollment module. CMake: configure with -DSECP256K1_EXPERIMENTAL=ON -DSECP256K1_ENABLE_MODULE_FROST_ENROLLMENT=ON builds clean and ctest passes 391 tests. nm shows the five new symbols exported from libsecp256k1.so; tools/symbol-check.py could not be run here because python3-lief is not installed in this environment, but all five carry the required secp256k1_ prefix. make dist succeeds and the tarball carries src/modules/frost_enrollment/frost_enrollment.md alongside the other module documents. One unrelated observation from this build: a stale src/ctime_tests-ctime_tests.o left over from an earlier configure with a different module set will fail to link, because automake does not track CPPFLAGS changes across reconfigures. make clean between configurations with different module sets, not a fault in this change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 03:53:03 +02:00
--enable-module-frost-enrollment="$FROST_ENROLLMENT" \
--enable-examples="$EXAMPLES" \
2022-12-20 12:28:48 -05:00
--enable-ctime-tests="$CTIMETESTS" \
--with-valgrind="$WITH_VALGRIND" \
--host="$HOST" $EXTRAFLAGS
# We have set "-j<n>" in MAKEFLAGS.
build_exit_code=0
make > make.log 2>&1 || build_exit_code=$?
cat make.log
if [ $build_exit_code -ne 0 ]; then
case "${CC:-undefined}" in
*snapshot*)
# Ignore internal compiler errors in gcc-snapshot and clang-snapshot
grep -e "internal compiler error:" -e "PLEASE submit a bug report" make.log
exit $?
;;
*)
exit 1
;;
esac
fi
# Print information about binaries so that we can see that the architecture is correct
2021-04-17 10:57:16 -07:00
file *tests* || true
file bench* || true
file .libs/* || true
2025-03-10 11:22:38 +00:00
if [ "$SYMBOL_CHECK" = "yes" ]
then
python3 --version
case "$HOST" in
*mingw*)
ls -l .libs
2025-07-21 16:35:16 +02:00
python3 ./tools/symbol-check.py .libs/libsecp256k1-*.dll
2025-03-10 11:22:38 +00:00
;;
*)
python3 ./tools/symbol-check.py .libs/libsecp256k1.so
;;
esac
fi
# This tells `make check` to wrap test invocations.
export LOG_COMPILER="$WRAPPER_CMD"
2020-12-22 16:42:08 +01:00
make "$BUILD"
2020-12-22 16:42:08 +01:00
2022-12-06 18:53:51 -05:00
# Using the local `libtool` because on macOS the system's libtool has nothing to do with GNU libtool
EXEC='./libtool --mode=execute'
if [ -n "$WRAPPER_CMD" ]
then
EXEC="$EXEC $WRAPPER_CMD"
fi
if [ "$BENCH" = "yes" ]
then
{
$EXEC ./bench_ecmult
$EXEC ./bench_internal
$EXEC ./bench
2023-02-06 13:53:02 -08:00
if [ "$BPPP" = "yes" ]
2023-01-20 13:41:01 +00:00
then
2023-02-06 13:53:02 -08:00
$EXEC ./bench_bppp
2023-01-20 13:41:01 +00:00
fi
if [ "$ICEBERG" = "yes" ]
then
$EXEC ./bench_iceberg
fi
} >> bench.log 2>&1
fi
2021-11-13 15:52:44 +01:00
2022-12-20 12:28:48 -05:00
if [ "$CTIMETESTS" = "yes" ]
then
2022-12-06 18:53:51 -05:00
if [ "$WITH_VALGRIND" = "yes" ]; then
./libtool --mode=execute valgrind --error-exitcode=42 ./ctime_tests > ctime_tests.log 2>&1
else
$EXEC ./ctime_tests > ctime_tests.log 2>&1
fi
fi
2021-11-13 15:52:44 +01:00
# Rebuild precomputed files (if not cross-compiling).
if [ -z "$HOST" ]
then
make clean-precomp clean-testvectors
make precomp testvectors
2021-11-13 15:52:44 +01:00
fi
# Check that no repo files have been modified by the build.
# (This fails for example if the precomp files need to be updated in the repo.)
git diff --exit-code