3dbfb48946b9d2a98acef23674617510cf1b3386 tests: restore scalar_mul test (Jonas Nick)
Pull request description:
Without this commit, the res[i][1] test vectors are unused. They were introduced to test the correctness of scalar_sqr(x) and scalar_mul(x, x). These tests were deleted as part of removing scalar_sqr in commit
[5437e7bdfbffddf69fdf7b4af7e997c78f5dafbf](5437e7bdfb (diff-c2d5f1f7616875ab71cd41b053cfb428696988ff89642b931a0963d50f34f7e8L2195)).
Discovered in https://github.com/bitcoin-core/secp256k1/discussions/1463 by Coding-Enthusiast (thanks!).
ACKs for top commit:
real-or-random:
utACK 3dbfb48946b9d2a98acef23674617510cf1b3386
Tree-SHA512: 914e08db3efaa1cef546a9730096e740478c422d41fedb2b71ec3a7ea962f81740a05dc7e7c1fb191088f6d38b5690479c7d0864ca8abf2b2e9c4334f03ca605
Without this commit, the res[i][1] test vectors are unused. They were introduced
to test the correctness of scalar_sqr(x) and scalar_mul(x, x). These tests were
deleted as part of removing scalar_sqr in commit
5437e7bdfbffddf69fdf7b4af7e997c78f5dafbf.
74a4d974d5c81fbc437287dffc453028509682ab doc: Add ABI checking with `check-abi.sh` to the Release Process (Hennadii Stepanov)
e7f830e32c61ac4cf6c562b477063ccf35940ba9 Add `tools/check-abi.sh` (Hennadii Stepanov)
Pull request description:
ACKs for top commit:
real-or-random:
ACK 74a4d974d5c81fbc437287dffc453028509682ab it compares the right commits now
jonasnick:
re-Concept ACK 74a4d974d5c81fbc437287dffc453028509682ab
Tree-SHA512: bcca5246837f899d43ced3b0099a8e123f4fd2db7d15684bda22657649521db0c87f76696bfbd93b4dfdec6c4851e99c26c7e37cc5a1a78e9b1a296850a067fe
3928b7c38367947756b7d506f431bfb7bbbac5d0 doc: improve secp256k1_fe_set_b32_mod doc (Coding Enthusiast)
Pull request description:
As discussed in #1453
This only changes the `secp256k1_fe_impl_set_b32_mod` comment since I think `secp256k1_fe_set_b32_limit` doc is clear enough.
ACKs for top commit:
sipa:
ACK 3928b7c38367947756b7d506f431bfb7bbbac5d0
theStack:
ACK 3928b7c38367947756b7d506f431bfb7bbbac5d0
Tree-SHA512: ad62c1b72d6a487473b182e6aadc7765711385add8c6576bf15c2015db82721f19e3d635f7a29316c2ee7e3c73bc55e2cd4f46ec13180be93d6fe8641f47e7d2
e02f313b1f251ccb363ae1ac24016d87c1be9009 Add comment on length checks when parsing ECDSA sigs (Tim Ruffing)
Pull request description:
I claim the check can be removed but I don't want to touch this
stable and well-tested code.
On the way, we fix grammar in another comment.
ACKs for top commit:
sipa:
ACK e02f313b1f251ccb363ae1ac24016d87c1be9009
RandyMcMillan:
ACK e02f313
Tree-SHA512: f82691a8f5db82a1e9683e52ce8e952ebd56b476a2817c5a876ce4638254b7b4ac93175318fb59598ed5532f33433951d75afea03724ef4419c3e1bd12ca8c20
0e5ea6220707d9c96e06efc43bce3d5a3b3a06f2 CONTRIBUTING: add some coding and style conventions (Jonas Nick)
1a432cb98220f29ac47639d30a6dbb3aa679a441 README: update first sentence (Jonas Nick)
0922a047fb2a225fd89802bbd6f2d0919cd50bca docs: move coverage report instructions to CONTRIBUTING (Jonas Nick)
76880e40151ddb18d0cd0549502d5ade95f501d6 Add CONTRIBUTING.md including scope and guidelines for new code (Jonas Nick)
Pull request description:
Following offline discussions, this PR documents the scope of the library and the requirements for adding new modules. I think this fixes most of #997. It also updates the README very slightly.
In addition, I added some coding conventions that I remembered explaining to new contributors in the past year. Even though it's far from exhaustive, I think this is an easy improvement to the CONTRIBUTING.md. Feel free to suggest more conventions.
ACKs for top commit:
sipa:
ACK 0e5ea6220707d9c96e06efc43bce3d5a3b3a06f2
real-or-random:
ACK 0e5ea6220707d9c96e06efc43bce3d5a3b3a06f2
Tree-SHA512: ffdbab22982fd632de92e81bd135f141ac86e24cc0dcfc0e1ae12b0d2a2e4f91377ab2c0cc440cb919889eaed8bfc1447b880fa1430fd771b956f2af0fe3766e
04af0ba162b152073455a5ccbb2c5833ae6d1d57 Replace ge_equals_ge[,j] calls with group.h equality calls (Pieter Wuille)
60525f6c14ad37719c6ea2deee19ec7b3654f061 Add unit tests for group.h equality functions (Pieter Wuille)
a47cd97d51e37c38ecf036d04e48518f6b0063f7 Add group.h ge/gej equality functions (Pieter Wuille)
Pull request description:
This pull requests removes the test-only functions `ge_equals_ge` and `ge_equals_gej`, and replaces them with proper group.h functions `secp256k1_ge_eq_var` and `secp256k1_gej_eq_ge_var` (mimicking the existing `secp256k1_gej_eq_var` function).
This drops some of the arbitrary and undocumented magnitude restristrictions these functions have, makes them properly tested on their own, and makes their semantics cleaner (I'm always left checking whether `ge_equals_ge` does a `CHECK` internally or whether it returns a value...).
ACKs for top commit:
real-or-random:
utACK 04af0ba162b152073455a5ccbb2c5833ae6d1d57
stratospher:
ACK 04af0ba.
Tree-SHA512: 49bc409ffa980144d1305c9389a846af45f0a97bfec19d016929056aa918c6a9f020dbe8549f5318fa8e6a4108621cc3cce60331aa0634f84619a1104d20a62a
bb4672342efce7fae1cfd30e007c6835a25286a7 remove VERIFY_SETUP define (Sebastian Falbesoner)
a3a3e11acdb473f96a8972ed40cd3310057aec23 remove unneeded VERIFY_SETUP uses in ECMULT_CONST_TABLE_GET_GE macro (Sebastian Falbesoner)
a0fb68a2e7db14c6b27f92217bf2307681b6b6ea introduce and use SECP256K1_SCALAR_VERIFY macro (Sebastian Falbesoner)
cf25c86d05bbaacd37f42a190e39eab4863cdaf7 introduce and use SECP256K1_{FE,GE,GEJ}_VERIFY macros (Sebastian Falbesoner)
5d89bc031b25dc0aaba8c7d2eeba88ae92facb09 remove superfluous `#ifdef VERIFY`/`#endif` preprocessor conditions (Sebastian Falbesoner)
c2688f8de9fb9a44dc953d2f8a0e9226d8e19349 redefine VERIFY_CHECK to empty in production (non-VERIFY) mode (Sebastian Falbesoner)
Pull request description:
As suggested in #1381, this PR reworks the policy for VERIFY_CHECK and when to use #ifdef VERIFY, by:
- redefining VERIFY_CHECK to empty in production (non-VERIFY) mode
- removing many then superflous #ifdef VERIFY blocks (if they exclusively contained VERIFY_CHECKs)
- introducing uppercase macros around verify_ functions and using them for better readabiliy
What is _not_ included yet is the proposed renaming from "_check" to "_assert":
> And while we're touching this anyway, we could consider renaming "check" to "assert", which is a more precise term. (In fact, if we redefine VERIFY_CHECK to be empty in production, we have almost reimplemented assert.h...)
This should be easy to achieve with simple search-and-replace (e.g. using sed), but I was hesitant as this would probably case annoying merge conflicts on some of the open PRs. Happy to add this if the rename if desired (#1381 didn't get any feedback about the renaming idea yet).
ACKs for top commit:
stratospher:
ACK bb46723.
real-or-random:
utACK bb4672342efce7fae1cfd30e007c6835a25286a7
Tree-SHA512: 226ca609926dea638aa3bb537d29d4fac8b8302dcd9da35acf767ba9573e5221d2dae04ea26c15d80a50ed70af1ab0dca10642c21df7dbdda432fa237a5ef2cc
This define was seemingly introduced for VERIFY mode code with side
effects (for setup purposes), that should just be executed without any
checks. The same can be achieved by putting it in an `#if VERIFY` block,
so we can remove it.
By providing an uppercase variant of these verification functions, it is
better visible that it is test code and surrounding `#ifdef VERIFY`
blocks can be removed (if there is no other code around that could
remain in production mode), as they don't serve their purpose any more.
At some places intentional blank lines are inserted for grouping and
better readadbility.
Now that the `VERIFY_CHECK` compiles to empty in non-VERIFY mode, blocks
that only consist of these macros don't need surrounding `#ifdef VERIFY`
conditions anymore.
At some places intentional blank lines are inserted for grouping and
better readadbility.
As suggested in issue #1381, this will make things simpler and
improve code readability, as we don't need to force omitting of
evaluations on a case-by-case basis anymore and hence can remove
lots of `#ifdef VERIFY`/`#endif` lines (see next commit). Plus,
VERIFY_CHECK behaves now identical in both non-VERIFY and coverage mode,
making the latter not special anymore and hopefully decreasing
maintenance burden. The idea of "side-effect safety" is given up.
Note that at two places in the ellswift module void-casts of return
values have to be inserted for non-VERIFY builds, in order to avoid
"variable ... set but not used [-Wunused-but-set-variable]"
warnings.
dcdda31f2cda13839a4285d8601118c041b18c13 Tighten secp256k1_fe_mul_inner's VERIFY_BITS checks (Russell O'Connor)
8e2a5fe908faa2ad0b847b3e5c42662614c8fa88 correct assertion for secp256k1_fe_mul_inner (roconnor-blockstream)
Pull request description:
Based on the surrounding asserts, 112 bits before this line, and 61 bits after this line, this assertion should be 113 bits. Notably the commensurate line in secp256k1_fe_sqr_inner is correctly assert to be 113 bits.
ACKs for top commit:
real-or-random:
ACK dcdda31f2cda13839a4285d8601118c041b18c13 tested with asm disabled
Tree-SHA512: c35170e37d9a6d1413dd625032028129ab2eccee7da86697ab9641b68ad78efd7251953d51e7acaefd14888d3fd61877f9f05349c44f6fc0133ce9b3921b0e1a
1ddd76af0a735b7fcbec7f37c0d99a7db9893ac1 bench: add --help option to bench_internal (Sebastian Falbesoner)
Pull request description:
While coming up with commands for running the benchmarks for issue https://github.com/bitcoin-core/secp256k1/issues/726#issuecomment-1824625653, I noticed that in contrast to `bench{_ecmult}`, `bench_internal` doesn't have a help option yet and figured it would be nice to have one. A comparable past PR is https://github.com/bitcoin-core/secp256k1/pull/1008. Benchmark categories appear in the same order as they are executed, the concrete benchmark names in parantheses per category are listed in alphabetical order.
ACKs for top commit:
real-or-random:
utACK 1ddd76af0a735b7fcbec7f37c0d99a7db9893ac1
siv2r:
ACK 1ddd76a, tested the `--help` option locally, and it works as expected.
Tree-SHA512: d117641a5f25a7cbf83881f3acceae99624528a0cbb2405efdbe1a3a2762b4d6b251392e954aaa32f6771069d31143743770fccafe198084c12258dedb0856fc
Widely available versions of GCC and Clang beat our field asm on -O2.
In particular, GCC 10.5.0, which is Bitcoin Core's current compiler
for official x86_64 builds, produces code that is > 20% faster for
fe_mul and > 10% faster for signature verification (see #726).
These are the alternatives to this PR:
We could replace our current asm with the fastest compiler output
that we can find. This is potentially faster, but it has multiple
drawbacks:
- It's more coding work because it needs detailed benchmarks (e.g.,
with many compiler/options).
- It's more review work because we need to deal with inline asm
(including clobbers etc.) and there's a lack of experts reviewers
in this area.
- It's not unlikely that we'll fall behind again in a few compiler
versions, and then we have to deal with this again, i.e., redo the
benchmarks. Given our history here, I doubt that we'll revolve
this timely.
We could change the default of the asm build option to off. But this
will also disable the scalar asm, which is still faster.
We could split the build option into two separate options for field
and scalar asm and only disable the field asm by default. But this
adds complexity to the build and to the test matrix.
My conclusion is that this PR gets the low-hanging fruit in terms of
performance. It simplifies our code significantly. It's clearly an
improvement, and it's very easy to review. Whether re-introducing
better asm (whether from a compiler or from CryptOpt) is worth the
hassle can be evaluated separately, and should not hold up this
improvement.
Solves #726.
33dc7e4d3e1947af4c84c09ecc75ea2eeed3f7e0 asm: add .note.GNU-stack section for non-exec stack (fanquake)
Pull request description:
With this in place, we no-longer see warnings like the following:
```bash
/usr/lib/gcc-cross/arm-linux-gnueabihf/12/../../../../arm-linux-gnueabihf/bin/ld: warning: field_10x26_arm.o: missing .note.GNU-stack section implies executable stack
/usr/lib/gcc-cross/arm-linux-gnueabihf/12/../../../../arm-linux-gnueabihf/bin/ld: NOTE: This behaviour is deprecated and will be removed in a future version of the linker
```
Should close#1434.
ACKs for top commit:
sipa:
utACK 33dc7e4d3e1947af4c84c09ecc75ea2eeed3f7e0
real-or-random:
utACK 33dc7e4d3e1947af4c84c09ecc75ea2eeed3f7e0
Tree-SHA512: f75ded8d971f54d1e871bcc4d815ba367b3e154eea2f18309ecaf9053e22f986bfffcf28418367f8055b65a5a0b245fee045adfcb63a2196df5e2f3aa6c97b89
10271356c8fc34395850ac70df5902571945fbea Return temporaries to being unsigned in secp256k1_fe_sqr_inner (roconnor-blockstream)
Pull request description:
These temporaries seem to been inadvertently changed to signed during a refactoring. Generally, bit shifting is frowned upon for signed values.
ACKs for top commit:
sipa:
utACK 10271356c8fc34395850ac70df5902571945fbea
real-or-random:
utACK 10271356c8fc34395850ac70df5902571945fbea
Tree-SHA512: a9fefe4b146163209662cd435422beb3c9561eb9e83110454184f70df2292992f39ec1971143428e039a80cad2f6285db74de2f059e877ad8756ff739269b67a
With this in place, we no-longer see warnings like the following:
```bash
/usr/lib/gcc-cross/arm-linux-gnueabihf/12/../../../../arm-linux-gnueabihf/bin/ld: warning: field_10x26_arm.o: missing .note.GNU-stack section implies executable stack
/usr/lib/gcc-cross/arm-linux-gnueabihf/12/../../../../arm-linux-gnueabihf/bin/ld: NOTE: This behaviour is deprecated and will be removed in a future version of the linker
```
Should close#1434.
8185e72d299bc77de9c06cc92fd1988676df3bc1 ci: Ignore internal errors in snapshot compilers (Hennadii Stepanov)
Pull request description:
It was discussed on today's IRC meeting.
ACKs for top commit:
real-or-random:
ACK 8185e72d299bc77de9c06cc92fd1988676df3bc1
Tree-SHA512: 0f41ca8303bd3d6efefcd3a544c7bd7dfcf464c57c779c876da4a77cacd262e6c963449d493fdf5a641b0d10b655c8c67fe8a147145b6533328d7bf5344313e1
355bbdf38a2f932daadd02325a0d90d902cb2af4 Add changelog entry for signed-digit ecmult_const algorithm (Pieter Wuille)
21f49d9bec518a769029f809817444a984e735ab Remove unused secp256k1_scalar_shr_int (Pieter Wuille)
115fdc7232a80872c99f88589a5a3608ba757f1d Remove unused secp256k1_wnaf_const (Pieter Wuille)
aa9f3a3c004469033709dc8138892e66adf0b030 ecmult_const: add/improve tests (Jonas Nick)
4d16e90111c050de3b7e25ac451d87cd4e3f874e Signed-digit based ecmult_const algorithm (Pieter Wuille)
ba523be067d6e45957d154838cb9da942704f01a make SECP256K1_SCALAR_CONST reduce modulo exhaustive group order (Pieter Wuille)
2140da9cd5d490d8462d5c7cc909755edc10c1e6 Add secp256k1_scalar_half for halving scalars (+ tests/benchmarks). (Pieter Wuille)
Pull request description:
Using some insights learned from #1058, this replaces the fixed-wnaf ecmult_const algorithm with a signed-digit based one. Conceptually both algorithms are very similar, in that they boil down to summing precomputed odd multiples of the input points. Practically however, the new algorithm is simpler because it's just using scalar operations, rather than relying on wnaf machinery with skew terms to guarantee odd multipliers.
The idea is that we can compute $q \cdot A$ as follows:
* Let $s = f(q)$, for some function $f()$.
* Compute $(s_1, s_2)$ such that $s = s_1 + \lambda s_2$, using `secp256k1_scalar_lambda_split`.
* Let $v_1 = s_1 + 2^{128}$ and $v_2 = s_2 + 2^{128}$ (such that the $v_i$ are positive and $n$ bits long).
* Computing the result as $$\sum_{i=0}^{n-1} (2v_1[i]-1) 2^i A + \sum_{i=0}^{n-1} (2v_2[i]-1) 2^i \lambda A$$ where $x[i]$ stands for the *i*'th bit of $x$, so summing positive and negative powers of two times $A$, based on the bits of $v_1.$
The comments in `ecmult_const_impl.h` show that if $f(q) = (q + (1+\lambda)(2^n - 2^{129} - 1))/2 \mod n$, the result will equal $q \cdot A$.
This last step can be performed in groups of multiple bits at once, by looking up entries in a precomputed table of odd multiples of $A$ and $\lambda A$, and then multiplying by a power of two before proceeding to the next group.
The result is slightly faster (I measure ~2% speedup), but significantly simpler as it only uses scalar arithmetic to determine the table lookup values. The speedup is due to the fact that no skew corrections at the end are needed, and less overhead to determine table indices. The precomputed table sizes are also made independent from the `ecmult` ones, after observing that the optimal table size is bigger here (which also gives a small speedup).
ACKs for top commit:
jonasnick:
ACK 355bbdf38a2f932daadd02325a0d90d902cb2af4
siv2r:
ACK 355bbdf
real-or-random:
ACK 355bbdf38a2f932daadd02325a0d90d902cb2af4
Tree-SHA512: 13db572cb7f9be00bf0931c65fcd8bc8b5545be86a8c8700bd6a79ad9e4d9e5e79e7f763f92ca6a91d9717a355f8162204b0ea821b6ae99d58cb400497ddc656
Based on the surrounding asserts, 112 bits before this line, and 61 bits after this line, this assertion should be 113 bits. Notably the commensurate line in secp256k1_fe_sqr_inner is correctly assert to be 113 bits.
