Files
giza-kmp/docs/submodule-refresh.md
T
Kgothatso NgakoandClaude Opus 5.5 d4dd87f2a2
Material Design conformance / budgets (push) Canceled after 0s
Material Design conformance / tests (push) Canceled after 0s
docs: the submodule refresh is pushed, and what is left
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:00:32 +02:00

330 lines
17 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Submodule refresh: carrying reduced FROST enrollment up the chain
Giza builds against five nested repositories, each pinning the one below it. Today's work in
secp256k1-kmp — reduced (version 2) FROST enrollment — has reached the bottom two levels and stops
there. This plan moves every pointer to the tip of the branch it tracks, bottom up, so each level
builds against the one beneath it as it now is.
Measured on 2026-11-19 after `git fetch` at every level.
## Where it stands
Phases 1–5 are built, committed and pushed. Each submodule level's work is on
`claude/reduced-frost-enrollment`, and bitcoin-kmp `threshold`, lightning-kmp `threshold` and
lightning-kmp-app `master` were fast-forwarded to it on 2026-11-19, so the hashes below are on
the tracked branches:
| level | commit | what | checked |
|---|---|---|---|
| secp256k1-kmp | — (at `ce05021`) | Phase 1 | `:tests:jvmTest` 107/107, natives built from `a04dc1ed` |
| bitcoin-kmp | `36e2ed5` | bump secp256k1-kmp | `jvmTest` 258/258 |
| bitcoin-kmp | `1982eba` | `FrostReducedEnrollment` + 9 tests | `jvmTest` all pass |
| lightning-kmp | `df778bc4` | bump bitcoin-kmp | `:lightning-kmp-core:jvmTest` 1001/1002 — see below |
| lightning-kmp-app | `e4779ff` | bump lightning-kmp | `:library:compileKotlinJvm`, `:library:compileAndroidMain` |
| Giza | `051b339a` | bump lightning-kmp-app | `compileDebugKotlinAndroid`, `jvmTest` 1171/1171, `check` |
The one lightning-kmp failure is `ElectrumClientTest` "get headers", a 15 s timeout fetching
2016 headers from a live Electrum server. It fails the same way with bitcoin-kmp, secp256k1-kmp
and zkp put back to `9e3c4f8` / `8adf691` / `7a35fde1`, so it predates this work.
lightning-kmp's GitHub default branch is `master` (`44c8d945`, 2026-08-29), but
lightning-kmp-app's `.gitmodules` tracks `threshold`, which is where this went.
Still to do:
- Tag lightning-kmp-app at `e4779ff` for JitPack consumers. It has no tags yet, so the scheme
is yours to choose.
- The checks Linux and this session could not make: an Android install over the current build
and a desktop run, both signing in a group; secp256k1-kmp's `connectedAndroidTest`; an iOS
build on a Mac.
## The chain
| level | repository | tracks | pinned by its parent | tip of tracked branch | behind |
|---|---|---|---|---|---|
| 0 | `giza-kmp` | `giza` | — | — | — |
| 1 | `kngako/lightning-kmp-app` | `master` | `67a0376` | `949bfcb` (2026-11-02) | **3** |
| 2 | `kngako/lightning-kmp` | `threshold` | `688d36ad` (by `67a0376`) / `065dcf89` (by `master`) | `065dcf89` (2026-11-02) | 0 from `master`, 3 from what Giza pins |
| 3 | `kngako/bitcoin-kmp` | `threshold` | `9e3c4f8` | `9e3c4f8` (2026-09-15) | 0 |
| 4 | `kngako/secp256k1-kmp` | `threshold` | `8adf691` | `ce05021` (2026-11-19) | **4** |
| 5 | `frost/secp256k1-zkp` | `main` | `a04dc1ed` (by `ce05021`) | `a04dc1ed` (2026-11-18) | 0 |
Paths, from the Giza root:
```
lightning-kmp-app
└─ experimental/lightning-kmp
└─ experimental/bitcoin-kmp
└─ experimental/secp256k1-kmp
└─ native/secp256k1 (secp256k1-zkp)
```
So the gap is in two places:
- **bitcoin-kmp → secp256k1-kmp.** bitcoin-kmp pins `8adf691`; the four commits after it are
today's enrollment work (`1b70238` bump zkp, `3de04cd` bindings, `2f60bc3` tests, `ce05021`
docs). Every level above bitcoin-kmp inherits the gap.
- **Giza → lightning-kmp-app.** Giza pins `67a0376`; `master` is three commits ahead with the
lightning-kmp bumps for offers (`5478993`, `7aaacad`) and the swap-in signer (`949bfcb`).
secp256k1-kmp → zkp is already current: `1b70238` moved it from `7a35fde1` to `a04dc1ed`, the
tip of zkp `main`, which carries the six `frost_enrollment:` commits of 2026-11-18.
## What today's change is
Additive. `Secp256k1` gains five functions and changes none:
- `frostEnrollmentReducedParamsHash` — the v2 parameters hash, bound to a `sessionId32`.
- `frostEnrollmentReducedSharesGen` — round 1.1, a triangular share row: u(u−1)/2 helper
messages instead of u(u−1).
- `frostEnrollmentReducedShareAgg` — round 1.2, returns `FrostEnrollmentShareAggResult`, naming
the helper at fault by identifier.
- `frostEnrollmentReducedSecshareGen` — round 2, with the expected public share mandatory.
- `frostEnrollmentPubshareDeriveFromCommitments` — derives the expected public share from
authenticated aggregate VSS commitments rather than helper public shares.
The v1 (`frostEnrollment…`) functions are untouched apart from KDoc; the v1 test diff is
comments only. v1 and v2 messages must never be mixed within a run, and the hashes are
incompatible by design, so nothing that speaks v1 today can be silently broken by v2.
The module is marked experimental upstream, and the generalized protocol with more helpers than
the threshold "requires independent review". That is a reason to bring it in and test it, not a
reason to ship a user-facing feature on it — see Phase 6.
Who calls enrollment today:
- bitcoin-kmp: `fr.acinq.bitcoin.crypto.frost.FrostEnrollment` wraps v1 only (`7de8761`,
2026-09-04), with `FrostEnrollmentTestsCommon`.
- lightning-kmp, lightning-kmp-app, Giza: nothing.
So a pointer bump alone changes no behaviour anywhere above secp256k1-kmp. It only makes the v2
functions reachable.
## Before starting
**Use a worktree with its own submodule checkout, not the main checkout.** The main checkout at
`giza-kmp/` has uncommitted content in zkp (`native/secp256k1/README.md` modified), which shows
as `m`/`M` at every level above it. Don't commit over it or reset it — it isn't part of this work.
`docs/jvm-target.md` explains that linked worktrees share `.git/modules/` with the main checkout;
moving a submodule's HEAD in one is visible as drift in the other. Check `git submodule status
--recursive` in both before and after.
A concurrent fetch was seen on zkp while this was measured (`cannot lock ref
'refs/remotes/origin/frost'`), so another session may be working in these repositories. Check
before pushing any `threshold` or `master`.
In this worktree:
```bash
git submodule update --init --recursive
```
Then `local.properties` at all five Gradle roots (the loop is in `README.md`).
**Branch per level.** At each level, do the bump on a `claude/reduced-frost-enrollment` branch
cut from the tracked branch, and fast-forward the tracked branch only once Phase 5 is green. A
failure at the top should not leave `threshold` advanced at three levels below it. This matches
how `claude/group-swap-in-signer` went in.
**Push order is bottom up, and nothing is pushed until the level above has been built against
it.** A parent may only pin a commit that exists on the child's remote; push the child's branch
before the parent's commit is pushed, never the other way.
## Phase 1 — verify secp256k1-kmp at `ce05021`
No commit. Prove the bottom of the chain works before stacking on it.
1. In `…/secp256k1-kmp`, check out `ce05021` and `git submodule update` so zkp is at
`a04dc1ed`. Confirm with `git submodule status`.
2. **Rebuild the natives.** The JNI and iOS libraries are built from zkp by
`native/build.sh`, `native/build-android.sh` and `native/build-ios.sh`, and the main
checkout's `native/build/{linux,android-*}` were built from an older zkp. A stale
`libsecp256k1` shows up as a missing `secp256k1_frost_enrollment_reduced_*` symbol at link or
load time, so delete `native/build/` and rebuild Linux and Android.
3. Run the tests that came with the change, on JVM:
- `FrostReducedEnrollmentTest` — the end-to-end v2 run.
- `FrostEnrollmentVectorsTest` — frozen vectors from zkp's Python generator.
- `FrostEnrollmentTest` — v1, must still pass unchanged.
- The rest of `:tests:jvmTest`, for ChillDKG and FROST signing, which Giza uses.
4. If an Android emulator is available, `:tests:connectedAndroidTest` for the same classes, since
the JNI glue (`fr_acinq_secp256k1_Secp256k1CFunctions.c`, +226 lines) is where a marshalling
mistake would hide.
**Done when** all of the above pass with natives built from `a04dc1ed`.
**Decision recorded here, not made:** zkp has a `frost` branch, 25 commits ahead of `main`,
holding the BIP 445 rewrite of the `frost` module (merged with `main` today, `c3ad9e7e`).
secp256k1-kmp tracks `main` (`.gitmodules: branch = main`) and is current with it. Moving to
`frost` would replace the FROST signing module Giza signs with — a different change with its own
plan, not part of a refresh.
## Phase 2 — bitcoin-kmp: bump secp256k1-kmp
One commit on bitcoin-kmp `claude/reduced-frost-enrollment`, from `threshold` (`9e3c4f8`):
```
Bump secp256k1-kmp: reduced FROST enrollment
```
moving `experimental/secp256k1-kmp` from `8adf691` to `ce05021`.
Then in bitcoin-kmp:
1. `./gradlew jvmTest`, with the attention on `crypto/frost/` — `FrostEnrollmentTestsCommon`
(v1 through the wrapper), the ChillDKG tests, and the FROST signing tests. The wrapper calls
only functions whose signatures did not change, so this should compile without edits; if it
does not, that is a finding about Phase 1, not something to patch here.
2. `./gradlew compileKotlinIosArm64` on a Mac if one is to hand. The cinterop declaration is
regenerated from the new `secp256k1_frost_enrollment.h`, which grew by 180 lines.
**Done when** bitcoin-kmp's JVM tests pass against `ce05021`.
## Phase 3 — bitcoin-kmp: wrap reduced enrollment (recommended, separable)
The wrapper is what Giza would call, in Giza's types (`PublicKey`, `ByteVector32`, `Either`), so
"making use of" v2 one level up means extending `FrostEnrollment`. It can land after Phases 4–5
without blocking them; doing it now means the bump that reaches Giza carries a usable API rather
than only a reachable one.
Second commit on the same branch: a new object, `FrostReducedEnrollment`, in
`crypto/frost/FrostReducedEnrollment.kt`, beside `FrostEnrollment` rather than inside it:
- `paramsHash(…, sessionId: ByteVector32)`.
- `generateShares(…)` returning `ReducedEnrollmentShares(kept, outgoing: Map<UInt, ByteVector32>, paramsHash)`.
- `aggregateShares(kept, received: Map<UInt, ReducedEnrollmentMessage>, …)` returning
`ReducedShareAggregation` (`Aggregated(message)` / `HelperAtFault(helperId)`).
- `derivePublicShareFromCommitments(thresholdPublicKey, commitments: List<PublicKey?>, newId, nParticipants)`
— `null` for the point at infinity, encoded as 33 zero bytes on the way down.
- `generateSecretShare(received: Map<UInt, ReducedEnrollmentMessage>, …, expectedPublicShare: PublicKey)`
— non-nullable, because upstream made it mandatory.
An earlier draft of this plan reused v1's `EnrollmentShares` and `ShareAggregation`. It did
not survive contact with the routing: v1's lists are aligned with the helper ids and every
non-own entry is sent, while v2 sends only to larger identifiers and smaller ones get nothing.
A position-aligned list would have made the mistake upstream warns about — routing by array
position — the natural way to call it. So v2 takes and returns messages *keyed by identifier*,
and checks that exactly the expected senders are present: a missing or extra message is the
caller's error (`Either.Left`), never a helper at fault, since blaming a helper for a message
the caller failed to route is indistinguishable, to the group, from that helper misbehaving.
v1 and v2 are separate objects with separate message types, not a `version` parameter, so a
mix within a run is a type error.
Copy upstream's warnings into the KDoc: experimental, not for production, and the
more-helpers-than-threshold case unreviewed.
Tests in `FrostReducedEnrollmentTestsCommon`, mirroring `FrostReducedEnrollmentTest`:
- enrollment for every n in 3..5, t in 2..n, u in t..n, with helpers listed out of order, and a
repair at an existing identifier, end to end through the wrapper;
- shares routed only to larger identifiers, u(u−1)/2 in all;
- `HelperAtFault` carrying the right identifier when one helper's message is replayed from
another `sessionId`;
- a missing or extra sender being an error rather than a fault;
- round 2 failing on a wrong `expectedPublicShare`, a wrong `sessionId`, or a tampered hash;
- `derivePublicShareFromCommitments` agreeing with `derivePublicShare` for the same group, and
`null` encoding the point at infinity.
**Done when** those pass and the v1 tests are unchanged.
## Phase 4 — lightning-kmp, then lightning-kmp-app
**lightning-kmp.** One commit on `claude/reduced-frost-enrollment` from `threshold`
(`065dcf89`):
```
Bump bitcoin-kmp: reduced FROST enrollment
```
lightning-kmp uses no enrollment and no FROST enrollment wrapper, so the check is that it still
compiles and its JVM tests pass against the new bitcoin-kmp. The full suite is long; at minimum
`compileKotlinJvm` and the `crypto` and `channel` test packages.
**lightning-kmp-app.** One commit on `claude/reduced-frost-enrollment` from `master`
(`949bfcb`), bumping `experimental/lightning-kmp` from `065dcf89` to the Phase 4 commit:
```
Bump lightning-kmp: reduced FROST enrollment down the chain
```
Check `:library:compileKotlinJvm` and `:library:compileDebugKotlinAndroid`.
Tag it once the tracked branch is fast-forwarded (end of Phase 5): JitPack resolves by tag, and
`docs/nsec-sign-in.md` (Phase 8) set that precedent.
## Phase 5 — Giza: bump lightning-kmp-app
One commit on this branch, `claude/submodule-dependency-updates-64bd3d`, moving
`lightning-kmp-app` from `67a0376` to the Phase 4 commit.
This brings in more than enrollment: the three commits between `67a0376` and `master` (offers
decline/answer, swap-in user key held by a signer). They are wallet code in lightning-kmp. Giza
calls none of it, and `wallet-audit.sh` looks at Giza's own code rather than the submodule, but
the commit message should say they arrive, so a later bisect is not surprised.
Checks, in the order they fail cheapest:
```bash
./gradlew :composeApp:compileDebugKotlinAndroid :composeApp:jvmTest
```
```bash
./gradlew check
```
`check` runs the M3, brand and wallet audits. Of the JVM tests, the ones that touch this change
are:
- `ForkCompatibilityTest` — pins the npub and profile id derived from a phrase; it must not move.
- The ChillDKG and FROST signing tests under `managers/` and the subgroup ceremony tests
(`SubgroupCeremonyInParentRoomTest`), since they drive the natives rebuilt in Phase 1.
Then:
- **An Android debug build installed over the current one**, opening an existing profile and
signing in a group. This is the only check that exercises the Android JNI `.so` built in
Phase 1 inside the app. `composeApp/build.gradle.kts` resolves
`secp256k1-kmp-jni-android:0.24.0` from source through the composite build; the version string
did not change (`ce05021` still says `0.24.0`), so that line needs no edit.
- **A desktop build** (`jvm` target) doing the same.
- **An iOS build on a Mac.** Not possible on Linux. Until it is done, say so in the PR.
**Done when** all of the above pass. Then, bottom up: fast-forward `threshold` in secp256k1-kmp
(already there), bitcoin-kmp, lightning-kmp, and `master` in lightning-kmp-app to their
`claude/reduced-frost-enrollment` branches, push each, tag lightning-kmp-app, and open the Giza PR.
Fast-forward rather than merge: a fast-forward makes no new commit, so every pointer already
committed in Phases 2–5 stays valid. A merge commit would mean redoing each bump above it.
## Phase 6 — Giza uses reduced enrollment (not part of this refresh)
Nothing in Giza enrolls or repairs today. Doing so is a feature: adding a member to a group key
without rerunning ChillDKG, or recovering a lost share. It needs its own document in the manner of
`docs/shared-key-ceremony.md`, because upstream leaves exactly the hard parts to the application:
transport, deduplication, authorization, consensus on the extended public-share table, and atomic
persistence before signing. Two things this refresh does decide for it:
- Use v2 (`Reduced`) only. There is no installed base of v1 enrollment messages to stay
compatible with.
- Get the expected public share from authenticated group state — the `GroupKeyStateEvent`
table, or VSS commitments if ChillDKG's are kept — never from the helpers.
And it should not ship to users while upstream calls the module experimental.
## Afterwards
`docs/giza-wallet-removal.md`, Phase 7, plans to bring bitcoin-kmp in as Giza's own submodule
and drop lightning-kmp-app. If that lands, this chain shortens to three levels and Phases 4–5
collapse into one bump. The two do not conflict: do this refresh now, and that move starts from
a bitcoin-kmp that already has v2.
## Summary
| phase | where | commit | done when |
|---|---|---|---|
| 1 | secp256k1-kmp | none | natives rebuilt from `a04dc1ed`; v1, v2 and vector tests pass |
| 2 | bitcoin-kmp | bump secp256k1-kmp `8adf691` → `ce05021` | JVM tests pass |
| 3 | bitcoin-kmp | `FrostEnrollment` v2 wrapper + tests | wrapper tests pass, v1 unchanged |
| 4 | lightning-kmp, lightning-kmp-app | two bumps | both compile, lightning-kmp tests pass |
| 5 | Giza | bump `lightning-kmp-app` `67a0376` → new | `check`, `jvmTest`, Android + desktop on device; iOS noted |
| 6 | Giza | — | separate design document |