docs: write down the shared-key subsystem and how Marmot membership fails
First docs in the repo -- README.md is still the stock KMP template. Three
documents plus an index, covering the parts whose behaviour is not recoverable by
reading the code: where the reasoning lives in a protocol, where a failure mode is
silent, or where a decision looked arbitrary and was not.
marmot-membership.md is the one that earns its place. Everything about adding a
member compiles, the invite reports success, and a member simply never appears --
and the reason is never in the invite code. It records that
inviteMemberToChatRoom hardcodes isOneMemberInitialGroupCreation = false and that
ChatRepository does not expose it, so every group invite takes the deferred-welcome
path including the first, when the group is still just its creator and the commit
has no audience at all. Then why that is silent rather than noisy:
MarmotInboundManager refuses future-epoch messages outright, on both wire formats,
with no queue and no replay, so a commit arriving before its recipient's welcome
is dropped and that member never advances. EPOCH_RETENTION_WINDOW retains past
epochs and does nothing for messages from ahead. Three options are set out with the
per-invite correctness table, including the honest limit that the recommended one
narrows the race without closing it.
shared-key-derivation.md argues why the paths are not BIP32 -- no chain code
exists, hardened derivation is impossible rather than unimplemented, and a FROST
tweak takes the scalar as input so the chain code leaves the problem entirely. It
records the x-only serialisation trap avoided by choosing the scalar directly, and
states the rule that must not be broken: never reconstruct a derived key in the
clear, because k = k' - t hands over the group key rather than one derived key.
shared-key-ceremony.md covers the seven kinds, the three approval gates and why
the coordinator's aggregations are deliberately not among them, faults as values
rather than exceptions, and the transcript's idempotency-by-construction. It also
writes down the invariant that produces no error when broken: pendingApproval must
mirror the gates in advance, or the screen offers an approval that does nothing --
or none while the ritual sits still.
Every factual claim was checked against the source rather than recalled, which
turned up one correction worth having: there are two future-epoch refusals, for
PrivateMessage and for Commit, so the drop covers both wire formats and not just
one.
Each document leads with the failure mode rather than the architecture, on the
grounds that a failure is what sends somebody to docs in the first place, and each
lists its known gaps -- including that none of this has run on a physical device.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-05 14:39:19 +02:00
# mantra docs
Notes on the parts of this app whose behaviour is not recoverable by reading the
code alone — where the reasoning lives in a protocol, a failure mode that is
silent, or a decision that looked arbitrary and was not.
| document | covers |
|---|---|
| [shared-key-ceremony.md ](./shared-key-ceremony.md ) | ChillDKG over NIP-17: the rounds, the approval gates, the chat transcript, participant ordering |
| [shared-key-derivation.md ](./shared-key-derivation.md ) | deriving further keys from the group's threshold key with FROST tweaks — why not BIP32, why no chain code, and the one rule that must not be broken |
| [marmot-membership.md ](./marmot-membership.md ) | how members join an MLS group, and the epoch race that makes a missing member look like a successful invite |
fix: keep a room's MlsGroup alive so a late message can still be read
Two events published in the same second reliably lose one of them. The
receiver stores the kind:445 and produces nothing from it -- no inner
event, no chat line, no error anybody sees, because MarmotGroupEvent is
written before the message is decrypted and so survives while everything
downstream silently does not.
Observed as a FROST signing session that never started on the receiver:
proposeSigning publishes the proposal and then the proposer's own nonce,
the relay handed them back in the other order, and the proposal was
dropped. The nonce is still sitting there filed against a session that
will never exist. The same bug ate a dialect earlier, which then took out
the artifact referencing it via a foreign key.
MLS is specified to tolerate this. RFC 9420 says a receiver that gets
generation N+1 before N keeps the intermediate keys so the older message
can still be read, and quartz's SecretTree does exactly that, in a
private skippedKeys map. What it does not do is persist it:
exportSenderStates() returns the ratchet positions only, so saveState()
drops the cache. NostrDao rebuilt the group from stored state for every
inbound event, so the cache was empty every single time, and generation N
arriving after N+1 failed `require(generation >= applicationGeneration)`
and was swallowed. Terminal -- the key is derived from a ratchet that has
moved past it, and nothing asks the sender to resend.
This keeps the instance alive instead. MlsGroupCache holds one MlsGroup
per room, and the inbound path goes through it, so skippedKeys survives
from one message to the next. That covers the case that actually bites --
a burst arriving in one sync, decrypted one after another against the
same tree -- which is what every bursty flow needs: proposeRitual sends
two, addArtifact sends two, and addChapter sends one per paragraph plus
one, of which only the ones arriving in ascending generation order
survived.
Reuse is conditional on the stored state still being exactly what the
cache last wrote. Sending a message advances the sender ratchet and saves;
so does adding a member. When that happens the cache rebuilds rather than
carrying on from a group that has been overtaken -- which is what keeps
this from being worse than no cache at all: the fallback is always the old
behaviour, never a diverged ratchet.
One lock per room, not one overall, because the group is mutable and
decryption advances it: two events for the same room decrypted at once
would corrupt the tree, and a busy room should not hold up a quiet one.
**This is a mitigation, not the fix.** It does not survive a restart, and
it does not survive another writer, so a long enough reorder still loses
the message. The fix belongs in quartz -- carry skippedKeys through
saveState/restore -- and quartz is a mavenCentral binary, not a fork, so
it cannot be made here. docs/mls-skipped-keys.md has the analysis, the
patch, the migration constraint on the persisted state format, and the
three ways to actually land it.
Not verified end to end: the proposal that exposed this cannot be
recovered, since its generation is already past, so confirming the fix
needs a fresh burst.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-05 23:09:53 +02:00
| [mls-skipped-keys.md ](./mls-skipped-keys.md ) | why a group event that arrives a moment late is dropped for good, which flows trigger it, the quartz fix, and the partial mitigation in this app |
docs: scope the jvm target, and separate it from testing the daos
Two questions arrived together -- whether Room's own testing guidance
applies to this project, and what desktop support would cost -- and they
turned out to have opposite answers. Both are now in docs/jvm-target.md,
phased, with the blocking work separated from the mechanical work.
**The expensive part is already done.** The four-deep native chain --
secp256k1 -> bitcoin-kmp -> lightning-kmp -> lightning-kmp-app -- already
builds for JVM, on every android build we do. The comment at
composeApp/build.gradle.kts:50 records the mechanism without drawing the
conclusion: lightning-kmp-core publishes no android variant, so our
android target resolves it to the *jvm* one, which pulls
secp256k1-kmp-jni-jvm desktop natives, which is exactly why the build has
to name the android artifact by hand. Read the other way round, every JVM
artifact in the chain is already compiled from source by the composite
build. A jvm target adds no cinterop, no C compilation and no new native
constraints. That was the part worth being afraid of, and it is finished.
**The blocker is one level down, and smaller than it looks.**
lightning-kmp-app/library declares 25 expects and implements them across
35 androidMain files. Its jvmMain holds exactly one: fibiprops.jvm.kt, the
Kotlin multiplatform library template's Fibonacci boilerplate, satisfying
two of the 25 -- both of them the template's own. So 23 actuals are
missing, which is why jvm() is commented out there
(library/build.gradle.kts:18), which is why it is commented out here
(composeApp/build.gradle.kts:46). Mantra cannot declare the target until
the fork does.
Six phases, ordered by that dependency. 0 build config; 1 the fourteen
mechanical phoenix actuals; 2 the three SQLDelight JDBC drivers and
NetworkMonitor; 3 key storage; 4 mantra's own sixteen expects; 5 the
desktop entry point. 1-3 are independent and parallelisable, 4 is where
the compiler finally checks the whole thing. Roughly a week to a
launchable build.
**Phase 3 has no day estimate, deliberately.** keyStoreEncryption /
keyStoreDecryption and their two graceful* wrappers delegate on android to
KeystoreHelper.kt -- 116 lines against AndroidKeyStore, StrongBox
attempted first and fallen back from, key material never leaving hardware.
Desktop JVM has no equivalent, so this is a decision rather than a port,
and the doc gives the three real options against what each actually
protects. A fixed-key JCEKS file is named there as a liability rather than
a stopgap: this is wallet seed material, and it lands on top of the
plaintext-key finding already open against this codebase. Recommended
sequencing is a passphrase-derived KEK with the desktop build marked
unsuitable for real funds, so phases 4 and 5 can proceed without the
security question being quietly treated as answered.
Two inherited mistakes are called out rather than carried forward. The old
Aux jvmMain put the database in java.io.tmpdir behind a TODO -- the doc
says not to inherit that in either phase that touches it. And
schedulePlatformLogic goes through WorkManager on android with no desktop
counterpart, so the doc asks for an explicit choice between a no-op and an
in-process coroutine, written down.
**The DAO answer is an appendix, because it is the opposite answer.** None
of the above is needed to test the DAOs, and burying that would have been
misleading. room3-runtime-android:3.0.1 already exposes the no-Context
inMemoryDatabaseBuilder(Function0<T>) overload, and
MantraDatabaseConstructor already supplies what it needs, so Room's
recommended host-machine form compiles in commonTest and runs under
testDebugUnitTest today. The one trap is native and is the secp256k1
problem mirrored: sqlite-bundled-android ships only android-ABI .so under
jni/, so a local unit test's JVM cannot load it and BundledSQLiteDriver
fails at construction; sqlite-bundled-jvm on the androidUnitTest classpath
is the fix. Robolectric neither helps nor is needed -- it cannot load
android .so on the host either.
Everything structural here was checked against the artifacts rather than
recalled: the Room builder overloads by javap on room3-runtime-android,
the two sqlite-bundled native layouts by unzipping both, and the
availability of room3-runtime-jvm, room3-testing, quartz-jvm and the two
SQLDelight drivers by request against the repositories this build actually
resolves from. The absence of android.* and java.* imports in commonMain,
and of any NFC reference from it, was likewise grepped rather than
assumed.
**Not verified: anything that requires compiling.** No jvm target was
turned on, nothing was built, and the day estimates are estimates. Phase 4
is where dependency-substitution surprises would surface if there are any,
and it is precisely the phase nothing here exercises.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-06 00:38:11 +02:00
| [jvm-target.md ](./jvm-target.md ) | what desktop support would cost, phased — why the native chain is already done, why an empty source set in our phoenix fork is the real blocker, and why DAO tests do not need any of it |
docs: write down the shared-key subsystem and how Marmot membership fails
First docs in the repo -- README.md is still the stock KMP template. Three
documents plus an index, covering the parts whose behaviour is not recoverable by
reading the code: where the reasoning lives in a protocol, where a failure mode is
silent, or where a decision looked arbitrary and was not.
marmot-membership.md is the one that earns its place. Everything about adding a
member compiles, the invite reports success, and a member simply never appears --
and the reason is never in the invite code. It records that
inviteMemberToChatRoom hardcodes isOneMemberInitialGroupCreation = false and that
ChatRepository does not expose it, so every group invite takes the deferred-welcome
path including the first, when the group is still just its creator and the commit
has no audience at all. Then why that is silent rather than noisy:
MarmotInboundManager refuses future-epoch messages outright, on both wire formats,
with no queue and no replay, so a commit arriving before its recipient's welcome
is dropped and that member never advances. EPOCH_RETENTION_WINDOW retains past
epochs and does nothing for messages from ahead. Three options are set out with the
per-invite correctness table, including the honest limit that the recommended one
narrows the race without closing it.
shared-key-derivation.md argues why the paths are not BIP32 -- no chain code
exists, hardened derivation is impossible rather than unimplemented, and a FROST
tweak takes the scalar as input so the chain code leaves the problem entirely. It
records the x-only serialisation trap avoided by choosing the scalar directly, and
states the rule that must not be broken: never reconstruct a derived key in the
clear, because k = k' - t hands over the group key rather than one derived key.
shared-key-ceremony.md covers the seven kinds, the three approval gates and why
the coordinator's aggregations are deliberately not among them, faults as values
rather than exceptions, and the transcript's idempotency-by-construction. It also
writes down the invariant that produces no error when broken: pendingApproval must
mirror the gates in advance, or the screen offers an approval that does nothing --
or none while the ritual sits still.
Every factual claim was checked against the source rather than recalled, which
turned up one correction worth having: there are two future-epoch refusals, for
PrivateMessage and for Commit, so the drop covers both wire formats and not just
one.
Each document leads with the failure mode rather than the architecture, on the
grounds that a failure is what sends somebody to docs in the first place, and each
lists its known gaps -- including that none of this has run on a physical device.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-05 14:39:19 +02:00
Start with the ceremony if you are new to this area; the other two both assume it.
fix: keep a room's MlsGroup alive so a late message can still be read
Two events published in the same second reliably lose one of them. The
receiver stores the kind:445 and produces nothing from it -- no inner
event, no chat line, no error anybody sees, because MarmotGroupEvent is
written before the message is decrypted and so survives while everything
downstream silently does not.
Observed as a FROST signing session that never started on the receiver:
proposeSigning publishes the proposal and then the proposer's own nonce,
the relay handed them back in the other order, and the proposal was
dropped. The nonce is still sitting there filed against a session that
will never exist. The same bug ate a dialect earlier, which then took out
the artifact referencing it via a foreign key.
MLS is specified to tolerate this. RFC 9420 says a receiver that gets
generation N+1 before N keeps the intermediate keys so the older message
can still be read, and quartz's SecretTree does exactly that, in a
private skippedKeys map. What it does not do is persist it:
exportSenderStates() returns the ratchet positions only, so saveState()
drops the cache. NostrDao rebuilt the group from stored state for every
inbound event, so the cache was empty every single time, and generation N
arriving after N+1 failed `require(generation >= applicationGeneration)`
and was swallowed. Terminal -- the key is derived from a ratchet that has
moved past it, and nothing asks the sender to resend.
This keeps the instance alive instead. MlsGroupCache holds one MlsGroup
per room, and the inbound path goes through it, so skippedKeys survives
from one message to the next. That covers the case that actually bites --
a burst arriving in one sync, decrypted one after another against the
same tree -- which is what every bursty flow needs: proposeRitual sends
two, addArtifact sends two, and addChapter sends one per paragraph plus
one, of which only the ones arriving in ascending generation order
survived.
Reuse is conditional on the stored state still being exactly what the
cache last wrote. Sending a message advances the sender ratchet and saves;
so does adding a member. When that happens the cache rebuilds rather than
carrying on from a group that has been overtaken -- which is what keeps
this from being worse than no cache at all: the fallback is always the old
behaviour, never a diverged ratchet.
One lock per room, not one overall, because the group is mutable and
decryption advances it: two events for the same room decrypted at once
would corrupt the tree, and a busy room should not hold up a quiet one.
**This is a mitigation, not the fix.** It does not survive a restart, and
it does not survive another writer, so a long enough reorder still loses
the message. The fix belongs in quartz -- carry skippedKeys through
saveState/restore -- and quartz is a mavenCentral binary, not a fork, so
it cannot be made here. docs/mls-skipped-keys.md has the analysis, the
patch, the migration constraint on the persisted state format, and the
three ways to actually land it.
Not verified end to end: the proposal that exposed this cannot be
recovered, since its generation is already past, so confirming the fix
needs a fresh burst.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-05 23:09:53 +02:00
Read the skipped-keys note before debugging any "the other device never got it"
report — it is silent, and it looks like every other kind of delivery failure.
