The crypto was tested; the logic that acts on it was not. Both untested
pieces were the security-critical ones, and neither fails loudly when it
goes wrong -- one silently widens who may impersonate whom, the other
silently destroys a message.
Extracted MarmotDirectMessage.classify, which decides what an arriving
wrap is to this device, from ChatMessage.directMessage, which turns that
decision into rows. The decision is pure; only the filing needs a
database, and Room-backed code cannot be unit-tested in this project. Same
split, and for the same reason, as pulling the wrap/open crypto out of the
DAO in the first place.
Extracted MarmotInboundManager.mip03Rejection for the same reason. Its
kind:1059 exemption is the most dangerous line in this feature: widened to
another kind, or stripped of its kind guard, it hands every member of
every group the ability to publish events as anybody, and nothing else in
the pipeline would notice. There is now a test that walks seven kinds and
asserts each is still held to MIP-03.
Fifteen cases, the ones worth naming:
`our own message is ours, even though we cannot open it` and `ours is
decided before anything is opened`. A sender cannot decrypt their own wrap
-- the key was discarded -- so by decryption alone this is
indistinguishable from a bystander's view, and only the MLS identity
separates them. Get it wrong and the inbound path files an empty
placeholder over the row sendChatMessage wrote, which holds the only copy
of those words. It is the one failure here that loses data rather than
rendering something wrong.
`words sealed by one member and sent by another are dropped`. The check
that replaces MIP-03 for this kind, tested directly rather than described
in a comment as it was before.
One test asserts something I had wrong. I expected a seal relabelled with
another member's pubkey to be caught by the signature check; it never
reaches it. NIP-44 derives the conversation key from the pubkey being
claimed, so relabelling a seal makes it undecryptable by the person it was
encrypted for -- the label is bound to the key, not merely asserted
alongside it. The outcome is Unreadable, which is the truth: the recipient
genuinely cannot read it. `a seal tampered with after signing is dropped`
covers what verify() does catch, using an alteration that survives
decryption.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A one-to-one message inside a Marmot group is a stock NIP-59 gift wrap
carried as the MLS application payload: a throwaway-keyed kind:1059 around
a sender-signed kind:13 seal around the kind:14 rumor holding the words.
Every member decrypts the MLS layer and sees the wrap; only the recipient
can open it. See docs/marmot-direct-messages.md.
This is the crypto on its own, with no database and no MLS state, because
the outbound path (the notary) and the inbound path (the kind switch in
ChatMessage) both need it and neither can be unit-tested -- there is no
sqlite driver on the JVM test classpath. Extracting it first is what makes
the ten tests here possible; real secp256k1 does load under
testDebugUnitTest, so none of this is mocked.
Three choices worth stating, all of them consequences of the wrap using a
throwaway key rather than the sender's own:
Nothing in the wrap names the sender. GiftWrapEvent.create mints and
discards its own random key, so who sent a message comes from the MLS
frame around it -- authenticated to a leaf, and unforgeable -- rather than
from a self-asserted pubkey field. The seal inside is the only layer the
sender signs, which is what the inbound path will bind to the MLS sender
identity before it renders a word.
The sender cannot reopen their own message. The throwaway key is gone at
send time and nothing reconstructs it. `the sender cannot reopen their own
message` asserts that rather than leaving it to be discovered, because the
obvious fix -- persisting the throwaway private key -- would be strictly
worse than the identity-keyed wrap this was chosen over, and would
reintroduce the attribution the throwaway key exists to remove.
No layer is fuzzed. NIP-59 randomises the wrap and the seal by up to two
days to frustrate correlation at a relay, and both GiftWrapEvent.create
and SealedRumorEvent.create default to it. There is no relay at this layer
and the kind:445 already carries the true time, so fuzzing would only
scatter the "sent a private message" line up to two days out of position
in every other member's transcript.
open() returns null rather than throwing on every way a wrap can fail to
open -- somebody else's message, a malformed payload, a layer that is not
the kind it claims. Its caller is midway through processing a kind:445
that may carry a perfectly good message for somebody else, and an
exception would abandon all of it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Once a ceremony completes, the shared-key screen offers its coordinator a Marmot
room named "<group> (#admins)" with every member of the ceremony in
MarmotGroupData.adminPubkeys. The room the ceremony ran in is NIP-17, where nobody
administers anything; this gives the same people a room where every one of them
can act, which is the shape a group that has just made a t-of-n key is asking for.
Built directly rather than through MarmotGroupData.bootstrap, which hardcodes a
single admin, and baked into the epoch-0 GroupContext so later invitees receive a
populated group from their welcome instead of chasing a bootstrap commit that
predates their membership.
## The id is derived, not random
Every other Marmot room mints `nostrGroupId` as RandomInstance.bytes(32). This one
derives it from the group's threshold key, settling the
`// TODO: Generate GID through frost...` already sitting in
SelectChatRoomTypeViewModel.
Derivation buys two things random cannot. Every member's device can compute the id
from a ceremony they all took part in, so the room is addressable without being
announced; and two members racing to create it arrive at the same id rather than
two rival rooms -- which is why createAdminGroup returns to the existing room
instead of minting a second one.
## Why the derivation is what it is
SharedKeyDerivation walks the path as successive FROST tweaks, one per index,
returning both the XonlyPublicKey and the TweakCache. The cache is not an
optimisation: a signing session created without the same tweaks aggregates to
signatures that verify against a different key, which is why the id is usable as
an identity later rather than only as a label.
It is not BIP32, and the doc comment argues that at length rather than leaving it
to be rediscovered. A BIP32 node is a key *and* a chain code; ChillDKG produces no
chain code. BIP32 wants one only because it computes the tweak scalar for you, and
a FROST tweak takes that scalar as an input -- so choosing it directly removes the
chain code from the problem rather than requiring one to be invented and agreed
forever. It also removes a trap: with x-only keys there is no single obvious
serP(K_par), and two devices picking different parity conventions would silently
derive different keys rather than fail.
Each scalar commits to the key being tweaked as well as the index, so steps cannot
be reordered or replayed at a different depth. Tests cover that, determinism
across calls, path and key sensitivity, and that the cache and the public key
agree.
Hardened derivation is not available here and never will be: it needs the parent
private key, which in a threshold group nobody has. That leaves the non-hardened
weakness -- k' = k + t with publicly computable t inverts -- so anyone learning one
derived private key recovers the group key and can sign with no quorum at all. The
rule that follows is stated at the top of the file: never reconstruct a derived key
in the clear.
## The path is recorded in the room
MIP-01's group data is a fixed TLS schema with no extension map, so a custom field
would emit bytes other Marmot clients cannot decode. The path rides in the
description instead, on its own line under a marker, so somebody rewriting the
rest of the description does not cost the group the record of how its key was
derived:
Admins of Ubuntu Collective.
Shared key path: m/9420/0/0
Worth storing although the path is currently a constant: it is what rebuilds the
TweakCache a signing session needs, and recomputing from the constant only holds
while the constant never changes. parsePath refuses hardened indices rather than
tolerating them -- such a path cannot have been walked here, so acting on one
would derive something other than what the room claims.
## Known limits
Members without a published MarmotKeyPackage cannot be invited; inviteAdmins
collects them and logs them, and the coordinator is not yet told.
Invites go one at a time, each advancing the MLS epoch, so the room is re-read
between them. That inherits a silent failure mode documented in
docs/marmot-membership.md: the first invite takes the deferred-welcome path even
though the group is still just its creator, and a commit reaching a member before
their welcome is dropped rather than queued. Not introduced here -- group creation
has always done this -- but more visible in a room whose whole membership is known
up front.
Nothing here has run on a device.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A negentropy exchange compares two sets defined by the SAME filter: the relay
builds its side from the filter carried in NEG-OPEN, and this device builds its
side from getNegentropicNostrFeedIds. Any clause we fail to apply locally makes
our set a superset of the relay's, and each extra row comes back as an id the
relay is "missing" -- which this app then queues as a broadcast. Any clause we
apply more tightly makes it a subset, and the difference comes back as ids to
re-download that we already hold. Neither shows up as an error; both show up as a
sync that never settles.
getNegentropicNostrFeedIds was a `when` over the shape of the filter, dispatching
to one of eight hand-written @Query methods. Each method could only bind the
parameters it happened to declare, so the branches disagreed with the filter they
were serving:
- `until` was expressible by NO branch. It is sent to the relay in NEG-OPEN and
was never applied here, so every local event past the requested window was
reported to the relay as one it lacked.
- `since` was strict (`createdAt > :since`) where NIP-01 is inclusive, so an
event stamped exactly on the boundary was a phantom "need" on every pass.
- `kinds && authors` was tested before any tag branch, so a filter carrying
kinds, authors AND tags silently dropped the tags. `kinds && ids` dropped
authors. Every branch dropped whatever it had no parameter for.
- tags were matched with `tags LIKE '%' || :value || '%'` -- a substring scan of
the serialized tag JSON that matches the value in ANY tag position. A pubkey
referenced in an `e` tag counted as a `p` match. And only `tags[name].first()`
was ever bound, so the second and later values of a tag were dropped.
- the reply branch matched `'%' || :eventId || '%reply%'`, which needs the
literal text "reply" to appear somewhere after the id: it misses
`["e","<id>"]` with no marker and false-positives on any later tag containing
the word.
- the `else` branch ignored the filter's kinds entirely and substituted
`arrayOf(TextNoteEvent.KIND)`. A filter with only authors, or only tags, got a
local set of kind-1 notes -- unrelated to what the relay was reconciling.
- more than one filter returned emptyList() with a "not yet supported" warning.
That is the worst available answer: an empty local set tells the relay we hold
none of these events, so it hands back its entire set as ids to download.
- the limit branches ordered `createdAt ASC LIMIT n`, returning the OLDEST n
where a relay answering a limited filter returns the newest.
## The replacement
NostrEventFilterQuery translates a SynchronizationFilter into one SQL statement
that applies every clause, and NostrEventDao.getNostrEventsMatchingFilter runs it
as a @RawQuery. Raw because a nostr filter is a variable set of constraints over
variable-length lists, which is precisely what @Query cannot express -- and what
drove the per-shape methods that dropped constraints in the first place.
Semantics follow quartz's FilterMatcher, which is what the relays this app talks
to implement: membership for ids/authors/kinds; AND between tag names and OR
between the values of one name for `tags`; AND both ways for `tagsAll`; inclusive
`since`/`until`; and a present-but-empty list matches nothing.
Tags are matched by looking for the `["<name>","<value>"` fragment, built by
encoding through the same serializer that wrote the column so escaping agrees,
with `%`/`_`/`\` escaped and `ESCAPE '\'` on the LIKE so a wildcard inside a value
cannot widen the match. Anchoring on the tag name and on the closing quote of the
value is what keeps a hex string from matching in an unrelated tag position.
Multiple filters are now the union of their matches, de-duplicated by id.
## The Marmot branch is kept, and narrowed
Group messages still answer from MarmotGroupEvent: that table carries the NIP-40
expiry a relay uses to decide whether it still serves an event, and an indexed
chatRoomId instead of a scan of the tags JSON. But the branch now only claims a
filter it can fully honour -- exactly kind 445, an `h` tag, and nothing else --
because it answers from a different table and would otherwise reproduce the same
silently-dropped-constraint bug it is an exception to. It also fills in the `h`
tag and the real signature on the NostrEvent it synthesizes rather than leaving
them empty.
## Tests
NostrEventFilterQueryTest pins the generated SQL and the bound values for each
clause, including tag escaping and the empty-list case. It asserts the
translation rather than eyeballing it, because a dropped clause is not an error
at runtime -- it is reconciliation quietly reporting differences that are not
real.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ChillDKG has no session-params object the group agrees on out of band: every step
takes the host public keys and the threshold and hashes them into the session
identity itself. A group whose devices order their participants differently
therefore gets no key at all, and nothing in the protocol tells you that is what
went wrong. ChillDkgRitualManager has each device derive that order
independently -- sort the collected host keys, and order each round's messages by
their sender's host key to match -- and until now nothing checked that the two
rules agree, or that they agree with what ChillDKG expects.
Four tests, against the real library rather than a stand-in:
a ritual ordered by host key produces one shared key
A full 2-of-3 run -- step1, coordinatorStep1, step2, coordinatorFinalize,
participantFinalize -- with the participant set built by hostPublicKeys()'s
rule and both rounds ordered by orderedPayloads()' rule. Asserts every
member lands on the same threshold public key and on distinct shares.
sorted host keys give every device the same participant order
The same members in three arrival orders, since relays deliver host keys in
whatever order they please, must sort to one order.
one device ordering participants differently gets no key
The negative that keeps the other two honest: with one member running the
same people in another order, some step has to fault. Without this a broken
ordering rule could pass the happy-path test by being uniformly broken.
host keys are not the nostr keys they come from
deriveHostSecretKey's two obligations: it must not hand ChillDKG the nostr
identity key (a flaw in either protocol would otherwise reach the other),
and it must be deterministic, or a reinstall cannot recover the share.
These live in commonTest and run under `./gradlew :composeApp:testDebugUnitTest`.
The secp256k1 natives do load there: the Android loader fails and falls back to
extracting the JVM platform build, so these are real curve operations, not
mocked ones. Room-backed code still cannot be tested this way, which is why the
manager's database behaviour is not covered here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
