39eac618380fdf4a29bcdad3a9434e55640f8535
543 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Kgothatso Ngako
|
39eac61838 |
Merge branch 'mantra' into claude/long-running-chat-sync-8983dc
mantra had moved on ~30 commits, several of them in exactly this area — and it turns out both branches independently found the same bug and drew the same conclusion about the same filter. **The overlap.** |
||
|
Kgothatso Ngako
|
bcdfd2ec94 |
Merge branch 'mantra' into claude/marmot-direct-message-type-7a0473
Twenty-two commits had landed on mantra since this branch left it, several of them in the same files. Merged this way round so mantra stayed untouched until the result compiled and its tests passed. The migration had to be renumbered, and this is the conflict that mattered. mantra is at database version 7 and already has its own 5.json -- for MarmotInnerEvent.payloadEventId, nothing to do with direct messages. This branch had also written a 5.json, for a different schema. Resolved by restoring mantra's 5.json untouched and moving the direct message columns to an AutoMigration(7, 8) with a regenerated 8.json. Taking either 5.json over the other would have left every device validating a migration chain against a schema it was never built from; keeping version = 5 would have made a v7 install refuse to open at all. The regenerated 8.json is two ADD COLUMNs and nothing else, same as before. fromGroupEventResult was restructured on mantra: the kind switch moved into applyInnerEvent, and a SubmissionEvent envelope now wraps nip30303 payloads. Took that structure and re-applied the direct message branch ahead of it rather than inside it -- a gift wrap is not a nip30303 payload to apply, and what happens to it depends only on whether this device's key opens it, so it does not belong in a function about applying submissions. The isUserMessage fix was re-applied to the eight call sites mantra's version has, up from the six it had here. ChatMessageListViewModel and ChatRoomMessagingScreen took mantra's versions with the composer state, the two renderings and the reply action layered back on. docs/README.md keeps both new rows and mantra's closing note about the skipped-keys document. 108 tests pass, up from 50 here and 83 on mantra. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
a909108300 |
feat: announce which key a room signs with, instead of rederiving it
A signer holds a different secret share under every ceremony it took part in, and signing with the wrong one produces a partial signature that cannot aggregate. Nothing said which was which: FrostSigningManager found a room's key by walking every ceremony this device holds a share for and rederiving each one's room id until one matched. That search can only find rooms derived at the one path the constant names. SharedKeyDerivation.parsePath was written to lift that limit and was never called, so a room derived anywhere else was invisible to signing. So the coordinator now says it. GroupKeyStateEvent (kind 30326) carries the threshold public key, the ceremony that made it and the path the room's id came from, posted into the room as its first application message and filed as a GroupKeyState row. completedKey reads that row first and follows it to the share. Nothing secret travels. Every member of the room can read the event, so a share on it would be each member holding everyone else's -- a 1-of-n key wearing a t-of-n's clothes. The event names the ceremony; the share stays in DkgSession.secretShare on the device that generated it. The coordinator is untrusted, as everywhere else in the ceremony, so a state is verified rather than believed: the room's id *is* the threshold key derived at the path, and one that does not rederive its own room is dropped. That is the same guarantee the rederivation gave, kept rather than traded for a lookup. The old scan stays behind it for rooms that predate the table. Announced after the members are added, which is the only order that works -- adding them commits a new epoch and MLS will not let a member read what was encrypted before the one they joined at. A member invited later still misses it and falls back to the scan, which is where every member was before this existed. Replacement is this app's job. These are rumors inside a Marmot group event, so no relay applies the 3xxxx rule, and the DAO keeps the newest announcement per room so a backfill cannot walk a room backwards. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
c8cbd936f1 |
docs: record where the sync's safety net is, and where it is not
Two updates after the test pass. long-running-sync.md gains a section naming what each test file pins and, more usefully, the three things they cannot reach: NostrSocketClientImpl's reconnect loop and ordered inbound (exercised only through their extracted arithmetic — covering them wants a fake WebSocketSession), everything downstream of saveNostrEvent (Room-backed, and there is no sqlite driver on the JVM test classpath), and the app on a device. The manual checks stay the manual checks. It also records that the tests were verified by mutation rather than by passing, so the next person knows the assertions were confirmed to bite. dead-code.md's line references are refreshed — the testability seams shifted most of them — and it now says which commit they were correct at and to confirm with the grep rather than trusting them. One entry added: the DefaultNostrSocketClientFactory overload taking an explicit HttpClient has no caller now that everything goes through the interface method. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
f5eb744ca7 |
test: cover the long-running sync, and open the seams needed to do it
The six commits that built the live chat sync added no tests. Everything they
touch fails silently by nature — a filter that drops messages, a subscription
that stops being replayed, a group whose id never reaches the `#h` tag — so the
symptom is always "some messages didn't arrive", days later, on someone else's
phone. 46 tests, in four files.
**What is covered**
RelayPoolSubscriptionTest (13) — the pool's half of surviving a dropped socket.
A query is retained and replayed on reconnect; a closed one is forgotten and
stops the socket reconnecting for it; closing one of two leaves the other alone;
a negentropy exchange is never replayed (its rounds are stateful, so resuming
one reconciles against a conversation the relay is no longer having); an update
to a live subscription replaces what gets replayed, including when the send
itself fails; dropping a relay or closing the pool forgets what they carried;
replay is scoped to the relay that reconnected. Plus the semantic the whole
change rests on, asserted in both directions: a live subscription keeps
delivering after EOSE, a one-shot query still ends at it.
LiveSubscriptionReconcileTest (12) — the requirement this all exists for: the
group filter follows group membership with nobody calling a subscribe function.
Joining widens the filter *in place* rather than reopening (a reopen would drop
the live tail of every other group in that chunk); leaving drops one; leaving
everything closes the subscription; churn inside the debounce window collapses
to one update; a NIP-17 room never becomes a group subscription. Then the
collect loop: events stored against the relay they came from, an event after
EOSE still stored, a CLOSED reopened once the back-off elapses and not before,
and a rate-limited CLOSED waiting far longer — but still coming back.
Backgrounding closes and foregrounding rebuilds, reconnects, and queues the
catch-up.
LiveSubscriptionPlanTest (11) — the filter and planning rules, led by the one
most likely to be "tidied up" later: the gift wrap filter carries no `since`,
because NIP-59 randomizes created_at into the past and a `since` near the
present silently drops new messages.
RelayBackPressureTest (4) and ReconnectBackoffTest (6) — the two pure decisions.
Which CLOSED reasons mean "ease off", and the backoff arithmetic including the
exponent clamp: 2.0.pow(4000) is Infinity and Duration * Double throws on it, so
without it a socket failing long enough turned its reconnect loop into a crash
loop, at the point the network was least likely to recover unaided.
**Seams opened to get there**, each a readability win on its own terms:
- NostrSocketClientFactory becomes an interface with DefaultNostrSocketClientFactory
behind it, so the pool can be driven by a fake socket.
- RelayPool takes its CoroutineScope, so the replay a reconnect triggers can be
observed rather than raced.
- LiveSubscriptionManager depends on a new LiveSubscriptionTransport (4
methods) rather than RelaysSocketManager, which observes the active wallet in
its init and cannot be stood up in a test at all.
- Its pure planning helpers move to the companion as `internal`, and its
launches inherit the caller's dispatcher instead of pinning Dispatchers.IO.
SynchronizationViewModel already launches observe() on IO, so nothing moves —
but a coroutine that picks its own dispatcher cannot be driven by a test
scheduler.
- reconnectDelay is extracted to ReconnectBackoff.kt with jitter as a
parameter, so the arithmetic can be pinned without randomness.
- endsLiveSubscription names the live-subscription termination rule next to
isTerminalFor, which is the one-shot rule. Having both named makes the
difference between them reviewable rather than implicit.
kotlinx-coroutines-test is added to commonTest: the pool's bookkeeping is all
suspend functions and there is no runBlocking in a common source set.
The tests were checked by mutation, not just by passing — reintroducing a
`since`, making EOSE terminal, dropping the leftGroupAt filter and removing
retention from query() each produce failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
0319f1613b | Merge branch 'mantra' into claude/nostr-event-save-issue-6e9467 | ||
|
Kgothatso Ngako
|
5321e4af72 | Merge branch 'mantra' into claude/distracted-franklin-e95ba4 | ||
|
Kgothatso Ngako
|
fb21678813 |
test: pin where a commit's bytes land when the row recording it is written
The mis-routed `framedCommitBytes` fixed in the previous commit was invisible for
one reason: nothing anywhere covered the persisted row. The bytes that reach a
relay come off the in-memory `CommitResult`, so the wire path stayed correct and
the stored path was wrong, and no test looked at the stored path.
## Why the mapping moved before it could be tested
A test that built `MarmotCommitResult` itself would have been writing its own copy
of the mapping and asserting against that. It would have passed against the buggy
code, because the bug was at the call site the test was not using.
So the mapping is now `MarmotCommitResult.from`, called by
`MarmotOutboundDao.inviteMember` and exercised directly by the test. That also
removes the shape that produced the bug rather than just the instance of it: the
old call site listed its named arguments in an order different from the
declaration, which is what put `preCommitExporterSecret` and `framedCommitBytes`
two lines apart. `from` lists the payload in declaration order, in one place, so
there is no second site to get wrong.
## What is covered
Four tests, each payload given a distinct self-identifying value so that a field
arriving in the wrong column names both halves of the mistake instead of comparing
equal by accident:
- every payload field lands in its own column.
- the framed commit column never holds the exporter secret -- the regression,
stated as an invariant rather than an equality so it keeps holding for a
`CommitResult` this test did not anticipate.
- a `CommitResult` that never framed its commit still stores a commit. quartz
defaults `framedCommitBytes` to `commitBytes` and the entity repeats that
default; the fallback must not quietly become the secret either.
- the bookkeeping `DatabaseNostrRepository` reads back on acknowledgement is
carried through. `id`, `chatRoomId`, `userPublicKey` and
`peerKeyPackageEventId` are all 64-char hex, so two of them swapped in `from`
would typecheck exactly as silently as the original bug.
Checked by reintroducing `framedCommitBytes = commitResult.preCommitExporterSecret`
into `from`: three of the four fail. A green suite that would stay green against
the bug it names is not coverage.
## What is not covered, and why
That the bytes published equal the bytes stored -- the property one level above
this one -- still is not. It needs the DAO, and the DAO needs Room: `commonTest`
carries only `kotlin.test`, the room3 KSP processor is registered for the android
and ios targets alone with `kspJvm` commented out, and `getInMemoryDatabaseBuilder`
wants a `PlatformContext` no unit test has. That is a Robolectric or instrumented
target, which is a larger change than this fix earns and is better decided on its
own merits than smuggled in here.
The ack-triggered rebroadcast that would have turned the bug into a live fault does
not exist yet, so there is nothing to test there either. When it is written, the
invariant it needs is already asserted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
ad3304a665 |
refactor: build the DM inbox filter once, where it can be asserted
The filter fix a commit ago changed a value inline in a ViewModel, which is
not a place a test can reach: ChatMessageListViewModel needs a repository and
a coroutine scope to construct, and NostrDao needs Room. So the filter that
had just been wrong in three call sites went back to having no coverage at
all.
Nip17Filters.inbox is that filter with one definition. ChatMessageListViewModel
and ChatRoomListViewModel now both call it — they had been building it
separately and identically, which is also what made their negentropy requests
collapse into one under computeId, a coincidence better expressed as shared
code than left to hold by luck.
Nip17FiltersTest asserts every clause that was got wrong in production:
- the p tag names us, not a peer
- there is no authors clause, because a wrap is signed by the throwaway key
GiftWrapEvent.create mints and discards, so authors=[anything knowable]
matches nothing on any relay
- there is no since cursor, because NIP-59 back-dates a wrap by up to two
days and a high-water mark taken from the newest wrap we hold skips mail
stamped behind it — the trap waiting for whoever acts on the TODO in
NegentropySynchronizeRequest.toSynchronizeNostrEventRequest
- the wire JSON is pinned, so an added default cannot quietly split the two
callers back into separate requests
- the SQL NostrEventFilterQuery builds from it bounds no author either,
since negentropy is only as good as the agreement between the set we build
locally and the set the relay builds from the same filter
Neither of the two failure modes this covers was visible from reading the
filter. The authors clause failed silently for as long as it existed, and the
peer p-tag failed loudly but somewhere else entirely — in a Room transaction,
three files away, as a MAC error out of Nip44.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
e1d35bbd6c |
test: pin who can open a gift wrap, and what happens to everyone else's
The Invalid Mac crash had no test standing between it and a repeat, so this adds one that reproduces it. GiftWrapMessageTest builds real NIP-59 wraps with real secp256k1 rather than recorded fixtures. The property under test is the key agreement itself — whether ECDH(ourPriv, ephemeralPub) can stand in for the conversation key the wrap was sealed under — and a fixture would only prove that the fixture still parses. Three cases carry the regression: - someone else's mail comes back null rather than throwing - not even the sender can reopen what they sent - isAddressedTo answers exactly what unsealing would Checked against the reverted fix, those three fail with the production exception verbatim (java.lang.IllegalStateException: Invalid Mac: Calculated bf2e6480…), while the two describing behaviour that never broke — the happy path, and isAddressedTo's reading of the p tag — stay green. A test that cannot fail against the bug it names is not worth the run time, so the split matters. The last of the three is the one guarding the fix's structure rather than its outcome. NostrDao decides whether to index on isAddressedTo, then throws GiftWrapUnsealException if decryptGiftWrapSeal returns null anyway; those two answers have to agree for either path to be correct. If they drift, the DAO either skips mail we can open or resumes rolling back transactions, and neither shows up as a failure anywhere near the change that caused it. commonTest gains kotlinx-coroutines-test for runTest. decryptGiftWrapSeal is suspending, runBlocking does not exist in common code, and every layer worth testing below the ViewModels — DAOs, repositories, the model's crypto — is suspending too, so the dependency pays for more than this file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
42dd38cfc4 |
test: pin the two invariants this session left unguarded
Both are silent when broken, which is why they are worth asserting rather than reasoning about. **The cache's reuse decision.** MlsGroupCache exists because quartz drops a secret tree's skipped-generation keys on save, so rebuilding a group between two messages loses any that arrives late. Its safety argument is one comparison: reuse while the stored state is still what the cache last wrote, rebuild when it is not. Get that wrong in either direction and nothing complains -- reuse too eagerly and a group carries on from a ratchet another writer already moved, which corrupts decryption rather than failing it; reuse too rarely and the cache does nothing and the original bug is back with no symptom. That decision is now a generic LiveInstanceCache with MlsGroupCache as a typed facade over it, so it can be tested without standing up an MLS group. Splitting it also made two behaviours explicit that were previously incidental: a failed build no longer leaves the old instance behind, and an instance whose use threw is deliberately not cached -- it is half-advanced and never persisted, so the next caller has to start from disk. **Rumor and row ids agreeing.** MantraDao writes an entity whose id comes from fromXEventTemplate and separately builds the rumor it submits with rumorOf, which hashes the template itself. Both are meant to produce one id and nothing checked it. Diverging would mean submissions naming an event nobody has, deleteByPayloadEventId silently un-queuing nothing so superseded translations go out anyway, and every receiver creating a second row instead of converging on the sender's -- all of it invisible, since the ids are opaque hex either way. Asserted per kind, plus the whole chain out through the submission envelope. Both suites were mutation-checked rather than trusted: inverting the staleness comparison fails one cache test, recording the pre-block state fails another, and hashing the rumor under a different author fails all six id tests. Still uncovered, and not cheaply fixable: FrostSigningManager's and MantraDao's state machines both need a Room harness, and commonTest has none. The FROST crypto path is covered by FrostSigningRoundTest; the message-driven parts around it are not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
c24cbed390 |
docs: record what the coverage work found, and what it left uncovered
Three additions. The decision the inbound path makes now has a name and a home -- MarmotDirectMessage.classify -- and the doc says why it is separate from the filing of it: only the filing needs a database, so splitting them is what lets the check that replaces MIP-03 be tested at all. A security property found while writing those tests, which I had asserted backwards. Relabelling a seal with another member's pubkey does not get as far as the signature check: NIP-44 derives the conversation key from the pubkey being claimed, so a relabelled seal is undecryptable by the person it was encrypted for. The label is bound to the key rather than asserted alongside it, and the outcome is a message the recipient genuinely cannot read. verify() catches the narrower case of a seal altered after signing in a way that survives decryption. An honest list of what has no automated test and why -- the recipient validation and the outbound id lookup (both need a database), the two transcript renderings (no Compose UI test dependency in this project), and anything touching a real MlsGroup. Better written down than rediscovered by someone assuming a green suite means the path is covered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
a74a4b71cf |
test: cover the two decisions that decide who said what
The crypto was tested; the logic that acts on it was not. Both untested pieces were the security-critical ones, and neither fails loudly when it goes wrong -- one silently widens who may impersonate whom, the other silently destroys a message. Extracted MarmotDirectMessage.classify, which decides what an arriving wrap is to this device, from ChatMessage.directMessage, which turns that decision into rows. The decision is pure; only the filing needs a database, and Room-backed code cannot be unit-tested in this project. Same split, and for the same reason, as pulling the wrap/open crypto out of the DAO in the first place. Extracted MarmotInboundManager.mip03Rejection for the same reason. Its kind:1059 exemption is the most dangerous line in this feature: widened to another kind, or stripped of its kind guard, it hands every member of every group the ability to publish events as anybody, and nothing else in the pipeline would notice. There is now a test that walks seven kinds and asserts each is still held to MIP-03. Fifteen cases, the ones worth naming: `our own message is ours, even though we cannot open it` and `ours is decided before anything is opened`. A sender cannot decrypt their own wrap -- the key was discarded -- so by decryption alone this is indistinguishable from a bystander's view, and only the MLS identity separates them. Get it wrong and the inbound path files an empty placeholder over the row sendChatMessage wrote, which holds the only copy of those words. It is the one failure here that loses data rather than rendering something wrong. `words sealed by one member and sent by another are dropped`. The check that replaces MIP-03 for this kind, tested directly rather than described in a comment as it was before. One test asserts something I had wrong. I expected a seal relabelled with another member's pubkey to be caught by the signature check; it never reaches it. NIP-44 derives the conversation key from the pubkey being claimed, so relabelling a seal makes it undecryptable by the person it was encrypted for -- the label is bound to the key, not merely asserted alongside it. The outcome is Unreadable, which is the truth: the recipient genuinely cannot read it. `a seal tampered with after signing is dropped` covers what verify() does catch, using an alteration that survives decryption. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
d110737f9a |
fix: keep a room's MlsGroup alive so a late message can still be read
Two events published in the same second reliably lose one of them. The receiver stores the kind:445 and produces nothing from it -- no inner event, no chat line, no error anybody sees, because MarmotGroupEvent is written before the message is decrypted and so survives while everything downstream silently does not. Observed as a FROST signing session that never started on the receiver: proposeSigning publishes the proposal and then the proposer's own nonce, the relay handed them back in the other order, and the proposal was dropped. The nonce is still sitting there filed against a session that will never exist. The same bug ate a dialect earlier, which then took out the artifact referencing it via a foreign key. MLS is specified to tolerate this. RFC 9420 says a receiver that gets generation N+1 before N keeps the intermediate keys so the older message can still be read, and quartz's SecretTree does exactly that, in a private skippedKeys map. What it does not do is persist it: exportSenderStates() returns the ratchet positions only, so saveState() drops the cache. NostrDao rebuilt the group from stored state for every inbound event, so the cache was empty every single time, and generation N arriving after N+1 failed `require(generation >= applicationGeneration)` and was swallowed. Terminal -- the key is derived from a ratchet that has moved past it, and nothing asks the sender to resend. This keeps the instance alive instead. MlsGroupCache holds one MlsGroup per room, and the inbound path goes through it, so skippedKeys survives from one message to the next. That covers the case that actually bites -- a burst arriving in one sync, decrypted one after another against the same tree -- which is what every bursty flow needs: proposeRitual sends two, addArtifact sends two, and addChapter sends one per paragraph plus one, of which only the ones arriving in ascending generation order survived. Reuse is conditional on the stored state still being exactly what the cache last wrote. Sending a message advances the sender ratchet and saves; so does adding a member. When that happens the cache rebuilds rather than carrying on from a group that has been overtaken -- which is what keeps this from being worse than no cache at all: the fallback is always the old behaviour, never a diverged ratchet. One lock per room, not one overall, because the group is mutable and decryption advances it: two events for the same room decrypted at once would corrupt the tree, and a busy room should not hold up a quiet one. **This is a mitigation, not the fix.** It does not survive a restart, and it does not survive another writer, so a long enough reorder still loses the message. The fix belongs in quartz -- carry skippedKeys through saveState/restore -- and quartz is a mavenCentral binary, not a fork, so it cannot be made here. docs/mls-skipped-keys.md has the analysis, the patch, the migration constraint on the persisted state format, and the three ways to actually land it. Not verified end to end: the proposal that exposed this cannot be recovered, since its generation is already past, so confirming the fix needs a fresh burst. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
f38a5f12f3 |
fix: ask relays for gift wraps addressed to us, not to our peers
Three kind:1059 sync filters named the wrong pubkey. ChatMessageListViewModel asked for `#p:[peer]` with no author constraint, which subscribes to every wrap anyone has ever sent that peer. None of it is decryptable by us, and it is the direct source of the Invalid Mac saves fixed in the previous commit. It now asks for `#p:[us]` on our own DM relays — the only shape of gift wrap filter that can return something we hold a key for. The peer's relays were the wrong place to look regardless: under NIP-17 a sender publishes to the *recipient's* DM relays, so our mail lands on ours. The two in NostrDao asked for `authors:[userPublicKey]` + `#p:[participant]`, commented "messages from this relay that were sent by us". A gift wrap is signed by the throwaway key from GiftWrapEvent.create, never by the sender's identity key, so no author value we could know will ever match one. These requests were queued once per participant and always reconciled to empty — failing silently rather than loudly, which is why they outlived the bug that made the third filter visible. Both `if (chatMessageRelayListEvent != null)` branches held nothing else, so each is inverted to the `== null` case that does the real work: warn, and queue a profile sync for the participant whose DM relay list we are missing. Nothing is lost; neither filter ever returned an event. Two things worth recording about what a filter can and cannot express here. A wrap discloses only its recipient, so "the messages in this conversation" is not askable — `#p:[us]` pulls the whole inbox and that is the narrowest correct request. That is the privacy property being paid for, not a limitation to work around. Sent-message recovery is likewise not a filter problem. It needs a second wrap addressed to ourselves at send time, which giftWrapAndBroadcast does not yet emit; the `#p:[us]` filters already in place would pick those up with no new subscription. purpose on the chat message request changes from "sent-messages" to "chat", matching the now-identical filter in ChatRoomListViewModel. Since computeId buckets by minute and NegentropySynchronizeRequestDao upserts, the two collapse into a single request rather than racing as separate rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
f57644aa1f |
fix: stop discarding gift wraps addressed to someone else
An inbound kind:1059 whose `p` tag is not our pubkey took down the entire
save transaction:
java.lang.IllegalStateException: Invalid Mac: Calculated f1db537e…, decoded: 45c8c86a…
at com.vitorpamplona.quartz.nip44Encryption.crypto.Hkdf.fastExpand
at com.vitorpamplona.quartz.nip44Encryption.Nip44v2.checkMessageKeys
…
at press.mantra.compose.database.model.GiftWrapMessage.decryptGiftWrapSeal
at press.mantra.compose.database.dao.NostrDao.indexNostrEvent
at press.mantra.compose.database.dao.NostrDao.storeNostrEvent
Two separate things were wrong.
The first is that decryptGiftWrapSeal attempted the decryption at all. When
the recipient did not match our key it logged "We are unwrapping a message we
may have sent" and called
Nip44.decrypt(content, privateKey = ourPrivKey, pubKey = giftWrapEvent.pubKey)
giftWrapEvent.pubKey is the wrap's ephemeral author. NIP-59 encrypts the wrap
under ECDH(ephemeralPriv, recipientPub), and GiftWrapEvent.create mints that
ephemeral key with NostrSignerSync(KeyPair()) and discards it on return.
ECDH(ourPriv, ephemeralPub) is a third, unrelated key, so the MAC check could
never pass. A sender genuinely cannot unwrap their own gift wrap; that is the
point of the construction, not a gap in it. The call threw its result away
anyway (keyPair.privKey?.let { …; null }) and fell through to the trailing
`return null`, so it was a probe whose only possible outcome was an exception.
The second is that a null seal was treated as a failure. indexNostrEvent
throws GiftWrapUnsealException on null, which unwinds out of the Room
transaction in storeNostrEvent and rolls back everything written for the
event: the NostrEvent row, its NostrEventRelay row, and the GiftWrapMessage
upserted moments earlier. The only catch sits in DatabaseNostrRepository,
which logs and continues — and that catch also swallows the
`status = "processed"` upsert on the SynchronizeNostrEventRequest, so the
event was re-fetched and re-failed on every later sync pass.
isAddressedTo now answers the question with no crypto at all, and the indexer
returns early for wraps that are not ours: the event and the wrap row survive,
the remainder of indexNostrEvent still runs, the transaction commits, and the
sync request is marked processed. GiftWrapUnsealException goes back to meaning
what it says — addressed to us, but unsealing failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
3e4166f13d |
feat: sign a dialect into existence instead of submitting one
Adding a dialect no longer creates one. It opens a signing session over a DialectEvent, and the dialect appears -- on every member's device at once, authored by the group's shared key rather than by whoever typed it -- when enough members have signed. That is the difference between the two envelopes. A submission says "I am putting this in front of the group"; the group's only recourse afterwards is social, and the row records the submitter as its author. A signature is the group saying it, it takes a quorum to say, and the author on the row is the group's key. For something as load-bearing as the set of dialects a group translates into, the second is the honest one. **Where the signed event becomes a row.** Every device has the event and the signature once the session completes, so each applies the result itself rather than waiting to be sent something it can already build -- the same reasoning the transcript lines are written on. Nothing goes on the wire for it, and nothing could: the outbound pipeline re-authors rumors as their sender, so a group-signed event pushed through it would come out stripped of the signature and attributed to whoever sent it. Applying reuses the inbound path's dispatch rather than repeating it. applyInnerEvent takes plain ids now instead of a GroupEvent, and both are null here, because there is no group event and no inner event behind a row a device derived for itself. A failure there is logged and the session still completes: the signature is made and valid, and failing the session would tell the group to abandon something that succeeded. **The screen.** One, not three. A ceremony asks three different questions so it gets three approval screens; signing asks one -- sign this or do not -- so a single screen has to carry the whole case: what is being signed, who else has agreed, and what the group is still waiting on. The event is shown as the thing it is, a dialect with its name and country and language, because a member deciding whether to sign is deciding about a dialect and "kind 30304" answers a question nobody asked. Anything unrecognised falls back to the raw kind, which is better than describing it wrongly. The member ladder names people rather than counting them, for the same reason the ceremony's does: "1 of 2" does not tell anyone whose door to knock on. It stays useful after the decision, since a member who has already signed is exactly who needs to see who has not. **Getting there.** Signing lines render in the transcript as system notices like ritual lines -- nobody said them either -- but they lead to the session rather than to the key. A chat row carries no session id and adding a column to the table every message uses would be a poor trade for a lookup, so FrostSigningRoute takes a nullable id and the screen resolves the room's live session. Approving is recorded as answered by the nonce line rather than the partial signature: agreeing is agreeing to take part, and the coordinator may then pick a quorum without you, which should not leave you looking like you never replied. **Proposing needs a key.** The FAB is disabled, and says why, when the room has none -- proposeSigning throws there, and it is not reachable outside the #admins room in the first place. AddDialectViewModel drops MantraRepository, which it no longer uses for anything. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
63c1879ace |
refactor: carry signing on marmot inner events, not gift wraps
A signing message is now an ordinary Marmot inner event: queued with a
null marmotGroupEventId, picked up by the outbound pipeline, MLS-encrypted
and broadcast as one kind:445 for the room. Inbound it arrives through
ChatMessage.fromGroupEventResult like every other inner event, and is
dispatched from NostrDao rather than from the gift-wrap branch.
The ceremony keeps NIP-17 because it has no choice: its participants are
not yet a Marmot group, and its purpose is to produce the key one would
be keyed on. Signing has that solved for it, so it was paying for
addressing it does not need -- a gift wrap is sealed once per recipient,
so every message cost one wrap per member, and every message had to name
the whole group in p-tags. A group event is encrypted to the group once.
That also removes a small dishonesty. The signer set is supposed to come
from the ceremony; carrying p-tags meant each message also asserted a
membership list, and two sources for one fact is one too many. Now who
can read a message is the MLS tree's business and who may sign is the
ceremony's.
Which room follows from the transport. A ceremony runs in a NIP-17 room
-- every member an equal admin, no MLS tree to be outside of -- and a
group event needs an MLS one, so signing cannot happen where the ceremony
did. It happens in the #admins room, which is the right venue anyway: it
already exists after a ceremony, its membership is exactly the share
holders, and its id *is* the key, derived by
SharedKeyDerivation.marmotGroupId.
So completedKey rederives rather than reading a column: a room cannot be
pointed at a key it was not derived from. Receivers were already
independent of this, naming their key in the proposal's frost_key tag and
looking it up locally.
Mechanical consequences:
- processSigningPayload, acceptProposal, record and isFromCoordinator
take the decrypted Event instead of a GiftWrapPayload.
- replayStoredMessages reads MarmotInnerEvent rows, via a new
getByChatRoomAndKinds, and rebuilds the rumor from the row's own
columns.
- applyInnerEvent returns null for the signing kinds. They are the
manager's, and it writes transcript lines naming who did what, so an
"unsupported" row would be a second and worse account of the same
thing.
- DkgSessionDao gains getKeyHoldingSessions for the derivation match.
The kind comment is rewritten rather than kept. 3032x was chosen to clear
the DKG, which now shares no transport with signing and cannot clash with
it; what it actually has to clear is the nip30303 document kinds, which
run 30300-30312 and are dispatched by the same inbound path. It still
does. The DKG's own overlap with those numbers is noted there as the
routing accident it is, so nothing added later leans on it.
No schema change: both tables and the columns landed in v6 with the
previous commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
b4ac65f5c9 |
feat: sign a nostr event with the group's shared key
A ceremony leaves every member holding a share of a t-of-n key and no way
to use it. This is the other half: a session that turns an unsigned nostr
event into one signed by the group.
The shape is ChillDkgRitualManager's, deliberately. The member who
proposes coordinates, protocol messages travel as gift-wrapped rumors on
the same NIP-17 pipeline chat messages use, each inbound message is
persisted and then the session is asked whether it can move, and every
step is recomputed from stored inputs so a device killed mid-round
resumes on the next message. Anyone who has read that manager can read
this one.
proposer --[ 30320 proposal ]-> everyone the unsigned event
signer --[ 30321 nonce ]-> everyone this device's public nonce
proposer --[ 30322 signer set ]-> everyone who signs, and their aggregated nonce
signer --[ 30323 partial ]-> everyone this device's partial signature
proposer --[ 30324 signature ]-> everyone the finished 64-byte signature
anyone --[ 30325 failure ]-> everyone abandon + blame
Three things are genuinely different, and each is why this is a separate
manager rather than another branch of that one.
**It does not need everybody.** A DKG cannot finish until every member
takes part; that is what makes the key. Signing needs t, and waiting for
n would throw away the property the group ran a ceremony to get. So the
coordinator waits for the threshold to be reachable, picks a set and says
who is in it. Members left out do nothing and stall nothing.
**Restart-safety is forced rather than chosen.** SecretNonce cannot be
serialised and refuses to be used twice, so storing the randomness it
derives from and regenerating on demand is the only way a session
survives the app closing. That is safe for exactly one reason: a session
signs one message and cannot be made to sign another. Two rules hold it
in place and both are load-bearing rather than tidy:
- the event id is written at creation, and a proposal that disagrees
with it is refused rather than applied;
- the aggregated nonce and signer set are write-once. A coordinator
that sends a second, different set is ignored. Obeying it would mean
two partial signatures over one secret nonce against two challenges,
which is precisely how a secret share is extracted. The session
stalls; the share does not.
**One approval, not three.** A DKG asks three times because each step
publishes something different and commits the member to something
different. Here every step serves one decision -- sign this event or do
not -- and the event is fixed before the member is asked, so a second
prompt would be the same question twice. Declining is broadcast rather
than silent: a t-of-n group can sign without you, but only if it knows.
Two things are checked rather than trusted, both because the coordinator
is untrusted by construction: the event id is recomputed from the
proposal's own fields, so a proposer cannot have the group sign one thing
while showing them another; and the finished signature is verified before
the session is called complete, so a bad aggregate is a failure here
rather than a rejection at every relay it reaches.
Signer ids are derived, not stored: a member's FROST id is their index in
the bytewise sort of the ceremony's host keys, the same ordering ChillDKG
hashed into the session identity and the same one the public shares are
in. Deriving means signing cannot disagree with the ceremony that made
the key.
DkgSession gains publicShares, kept because FROST validates each signer's
secret share against its public one. A ceremony finished before this
column reads back null and signing runs without that check rather than
refusing.
The tests run the same calls in the same order against real FROST and
assert the aggregate verifies as a nostr signature. That path was written
from reading the library rather than from a working example, so it is the
part most likely to be subtly wrong -- and wired up wrong it fails
silently, on every device.
Kinds start at 30320 with a gap. The DKG runs 30310-30316 and the
nip30303 document kinds run 30300 up; those two already collide at 30310
and 30311, and SubmissionEvent sits on 30312, which is also the DKG's
round-1 kind. They are kept apart today only by riding different
transports, which is luck. Signing shares a transport and rooms with the
DKG, so it starts clear of both.
No UI yet: this is the session logic, reachable through proposeSigning,
approve and decline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
fcc28de931 |
Revert "fix: hold a payload whose parent has not arrived instead of losing the event"
This reverts commit
|
||
|
Kgothatso Ngako
|
d7aac49cf1 |
fix: hold a payload whose parent has not arrived instead of losing the event
A receiver hit `FOREIGN KEY constraint failed` on an artifact submission and lost the whole group event. The artifact referenced a dialect the receiver did not have, MantraArtifact.dialectId is a foreign key, and SQLite answers a violated constraint by aborting -- which rolled back the entire transaction the inbound pipeline runs in. Gone with it: the NostrEvent, the MarmotGroupEvent, the submission's MarmotInnerEvent holding the payload verbatim, and the transcript line. Nothing retries, so the artifact stayed lost even once the dialect turned up. Every nip30303 entity is a child of another and the schema enforces all of it -- artifact→dialect, version→artifact, chapter→version, chunk→chapter, translations→both of theirs -- so this was every branch, not one. And submissions make arriving before your parent ordinary rather than exotic. That is the point of them: an admin submits a backlog in whatever order they hold it, and a member who joined last week can be sent what the group was told last month. Both produce payloads whose parents are not here yet, and both were losing data. So check the parents before inserting. A payload that arrives early is held on the submission row -- awaitingEventId names what it waits for -- and applied when that arrives. Releasing one can release another, a version freeing its chapters and those freeing their chunks, so it walks outward until nothing more comes unstuck. A payload with a second parent still missing is re-pointed at that one rather than retried on every arrival. Nothing is written to the transcript while a payload is held. Nobody has said anything yet; the line appears when it is applied, in the position its own timestamp gives it. Two things fall out of the shape: parentRefsOf is pure and separate from the lookups, because the mapping is the part that can silently drift from the schema and there is no database harness in commonTest to catch it. ParentRefsTest pins one case per kind. Which table an id lives in is carried as the kind of event that would have created it, so there is no second enum to keep in step. applyInnerEvent takes ids rather than a GroupEvent, since replay happens long after that object is gone. A released payload is recorded as not ours: we hold the parents of anything we wrote, having written those too. Also reconstructs a held bare nip30303 event from its own columns rather than parsing its content as an event -- only submissions carry an event there, and reading both that way would have stranded every bare one permanently. Verified: the v5→v6 migration runs clean on the receiver's real populated database. The hold path itself still needs a fresh submission from a sender to exercise end to end. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
2d0fe6f5fc |
fix: disable Add Artifact until a dialect is picked
Submitting without a dialect was rejected in the view model, which
called onFailure, which navigated to ImplementationPendingRoute("Failed
Artifact") -- a whole screen away from the form, saying nothing about
which field was wrong, and leaving the way back to the only sensible fix
as the back button.
That is a bad way to report any missing field, but the dialect is the
one where it is unrecoverable in place. A blank name or url is answered
by typing; a dialect has to already exist, and since dialects moved to
the group screen there is nothing on this form that can conjure one. So
an unpicked dialect is not a mistake to report after the fact, it is a
state the button should not be pressable in.
Material 3 gives ExtendedFloatingActionButton no `enabled` parameter, so
this paints the disabled colours from ButtonDefaults.buttonColors() --
the same ones every other disabled button in the app resolves from the
theme, rather than an alpha invented here -- and returns early from
onClick.
Also marks it disabled to accessibility services. Colours alone leave a
screen reader announcing a button it is happy to press, and pressing it
does nothing, which is worse than a button that says it is unavailable.
A group with no dialects at all is covered by the same condition, since
there is then nothing to select.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
6c63027912 |
feat: submit nip30303 events to the group instead of authoring them into it
With the receiving side able to open an envelope, start sending one.
Every nip30303 event now leaves as a SubmissionEvent payload and none
leaves on its own: addDialect, addArtifact, addArtifactVersion,
addChapter and each of its chunks, addTranslationArtifactVersion and
each of its translation chapters, and saveTranslation. Because all four
add screens reach the wire through MantraDao, none of them needed
touching.
Two helpers carry it:
rumorOf(template, publicKey) the unsigned event a template describes.
Its id is computed exactly as the
matching Mantra* entity computes its own,
so the row on disk and the payload on the
wire are one event rather than two copies
of one.
submitToGroup(...) wraps a payload, queues the submission as
an unprocessed rumor, and writes the chat
line.
Two things this drags in, neither optional:
The queued MarmotInnerEvent is now the envelope, so its id is the
envelope's and no longer the nip30303 event's. saveTranslation replaces
a chunk whenever its text changes -- the id is derived from the content,
so an edit is a new row -- and un-queued the superseded one by
deleteById(stale.id). That silently stops matching anything once the row
is a submission, leaving the stale translation to be sent anyway. It now
also deletes by what the submission carries, via deleteByPayloadEventId.
The add* methods return the entity rather than the queued rumor. This is
a correctness fix, not tidying: AddArtifactViewModel navigates to
ArtifactDetailRoute on that id, and AddTranslationArtifactVersionViewModel
feeds addDialect's id straight back in as a dialectId. Both used to be
handed a MarmotInnerEvent whose id happened to equal the entity's, and
both would now have been handed a submission id -- one navigating to an
artifact that does not exist, the other tagging a translation with a
dialect that does not. Returning MantraArtifact/MantraDialect/etc. makes
.id mean the entity everywhere and matches saveTranslationChunk, which
already returned its entity.
The sendMarmotInnerEvent overload taking a LocalChatRoom loses its last
caller; submitToGroup names the submitter explicitly, which is the thing
that matters now that it is not necessarily the author.
Outbound still only ever submits payloads authored by the submitter --
nothing in the app originates a foreign event yet. submitToGroup is
where that would attach.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
ce77b77240 |
feat: apply the nip30303 event a submission carries, keeping its author
Teach the receiving side to open an envelope before anything starts
sending one. In that order a client that has this can already handle
submissions from a client that does not yet send them; the reverse would
turn every artifact, dialect and chapter into an "unsupported" row for
anyone who had not updated.
Despite the name, MarmotInboundManager does not dispatch on inner-event
kinds -- it decrypts MLS and hands back a GroupEventResult. The kind
dispatch has always lived in ChatMessage.fromGroupEventResult, so that
is where support for a new kind goes.
The `when (event.kind)` body becomes applyInnerEvent, which takes the
event to apply separately from how it arrived:
event the nip30303 event, written by whoever wrote
it -- possibly nobody in this group
marmotInnerEventId the row the group actually delivered
senderPublicKey the member who delivered it
createdAt when they did
For a plain nip30303 event those all come from the one event, which is
exactly the old behaviour. For a submission they come from the envelope
while `event` is the payload. Entity rows take their author from the
payload via fromXEvent, so the chat line says who added something and
the row says who wrote it -- the point of the envelope, made real at the
only place it can be.
createdAt deliberately follows the envelope rather than the payload: a
submitted archive translation can be years old, and sorting the group's
transcript by when the source was written would file "X added a
translation" somewhere nobody will scroll to.
The stored MarmotInnerEvent stays the outer event -- that is what the
group sent -- and gains payloadEventId naming what it carries. The
payload is not given a row of its own: it is recoverable from the
submission's content, and a second row with a null marmotGroupEventId
would look to the outbound pipeline like something waiting to be sent.
Nullable column, so AutoMigration(4, 5) is all it needs; rumors queued
before this read back null, which is correct, since none of them were
submissions.
Two submissions are stored but not applied, because there is nothing in
them to make a row from: one whose payload will not parse, and one
carrying another submission. Both surface as "unsupported" rather than
disappearing.
The unsupported fallback also stops attributing to groupEvent.pubKey,
which is the ephemeral key every kind:445 is signed with and so names
nobody.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
fa380e94e1 |
feat: add a SubmissionEvent that carries a nip30303 event as its payload
Every nip30303 kind so far describes a thing: an artifact, a dialect, a
chapter, a translated chunk. None of them describes the act of putting
one in front of a group, and until now nothing needed to -- a group
event's sender was the author of the event inside it, so the two
questions had one answer by construction.
That construction is also the limit. It means a group can only ever hold
work written by its own members under their own keys. A translation
lifted from a public archive, a chapter transcribed by an outside
contributor, an artifact somebody published years ago: none of it can go
in without a member re-authoring it and taking the byline.
Kind 30312 is the envelope that separates them. Its content is the
payload event's JSON, whole -- same id, same pubKey, same signature,
nothing rewritten to look like the submitter's work. The submitter signs
for the envelope; the author still signs for the event. Two tags name
what is inside so a client can decide whether it can apply a submission
without parsing the content first:
payloadKind the payload's kind
payloadId the payload's id, with the author slot carrying the
payload's author -- who, unusually for an id tag in
this package, is often not the event's sender
Kinds 30300-30311 are taken (30305 and 30307 by contributor lists), so
30312 is the next free one.
A submission is not an endorsement and grants nothing. Who may submit is
the group's business; this only makes the question expressible.
The test covers the property the whole thing rests on: an event written
by an outsider goes into an envelope, comes out of a JSON round trip
with its id, author and signature intact, and does not acquire the
submitter as its author. It also pins payload() returning null rather
than something empty when the content will not parse -- which needed
android.util.Log stubbing, since quartz logs on that path and unmocked
Log methods throw, failing the test on the log line rather than on what
it came to check.
Nothing sends or reads one yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
e14be2d187 |
refactor: pick an artifact's dialect, do not invent one while adding it
Adding an artifact offered a "New dialect" chip that swapped in three more fields -- name, country, language -- and minted a dialect on the way to creating the artifact. Now that a group defines its dialects on its own screen, that path is a second, worse way to do the same thing: it creates a dialect as a side effect of an unrelated action, in a form where the fields belong to neither entity clearly, and with no sight of what the group has already defined beyond a row of chips. Drop it. The chip row is now exactly the dialects that exist, and selectedDialectId changes meaning from "null = create a new one" to "null = nothing picked yet" -- which the FAB rejects alongside the other required fields, rather than falling through to creating something. A group with no dialects yet gets a line saying so and pointing at the group screen, instead of a lone chip that opens a form. addArtifact loses the three TextFieldStates and existingDialectId for a single dialectId, and with them the branch that called addDialect and threaded its id back in. Validation is now one condition rather than one per path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
6ff9fd2d38 |
feat: let a group define the dialects it translates into
Dialects existed but had nowhere to come from. The only way to create
one was the "New dialect" branch buried inside the add-artifact form,
which meant a dialect could only be born as a side effect of adding the
first artifact written in it. A group that wanted to line up the
languages it works in before any source material arrived had no way to
say so, and a dialect created that way was invisible afterwards -- there
was no screen anywhere that listed what the group had defined.
Give the group detail screen a Dialects section between Library and
Projects: the dialects defined in this room, each showing its name over
"<language> · <country>", and an Add Dialect button. The cards do not
navigate -- there is no dialect detail screen to open, and a card that
goes nowhere is worse than one that plainly does not.
The add screen is the add-artifact form with the artifact half removed:
the same chat-room title bar, the same bottom bar with an extended FAB,
the same three fields (name, country, language) styled the same way.
On success it returns to the group with popUpTo<ChatRoomDetailRoute>
{inclusive = true}, replacing the stale detail screen beneath it so the
new dialect is actually in the list when you land -- these lists load
once, in the view model's initiate().
One deliberate difference from AddArtifactViewModel: it wraps its whole
body in `localChatRoom.chatRoom.toMlsGroup()?.let { ... }` and so does
nothing at all, silently, in a NIP-17 room. Nothing under
MantraRepository.addDialect needs an MLS group, so the gate is left out
rather than copied into a new screen as a button that does nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
635cef9311 |
docs: write down how a direct message travels, and what it costs
The reasoning behind this is not recoverable from the code, which is the bar docs/README.md sets for having a document at all. Three things in particular would otherwise have to be rediscovered by whoever changes this next, and two of them are traps. Why the wrap uses a throwaway key rather than the sender's own -- and what that does not buy. It does not hide the sender from the group: MLS authenticates every application message to a leaf, so the identity is there regardless. What it costs is a carve-out in MIP-03's pubkey check and the sender's ability to ever read their own messages back. Why the check that carve-out removes is not a hole. The authorship claim moves from the wrap's plaintext pubkey to the seal's verified signature, bound to the MLS leaf that sent it -- strictly harder to forge than what it replaced. The one query that would broadcast one of these. What this builds is a genuine, correctly signed NIP-59 gift wrap, indistinguishable from what the NIP-17 path would be right to publish, and the only thing keeping it off a relay is that it never becomes a GiftWrapPayload. Written against what shipped rather than what was planned, so it records two deviations. senderIdentity is resolved in NostrDao rather than added to GroupEventResult.ApplicationMessage, because quartz is a binary dependency here and the local checkout is a reference copy, not a build input. And a failed validation drops the message and logs rather than throwing, because the caller is inside storeNostrEvent's transaction. The unbuilt parts are listed as absences rather than left implied: there is no member picker, so a private message can only be a reply to one somebody already sent, and nothing in the UI yet tells a user in words that the group can see who they messaged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
b0d0199113 |
feat: let a member send and read a private message in the room
The UI half. Tapping somebody else's message in a group offers "Reply privately to <name>", which arms the composer; sending clears it. The riskiest thing about this feature is not cryptographic. It is somebody sending to the room what they meant for one person, or the reverse, and neither can be taken back once it is on the wire. So an armed composer carries three signals at once -- a chip naming the recipient, a placeholder that says "Private message to <name>" instead of "Say what now?", and a tinted field -- and the chip's close button is the single tap back to the room. The recipient is read and cleared together with the text before the send suspends, so a second message cannot inherit the first one's audience. Two renderings, because a direct message looks different depending on whether this device can open it. Readable: the ordinary bubble, plus a lock and "Private to <name>" (or "Private to you"). A private message must never pass for a public one, and the label names the other party because that is what a reader would otherwise assume was the whole room. Opaque: a system line -- "Alice sent a private message to Bob" -- in the shape of RitualNotice rather than a bubble. An empty bubble attributed to Alice would read as her having said nothing, and one with placeholder text would read as her having said the placeholder. It is not tappable; there is nothing behind it to open. Reply privately is offered only on somebody else's message in an MLS room. A NIP-17 room has no audience for a message to be private from, so there the option would be meaningless. Names resolve from the room's participants, which the view model already holds -- the recipient is a member by definition -- so neither line needs a join, and both fall back to a shortened key rather than to "unknown". A line that cannot say which member it means is worse than an ugly one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
79e62d8239 |
fix: attribute a group message to the member who sent it
isUserMessage was computed by comparing the active key against groupEvent.pubKey -- the kind:445's signer. That is a fresh random ephemeral key on every send (`NostrSignerInternal(KeyPair())` in encryptAndSendMarmotInnerEvent), so it could never equal anybody's identity. The comparison was false for every Marmot message in every room, which means every message a member sent themselves rendered as somebody else's: wrong side of the transcript, wrong colour, and no delivery status, which is drawn only for our own lines. All fourteen arms now read the MLS sender identity, which is authenticated to a leaf and is the only thing this can honestly be computed from. For the four results that carry no leaf index -- the commit and proposal statuses -- it is null and yields false, exactly what they got before. Separated from the direct message work that exposed it because it changes bubble alignment in every existing MLS room, and that is worth being able to revert on its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
5ae974517b |
feat: read a direct message, or say one was sent
Completes the inbound half. A member now files one of three things when a kind:1059 arrives as an application payload, and which one depends only on whether their key opens it. The carve-out first. MarmotInboundManager rejects any inner event whose pubkey is not the MLS sender's credential identity -- MIP-03, and the check that stops a member minting events attributed to somebody else. A gift wrap is keyed to a throwaway key by construction and names nobody, so it cannot satisfy a check about its author; kind:1059 is now exempt. The check is not weakened, it is relocated. What replaces it is `seal.pubKey == senderIdentity` on a seal whose signature verifies -- a signature bound to an MLS leaf, rather than a plaintext field compared to one. It is strictly harder to forge: the attack it stops is a member re-wrapping a seal they were legitimately sent and passing it off to a third party as its author's, and that fails here because the MLS frame says who actually sent this one. senderIdentity also stops being optional. It was previously only compared; now every sender-derived field reads from it, because the payload carries no author at all. A leaf with no identity is an error rather than a mismatch. Attribution is resolved in NostrDao and handed to fromGroupEventResult, rather than added to GroupEventResult.ApplicationMessage where it belongs. quartz is a binary dependency here (com.vitorpamplona.quartz:quartz:1.14.0) and the local checkout is a reference copy, not a build input, so the result type cannot gain a field without publishing a fork. NostrDao holds the group, the leaf index is already on the result, and an application message advances no epoch, so the tree has not moved by the time it reads it. Same value, no fork. The three outcomes: The recipient opens the wrap and gets the words. The rumor is stored as its own MarmotInnerEvent keyed on the rumor's id -- the id the sender queued -- so both sides of the conversation hold one message under one identity. The wrap keeps its own row as the wire artifact. A bystander gets a line with no content. That is the feature working: the group is meant to see that a private message was sent and to whom, and nothing else. The sender, on a re-sync, is indistinguishable from a bystander, because the wrap's key was discarded and we cannot open our own message. Left unguarded this files an empty placeholder over the row sendChatMessage wrote -- which is the only copy of those words anywhere. Hence the early return on senderIdentity == us, mirroring the guard NostrDao.persistInboundChatMessage already carries on the NIP-17 path. A failed validation drops the message and logs rather than throwing. The caller is inside storeNostrEvent's transaction, and a forged direct message should cost its own line, not the whole event. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
1700e6d899 |
feat: send a direct message into the group, wrapped for one member
Completes the outbound half: a message with a recipient is queued as the rumor a gift wrap will carry, and the notary wraps it on its way into MLS. Everything downstream -- MLS encrypt, outer ChaCha20, kind:445, persistence, broadcast -- is untouched and does not know the difference. sendChatMessage resolves the recipient against the room before it writes anything. A recipient outside the room cannot be sent to: the wrap would be undecryptable by every member including them, while the group still saw that a private message had gone somewhere. Sending to yourself is refused for a different reason -- the wrap's key is discarded, so it could never be read back. A direct message keeps kind:14 rather than being mapped down to the group's kind:9 the way an ordinary message is. It is a NIP-17 chat message that happens to travel inside a group, and the kind is what tells the two apart on the way back in. Two things here are less arbitrary than they look: The queued row IS the rumor -- same kind, same tags, same content, same timestamp -- so its id is the one the recipient computes after unwrapping. That is the identity of the message on both sides. Which means the wire event's id is NOT the row's, and one existing lookup assumed it was. `getChatMessagesByMarmotInnerEventId(innerEvent.id)` now keys on the queued row instead. Left alone, a direct message's wrap id would match no ChatMessage, the lookup would come back null, and no BroadcastNostrEventRequest would ever be inserted -- encrypted, persisted, and silently never sent, with no error anywhere. The two ids are the same value for every other kind of message, so nothing else changes behaviour. The plaintext is scrubbed from the queued row once it has been sent. It is already on the ChatMessage row, which is the sender's only copy; a second one would be cleartext left behind in a table that otherwise holds nothing but wire events. sealGiftWrapPayload now refuses any payload belonging to an MLS room. That path is the one way a gift wrap reaches a relay -- the notary watches for payloads with no seal, seals them, and broadcasts -- and a Marmot direct message is a real, correctly signed NIP-59 wrap, indistinguishable from something this path would be right to publish. Keeping direct messages out of GiftWrapPayload is what makes them unbroadcastable; this refuses at the other end too, rather than trusting every future caller to know that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
296011dcd5 |
feat: give a queued message somewhere to say who it is private to
Two nullable columns and the v5 migration that adds them, ahead of the code that fills them, so the schema lands on its own and can be reverted on its own. MarmotInnerEvent.directMessageRecipientPublicKey is the outbound signal. The notary reads a queued row and has no other way to know a message is meant for one member rather than the room -- the plaintext is identical either way -- so this is what routes it into the gift wrap path. Inbound rows leave it null on purpose: the recipient is on the wrap's `p` tag, which is where every member reads it from, so a second copy on the row would be a second thing that can disagree. ChatMessage.directMessageRecipientPublicKey is what the transcript reads. Both lines a direct message can produce need it -- the one its two parties see, and the "sent a private message to Bob" line everybody else gets -- and holding it on the row keeps the view model off a join for a fact it already has to render. MarmotInnerEvent hand-writes equals and hashCode over every field, so both are extended too. A field missing from those is not a compile error and not a test failure; it is two rows that differ comparing equal, which surfaces much later as an upsert that does nothing. Room generates the migration -- verified as two ADD COLUMNs with no table rebuild, so nothing is copied and nothing can be dropped: ALTER TABLE `ChatMessage` ADD COLUMN `directMessageRecipientPublicKey` TEXT DEFAULT NULL ALTER TABLE `MarmotInnerEvent` ADD COLUMN `directMessageRecipientPublicKey` TEXT DEFAULT NULL Rows written before this come back null, which reads as "not a direct message" -- the only answer that is true of all of them. v5 is an AutoMigration entry rather than a hand-written Migration like MIGRATION_3_4 next to it, because that one rewrote data without changing shape and this one changes shape without touching data. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
79e99ae702 |
feat: build the envelope a direct message travels in
A one-to-one message inside a Marmot group is a stock NIP-59 gift wrap carried as the MLS application payload: a throwaway-keyed kind:1059 around a sender-signed kind:13 seal around the kind:14 rumor holding the words. Every member decrypts the MLS layer and sees the wrap; only the recipient can open it. See docs/marmot-direct-messages.md. This is the crypto on its own, with no database and no MLS state, because the outbound path (the notary) and the inbound path (the kind switch in ChatMessage) both need it and neither can be unit-tested -- there is no sqlite driver on the JVM test classpath. Extracting it first is what makes the ten tests here possible; real secp256k1 does load under testDebugUnitTest, so none of this is mocked. Three choices worth stating, all of them consequences of the wrap using a throwaway key rather than the sender's own: Nothing in the wrap names the sender. GiftWrapEvent.create mints and discards its own random key, so who sent a message comes from the MLS frame around it -- authenticated to a leaf, and unforgeable -- rather than from a self-asserted pubkey field. The seal inside is the only layer the sender signs, which is what the inbound path will bind to the MLS sender identity before it renders a word. The sender cannot reopen their own message. The throwaway key is gone at send time and nothing reconstructs it. `the sender cannot reopen their own message` asserts that rather than leaving it to be discovered, because the obvious fix -- persisting the throwaway private key -- would be strictly worse than the identity-keyed wrap this was chosen over, and would reintroduce the attribution the throwaway key exists to remove. No layer is fuzzed. NIP-59 randomises the wrap and the seal by up to two days to frustrate correlation at a relay, and both GiftWrapEvent.create and SealedRumorEvent.create default to it. There is no relay at this layer and the kind:445 already carries the true time, so fuzzing would only scatter the "sent a private message" line up to two days out of position in every other member's transcript. open() returns null rather than throwing on every way a wrap can fail to open -- somebody else's message, a malformed payload, a layer that is not the kind it claims. Its caller is midway through processing a kind:445 that may carry a perfectly good message for somebody else, and an exception would abandon all of it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
c8c962e4f4 |
docs: inventory the unreferenced code in the sync and relay stack
Found while building the long-running sync. One item was orphaned by that
change; the rest was already dead and only became visible because the subsystem
was being read closely. Written down rather than deleted because several pieces
are one decision away from being wanted, and those decisions are not the sync
change's to make.
Every claim is "this identifier appears exactly once in composeApp/src, at its
own declaration", with the two things that method cannot see called out: Room
DAO methods are reached through generated code, and Compose entry points can be
invoked without a textual reference. The DAO cluster is flagged as the least
certain for exactly that reason.
Three findings are more than leftovers:
- RelaysSocketManager.userRelays is a field nothing ever writes. The
`userRelays` inside observeRelays is a different, shadowing local, so the
single-argument publishEvent always takes its FALLBACK_RELAYS branch and the
user's own relay list is never used for publishing. That is a bug wearing
dead code's clothes, and the fix is to populate the field, not to delete it.
- NostrPublisherRepository is entirely unreferenced, and it is the only
consumer of CachingImportRepository.importEvents. RelayPool and
RelaysSocketManager each take a cachingImportRepository parameter they store
and never dereference, satisfied by NO_OP_CACHING_IMPORT_REPOSITORY — so the
whole seam is a parameter passed from nowhere to nothing. Removing the
publisher lets the interface and both parameters go with it.
- sendAUTH is unused because NIP-42 is unimplemented, not because it is
surplus. AuthMessage is parsed and dropped, so a relay answering CLOSED with
auth-required is retried forever and can never succeed. Deleting sendAUTH
means deciding against authenticated relays; that is worth doing on purpose
or not at all. sendCOUNT and CountMessage are a similar matched pair — both
go or neither, since a CountMessage cannot arrive if nothing sends a COUNT.
isRecommendedRelay on the two request entities is separated out as its own risk
class: never written, never read, but a Room column, so it wants a migration
rather than a delete.
Ends with an order to do it in, cheapest and least risky first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
385c58ba7e |
docs: rewrite the sync note as what exists rather than what to build
The design landed across the six commits before this one, so the note is now
describing code. Reorganised around that: the reasoning that made it worth
writing is unchanged, but "the shape to build" is now "how it holds together"
and points at the classes, and the numbered traps have become properties of the
thing rather than warnings about a thing that did not exist yet.
Three sections earn their place after the fact:
- the two timestamp decisions, which are the ones most likely to be "cleaned
up" by someone who has not read this: no `since` on kind 1059 because our
own wraps are stamped up to two days in the past, and no watermark on 445
even though it would be safe, because `limit` already bounds the burst.
- the four ways a group id can appear, which is why the group filter is
derived from the room list rather than wired at the join sites.
- "Not done", which was previously implicit in a staging plan: connectivity
changes, NIP-42 AUTH, the collector-per-socket router the design originally
called for, and the fact that the DM relay set is one relay.
The "suggested order" section is gone; git log is a better record of it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
ca1093637c |
refactor(chat): stop syncing chat messages on screen open
The point of the previous four commits. Both chat screens scheduled a message
sync every time they were opened; both are now covered by subscriptions that are
already open, so the sync on open is work with nothing left to do.
**ChatRoomListViewModel** no longer syncs on initiate(). scheduleSynchronization
itself stays, and is unchanged: it is exactly the reconciliation
LiveSubscriptionManager runs when the app returns from the background, and it is
what an explicit user-initiated refresh should call. That is the one case the
live tier genuinely does not answer, because it is the user saying they believe
something is missing.
**ChatMessageListViewModel** loses its message sync entirely. The MLS branch
(negentropy over 445 h-tagged with this one room) is a strict subset of
live-groups-*. The NIP-17 branch's gift wrap sync is a subset of live-giftwrap.
What survives is discovery rather than sync: if we do not hold a participant's
kind-10050 we cannot address a message to them, and that is worth resolving the
moment a chat is opened rather than whenever a background pass reaches it. The
function now does only that, and only when the relay list is actually missing —
it used to queue a request in both branches of that test.
**The dead "sent-messages" reconciliation is deleted**, in the view model and at
both sites in NostrDao. It asked for kind 1059 with authors=[userPublicKey], and
could never match a single event: a gift wrap is signed with a throwaway KeyPair
(DatabaseChatRepository), so its pubkey is random and never ours. It was also
unnecessary — createNip17ChatRoom puts us in our own participant list, so we
wrap a copy to ourselves and the live gift wrap subscription picks our own sent
messages up on every device.
Removing it inverts the surrounding test in NostrDao from `if (relayList !=
null) { sync } else { discover }` to `if (relayList == null) { discover }`. The
discovery half is untouched.
Net effect on a session: opening the chat list queues nothing, opening a chat
queues at most a kind-10050 lookup for a participant we cannot yet address, and
messages arrive because a subscription is open rather than because a screen
asked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
0ef0a33350 |
feat(chat): close live subscriptions in the background, catch up on return
Subscriptions that stay open are only free while the app is on screen. Holding a socket open behind a doze window achieves nothing but battery, and the relay drops the subscription anyway — so this ties them to the app lifecycle and adds the reconciliation that covers the gap. Nothing in this app observed the app lifecycle at all: MainActivity only calls setContent. AppLifecycle is a small singleton holding an isForeground StateFlow, fed from a LifecycleEventObserver in MantraNavHost (ON_START/ON_STOP), and read by LiveSubscriptionManager. A singleton rather than something threaded through the composition because the consumers are not composables — they are application-scoped coroutines started before any screen exists and outliving all of them. It defaults to foreground: on a platform that has not wired the observer up, "always on" is the behaviour that predates this file, and a subscription that never opens is a far worse failure than one that stays open too long. collectLatest over that flow is the entire mechanism. Backgrounding cancels the block holding the subscriptions, and each one's finally sends its CLOSE and releases the retained REQ on the way out — which is also what tells the socket it no longer has a reason to reconnect. On the way back: **Reconnect before asking for anything.** RelayPool.reconnectAll tears every socket down and immediately rebuilds it. Trusting the connection is the mistake here: a socket that was open when the OS suspended the process reports itself connected on the way back while being functionally dead. Re-opening eagerly rather than leaving it to the next send is deliberate — it is what makes this a RE-connection, so retained subscriptions are replayed and any collector still attached from before the gap starts receiving again. That also covers queue requests that were mid-flight when we went away, which would otherwise sit until SUBSCRIPTION_TIMEOUT. **Then a catch-up reconciliation.** A live subscription answers "what is new since I connected"; negentropy answers "what do you have that I don't". Coming back from a gap is exactly the question only the second can answer — `limit` on the re-opened subscriptions is a window, not a guarantee. queueCatchUpSynchronization queues the same two negentropy requests ChatRoomListViewModel queues on open: gift wraps p-tagged to us, and group events h-tagged with every group we are in. Deliberately the same filter shape as the screen's, down to limit=50. A negentropy request is stored under a hash of its filter, so an identical shape collapses into one row instead of queueing the same reconciliation twice while both callers exist. The value itself barely matters — the negentropy pump drops `limit` outright and it only survives into the plain-REQ fallback. The room-to-group-id rule (has MLS state, not left, not deleted, sorted) now lives in one place, since the catch-up and the subscription reconcile have to agree on what "a group we are in" means. Not covered here: connectivity changes. A network switch mid-foreground is still only noticed by the socket's own reconnect loop, which handles the common case but cannot know the network changed underneath it. That wants a platform connectivity observer, and is its own change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
f81dc6af69 |
feat(chat): keep group subscriptions open, and follow membership as it changes
The second live subscription: kind 445 h-tagged with every Marmot group we
belong to, chunked 100 group ids to a filter, open on every DM relay. Group
messages now arrive whether or not that chat is on screen — or any chat.
**Derived from the room list, not from the join sites.** The filter comes from
chatRepository.observeChatRoomListByPublicKey, the same Flow the chat list
screen renders, collected at wallet scope so it keeps running with no chat
screen open. That is the whole reason this stays correct: a group id can appear
four ways, and only one of them is somewhere anyone would think to call a
subscribe function.
- we create a group (MarmotOutboundDao),
- we are added to one — the Welcome arrives inside a gift wrap and is
processed deep inside NostrDao.storeNostrEvent, on the inbound path,
- membership shifts under us via a commit
(MarmotInboundManager.processGroupMembershipChanges),
- we leave, or the room is deleted.
Observing the table catches all four, and it closes the loop with the previous
commit: a Welcome lands on the gift wrap subscription, a ChatRoom row is
written, this flow re-emits, the group filter widens — and the first message in
a group we were just added to arrives without anyone opening a chat.
Rooms are filtered to those with mlsGroupState (a NIP-17 room has none and is
served by the gift wrap subscription instead) and without leftGroupAt or
deletedAt — a room we have left keeps its history locally but must stop pulling
new messages. Ids are sorted before distinctUntilChanged so the same membership
in a different row order is the same value, then debounced 500ms: joining a
group writes the room, its participants and placeholder profiles in quick
succession, each of which re-emits the list.
**Reconciling, not rebuilding.** A chunk that already has a subscription is
updated in place — the relay replaces that subscription's filter on a repeated
REQ with the same id — so adding a group does not interrupt delivery on the
groups already being watched. Only chunks that no longer exist are torn down,
and they are cancelAndJoin'd rather than cancelled: the CLOSE is sent from the
coroutine's finally, and returning before it lands would let a later reconcile
open a subscription on that id which the old one then closes out from under.
The filter is read from a supplier at every open rather than captured at launch,
so a subscription re-opened after a CLOSED comes back with the current
membership rather than the membership it was created with.
**No `since` here either, but for a different reason.** Group events do carry
honest timestamps — MarmotOutboundDao stamps them with TimeUtils.now(), unlike
gift wraps — so a watermark would be safe. It is still not used: limit already
bounds the initial burst, and a watermark would have to be recomputed every time
a chunk's membership changed. The group limit is 500 rather than the gift wrap
subscription's 100, because one chunk covers up to 100 conversations.
Also in here: RelayPool.updateLiveSubscription now retains the new REQ before
attempting to send it, and treats a missing or failing socket as "send it on
reconnect" rather than throwing. Throwing would have left the OLD filter
retained, so a reconnect would restore a subscription the caller had already
moved on from — the one outcome worse than not sending.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
ba05ec704d |
feat(chat): keep a gift wrap subscription open for as long as the app is active
The first live subscription: kind 1059 p-tagged to us, on every DM relay, open until the wallet changes or the process ends. That covers direct messages and the Marmot Welcome events that make us a member of a group — so a DM now lands whether or not anyone has the chat list open, and a group invite is noticed without waiting for the next screen to schedule a sync. Nothing is removed from the queue yet. ChatRoomListViewModel and ChatMessageListViewModel still schedule what they always did; this runs beside them and the duplicate events cost a no-op storeNostrEvent each. The screens stop doing it in the last step, once groups are covered too. **No `since`.** NIP-59 randomizes a wrap's created_at into the past, and our own outbound path stamps them with TimeUtils.randomWithTwoDays() — so a wrap published right now can carry a timestamp two days old, and a `since` anywhere near the present would silently drop it. "Some messages just never arrive" is the worst failure mode to debug, and re-receiving a wrap costs one no-op write. **`limit` instead.** NIP-01 scopes limit to the initial query — the stored events a relay sends before EOSE — and explicitly not to the stream that follows, so limit=100 bounds what a reconnect costs without touching the live tail. Filling in the rest of the history stays negentropy's job. **What ends a subscription, and what doesn't.** CLOSED does, and is retried with backoff from 5s to 5m, jumping straight to the maximum when the reason parses as back-pressure — answering "too many subscriptions" by promptly opening another is how one refusal becomes a flood. EOSE does not end anything: it is the boundary between stored history and the live tail, and treating it as an end is precisely what made every sync a poll. It does reset the backoff, since a subscription that got that far was working and whatever ended it is a new problem. A dropped socket needs nothing from this loop at all — the pool replays the REQ on reconnect and the collector, attached to the socket client rather than to a session, simply starts receiving again. A NOTICE is logged and otherwise ignored. It carries no subscription id, so it reaches every collector on the socket; acting on one here would let an unrelated relay complaint tear this subscription down. Events are saved inline rather than in a launched coroutine, so they land in the order the relay sent them and the socket's buffer does the back-pressure, instead of fanning a burst out into a coroutine per event. Wiring: LiveSubscriptionManager is built by SynchronizationViewModel because it has to share that class's RelaysSocketManager — a second one would mean a second RelayPool and a duplicate socket per relay — and is launched inside the same per-wallet supervisorScope as the three pumps, so a wallet switch tears it down. It deliberately does not take from the pumps' Semaphore(4): a subscription that never finishes would hold a permit forever and permanently halve backfill throughput. The comment there now says so. isBackPressure moves out of SynchronizationViewModel to network/relays/RelayBackPressure.kt, unchanged, now that two callers need it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
3ade53607a |
feat(relays): plumbing for subscriptions that are meant to stay open
Three pieces, none of which opens a subscription yet — the manager that does
lands next.
**A subscription that survives EOSE.** RelayPool.openLiveSubscription is query()
without completeOnSubscriptionEnd. For a one-shot request EOSE is the end: it is
what finishes the collector, frees the subscription slot and triggers the CLOSE.
For a live subscription EOSE is only the boundary between stored history and the
live tail, and the relay owes us nothing further to mark an end — so the flow
ends when, and only when, the caller stops collecting.
The returned flow is attached to the socket CLIENT's message flow rather than to
a session, so it survives a drop: the socket reconnects, the previous commit's
replay re-sends the REQ, and the same collector starts receiving again with
nothing rebuilt. updateLiveSubscription re-sends under the same subscription id
to widen or narrow the filter in place — a relay answers a repeated REQ on an
existing id by replacing that subscription's filter, so there is no
close-and-reopen and the collector never notices.
**Ordered, buffered inbound.** The socket's incomingMessages was a rendezvous
SharedFlow emitted into from a coroutine launched per message. Two consequences a
short request/response collector never noticed, and a permanent one would:
- messages reached collectors in whatever order those coroutines happened to be
scheduled, and
- an emit with every collector busy blocked on the slowest of them.
It now has a 256-message buffer and is emitted into inline, on the reader, in
wire order. That also lets the 75ms sleep before every EOSE go: it existed to
hope that the events preceding an EOSE had already been delivered by their own
coroutines, which ordering now guarantees outright.
The buffer is the back-pressure boundary — a collector may fall 256 behind
before it slows its socket's reader down. Deep enough to absorb the burst
between a REQ and its EOSE while a collector writes each event to SQLite, not so
deep that a stuck collector is invisible.
**A saveNostrEvent that does not need a request row.** Both existing overloads
take the row an event was fetched for, because they also record which request
produced it and flip that row to "processed". A live subscription has no row and
never finishes, so the new overload carries relayURL and level itself and goes
straight to storeNostrEvent. Everything downstream — indexing, gift wrap
unwrapping, Welcome handling, MLS decryption — is unchanged.
Fixed while adding it: the negentropy overload was writing outside
storeNostrEventMutex while the other one held it. storeNostrEvent reads an event
and then writes it and its indexes, so two of those interleaving is a lost
update. Survivable while a single queue was the only writer; not survivable with
a live subscription writing alongside a backfill.
Deviation from docs/long-running-sync.md worth noting: the doc proposed one
permanent collector per socket dispatching by subscription id prefix. With the
inbound flow now buffered and only a handful of live subscriptions per relay, a
collector per subscription has the same properties for less machinery. If the
live subscription count per relay ever grows, the router is the next move.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
c3b0c8671b |
feat(relays): reconnect dropped sockets and replay what they were carrying
Nothing in this app reconnected a websocket. NostrSocketClientImpl caught a
failure, called close(), and fired onSocketConnectionClosed — which only flips a
boolean in relayPoolStatus that nobody reads. A socket that failed stayed
failed, and the only reason that was survivable is that every subscription is
short: the next queued request opens a fresh socket on its way out through
ensureSocketConnectionOrThrow.
That stops being survivable the moment a subscription is meant to outlive the
socket, so this lands first, on its own. It is already a fix without any of
that: a REQ interrupted mid-download used to sit there until SUBSCRIPTION_TIMEOUT
gave up 120s later, having saved whatever partial set arrived before the drop.
Now the socket comes back and the REQ is re-sent.
The socket client:
- a supervised reconnect loop with exponential backoff (1s doubling to 60s)
plus up to 25% jitter, because every relay in the pool drops at once when
the network does and without jitter they all come back in lockstep. The
exponent is capped so a socket failing for hours cannot overflow the
doubling into Infinity, which Duration * Double rejects outright.
- `autoReconnect`, off by default and owned by the pool. Reconnecting a socket
nobody is subscribed on is battery spent on nothing, so the pool turns it on
for exactly as long as it retains a subscription for that relay.
- `closedByClient`, so closePool() is not answered by every socket in it
politely reconnecting. Cleared by the next caller-driven connect.
- onSessionLost() as the single exit point for a session that ended without
the client asking, replacing the close()-from-inside-the-receiver dance. It
identity-checks the session before clearing it, so a reconnect that already
installed a newer one is not torn down by its predecessor's cleanup, and
runs NonCancellable because the receiver job is cancelled as part of a
replacement connect.
- Frame.Close now breaks the receive loop rather than closing by hand. The
relay closing us is not the client closing us, so it earns a reconnect too.
- a new SocketConnectionReopenedCallback, fired only when a session is
established on a socket that had connected before. Kept separate from
"opened" because on a FIRST connect a replay would double-send the very REQ
whose sendMESSAGE opened the socket.
Two bugs fixed in passing, both of the silent kind:
- the compression REQ in the post-connect handshake was written to `wsSession`
before the new session was assigned to it, so it went to the previous
(usually null) session and was dropped. wsSession is now assigned first.
- sendMESSAGE used `wsSession?.send(...)`, so a send on a dropped socket was a
no-op and the caller waited forever for an answer to a message never sent.
It now warns.
The pool:
- retains the REQ text per (relay, subscription id), and replays it when that
relay's socket is re-established. A relay answers a repeated REQ on the same
subscription id by replacing the filter, so replay is a send rather than a
close-and-reopen, and the collector already attached to the socket's message
flow simply starts receiving again.
- retains on query() BEFORE the send, so a socket that dies between there and
the relay's first answer is still covered; releases on closeQuery(), which
every pump already calls from a NonCancellable finally.
- deliberately does NOT retain negentropy. NEG-OPEN carries a fingerprint of
the local set and each round depends on the last, so replaying one
mid-exchange would reconcile against a conversation the relay is no longer
having. An interrupted negentropy request is abandoned and re-queued.
- drops retained work for relays removed by changeRelays/removeRelays/
closePool, so a relay edit does not leave a socket reconnecting for
subscriptions nobody wants.
- collapses the five hand-rolled `socketClients.find { normalize... }` lookups
into socketClientFor(), now that there were about to be several more.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
178ddd0181 |
docs: write down how a long-running chat sync would work
Every chat sync today is a pull: a screen queues a request row, a pump drains
it, the relay answers, the subscription is closed. Nothing arrives between
pulls, so a message sent one second after EOSE waits for the next time someone
opens a screen.
This note works out what it takes to hold the two chat subscriptions open for
as long as the app is active — kind 1059 p-tagged to us, and kind 445 h-tagged
with every group we belong to — and, more usefully, what in the current
pipeline quietly assumes a subscription is short:
- completeOnSubscriptionEnd finishes the flow at EOSE, which is what releases
the slot and sends the CLOSE,
- SUBSCRIPTION_TIMEOUT hard-kills anything still open at 120s,
- subscriptionSlots is a Semaphore(4) shared with the backfill queue, so a
permanent subscription is a permanently-held permit,
- both saveNostrEvent overloads need a request row to attach provenance to
and to flip to "processed",
- and nothing in the app reconnects a dropped socket at all. That is
invisible today only because every subscription is short and the next
queued request re-opens the socket on its way out.
The design keeps the queue and its three pumps exactly as they are: live
subscriptions replace polling, not reconciliation. Negentropy stays the tool
for first login, the catch-up after a background gap, and "load older".
The group filter is derived from chatRepository.observeChatRoomListByPublicKey
rather than wired at each join site, because a group id can appear four ways
and only one of them (creating a group) is somewhere anyone would think to call
a subscribe function — being added arrives as a Welcome processed deep inside
NostrDao.storeNostrEvent. Observing the room list also closes the loop: a
Welcome lands on the live gift wrap subscription, a ChatRoom row is written,
the Flow re-emits, and the group filter widens without anyone opening a chat.
Two findings fell out of checking the details against our own code:
- `since = now` on kind 1059 would silently drop messages. Gift wraps are
stamped with TimeUtils.randomWithTwoDays(), so a wrap published now can
carry a created_at two days in the past. Kind 445 uses TimeUtils.now() and
can take a watermark — opposite treatment for the two kinds we care about.
- the "sent-messages" filter (kinds=[1059], authors=[me]) cannot match
anything, because gift wraps are signed with a fresh throwaway KeyPair().
It is also unnecessary: createNip17ChatRoom puts the user in their own
participant list, so we wrap a copy to ourselves and the account-wide
#p=[me] subscription already picks it up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
248a527267 |
fix: store the framed commit on MarmotCommitResult, not the exporter secret
`MarmotOutboundDao.inviteMember` persisted the commit row with
framedCommitBytes = commitResult.preCommitExporterSecret,
two lines below the argument that value belongs to, which was already assigning it
correctly. It now reads `commitResult.framedCommitBytes`.
The row is written on the deferred branch, after the kind:445 commit has gone out
and while the welcome waits on a relay acknowledgement, so what it holds is meant
to be the record of what was published.
## Why the compiler had nothing to say
`MarmotCommitResult` carries quartz's `CommitResult` payload fields verbatim --
`commitBytes`, `welcomeBytes`, `groupInfoBytes`, `framedCommitBytes`,
`preCommitExporterSecret`, same names, same order, same defaults. Both of the
fields in question are `ByteArray`, so the wrong field of the right object is
indistinguishable from the correct one at the type level.
The call site lists its named arguments in a different order than the declaration,
which is what put `preCommitExporterSecret` and `framedCommitBytes` two lines
apart. The entity also repeats quartz's `framedCommitBytes: ByteArray = commitBytes`
default, so the explicit argument was overriding a fallback that -- while still the
raw commit rather than the framed envelope -- was at least a commit.
## What it cost, and what it would have cost
Nothing so far. `framedCommitBytes` has exactly two references in the tree: this
assignment, and `encryptedCommitEvent` at the top of the same branch, which takes
`commitResult.framedCommitBytes` from the in-memory `CommitResult` rather than from
the row. The bytes that reached the relay were always the right ones; the wrong
ones only ever sat in the column.
They would stop merely sitting there as soon as anything reads the row back.
`DatabaseNostrRepository` already reloads these rows on acknowledgement, at
`getMarmotCommitRequestById`, to pick up `welcomeBytes` and fire `deliveryWelcome`.
An ack-triggered rebroadcast or a replay reaching one field further along would
publish 32 bytes of exporter secret where a
`MlsMessage(PublicMessage(FramedContent(commit)))` envelope was expected: not a
message recipients drop, but a group key on a relay.
The smaller half holds whether or not anything ever reads it. The group's
pre-commit `MLS-Exporter("marmot", "group-event", 32)` output was being written to
a second column that is not intended to hold key material, doubling its footprint
at rest alongside the `preCommitExporterSecret` field that exists for it. Only at
rest -- the ack path logs the row, but the data class has no `toString` override,
so `ByteArray` prints as an identity hash rather than contents.
## Scope
`MarmotCommitResult` has a single construction site in the codebase, the one
changed here, so there is no second copy of this to fix. Worth checking rather than
assuming: the shape that produced it -- adjacent `ByteArray` fields with identical
names on both sides of the copy -- reproduces anywhere the entity is built again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
907dba3c3b |
fix: create the #admins room only once every member can be added
Every member's key package is now resolved before anything is created. If one is missing the room is not created at all, and the coordinator is told which member to go and ask rather than being handed a room quietly short of people. Previously the room was created and then whoever could be added was added, with the rest collected into a list that only reached the log. Two things make that the wrong trade here, and neither applies to ordinary group creation: MarmotGroupData.adminPubkeys is baked into the epoch-0 GroupContext and names every member of the ceremony. A room created without one of them therefore lists an admin who is not in the MLS tree -- a group that disagrees with itself from its first epoch, and MIP-01 leans on that list for most group operations. And the id is derived from the shared key, so there is exactly one room per group at this path. A half-created one occupies that address permanently; unlike a random id there is no second one to retry with. Creating nothing leaves the retry clean. The lookup moves ahead of group creation, which also means the batched add now receives a list it knows is complete -- `addMembers` no longer has to reason about absent key packages on this path. `inviteAdmins` goes with it. Its job was resolving key packages and then adding whoever it could; the first half moved into the precondition and the second is a direct `addMembers` call. The blocked members surface as `DkgRitualUIState.adminGroupBlockedOn`, carrying names rather than public keys -- the action this prompts is asking a particular person to open the app, so a name is what the coordinator needs. Cleared when the button is pressed again, so a retry does not show the previous answer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
3dea07135c |
fix: add a group's whole membership in one commit, closing the epoch race
`MarmotOutboundDao.addMembersToChatRoom` stages every member with `proposeAdd` and issues a single `commit()`. Both callers that know their membership up front now use it: `SelectChatRoomTypeViewModel.inviteMembers` at room creation, and `DkgRitualViewModel.inviteAdmins` for the #admins room. Inviting one at a time created an epoch per member, and each of those commits raced the previous member's welcome. MarmotInboundManager refuses future-epoch messages outright, on both wire formats, with no queue and no replay -- so the member who lost that race was silently stuck an epoch behind while the caller saw a successful invite. Deriving isOneMemberInitialGroupCreation narrowed that window; this removes it. No member ever has to process a commit for an epoch they were not yet in, so there is no longer a race to lose. One commit yields one welcome: `buildWelcome` emits an EncryptedGroupSecrets per added member and each joiner finds its own entry by key package reference. The blob is shared, delivery stays per peer, because each welcome event is tagged with that peer's key package. ## Why this needed no schema change Batching at creation time means the single commit happens while the group is still only its creator, which takes the immediate-welcome branch: nothing is broadcast and MarmotCommitResult is never written. The bookkeeping that assumes one peer per commit is simply not on this path. So the batch is taken only when `members().size == 1`, and anything else falls back to inviting sequentially -- correct, if not ideal. Batching into an established group would take the deferred branch, where `peerKeyPackageEventId` is singular and the ack-triggered delivery in DatabaseNostrRepository expects one welcome; making that work needs a list there and a fan-out on acknowledgement. Nothing currently adds several members to an established group, so that is left outstanding and documented rather than speculatively built. The group state is persisted after `commit()` and before any welcome goes out, so a crash between them leaves the group at the epoch the welcomes describe rather than one behind it. ## Reporting Members with no published key package still cannot be added -- a Marmot invite needs one -- and are now returned alongside any that failed to receive their welcome, rather than the two being conflated. Both still only reach the log; the coordinator is not yet told. docs/marmot-membership.md is updated in the same change: batching moves from outstanding work to described behaviour, with the schema constraint that shapes it and the remaining fan-out work recorded. The note about sequential invites is narrowed to where they still happen. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
8fc1c9e650 |
fix: stop deferring the first invitee's welcome behind a commit nobody needs
`inviteMember` now works out for itself whether the group it is adding to has
anybody to inform:
val isOneMemberInitialGroupCreation = mlsGroup.members().size == 1
read before `addMember` advances the tree. The parameter is gone from the
signature and no caller passes it any more.
Callers were the wrong place for this decision and both of them got it wrong.
`inviteMemberToChatRoom` hardcoded `false`, and `ChatRepository.inviteMember` did
not expose it at all, so every invite made through a group -- room creation in
SelectChatRoomTypeViewModel, and the #admins room -- took the deferred-welcome
path. That includes the first invite, when the group is still only its creator, at
which point:
- the commit has no audience. No other member exists, and nobody outside the
group can decrypt it, so it is noise on the relay.
- the welcome is then withheld until a relay acknowledges that noise. If the ack
never lands, the first invitee receives nothing at all.
Only createMlsDirectMessageChatRoom passed `true`, and only because a DM has
exactly one invite. A group of n has one such invite too -- the first -- and it was
not getting it.
The condition is right at any size, not just for DMs: "the group has nobody to
inform" is true exactly once. Invite two sees one member who must advance, invite
three sees two, and so on. Their commits are encrypted with
`commitResult.preCommitExporterSecret`, the epoch the earlier invitees received in
their own welcome, so they can decrypt and advance. `members()` skips empty leaves,
so this also stays correct for a group that has had members removed.
## What this does and does not fix
It removes a pointless commit and, with DefaultDMRelayList now a single relay, a
single point of failure sitting in front of every group's first member.
It also narrows a silent race rather than closing it. MarmotInboundManager refuses
future-epoch messages outright on both wire formats -- no queue, no replay -- so a
commit arriving before its recipient's welcome is dropped and that member never
advances, while the coordinator sees a successful invite. Previously both commits
went out before either welcome; now welcome 1 is sent before commit 2 exists, so
the first invitee is already at the right epoch. For n >= 3 the window between
welcome 1 and commit 2 remains.
Closing it needs the adds batched into one commit, which is the outstanding work
described in docs/marmot-membership.md. That doc is updated here to describe the
derived flag as current behaviour rather than a proposal, and to keep batching as
the remaining item.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
b99cb8fcd5 |
docs: write down the shared-key subsystem and how Marmot membership fails
First docs in the repo -- README.md is still the stock KMP template. Three documents plus an index, covering the parts whose behaviour is not recoverable by reading the code: where the reasoning lives in a protocol, where a failure mode is silent, or where a decision looked arbitrary and was not. marmot-membership.md is the one that earns its place. Everything about adding a member compiles, the invite reports success, and a member simply never appears -- and the reason is never in the invite code. It records that inviteMemberToChatRoom hardcodes isOneMemberInitialGroupCreation = false and that ChatRepository does not expose it, so every group invite takes the deferred-welcome path including the first, when the group is still just its creator and the commit has no audience at all. Then why that is silent rather than noisy: MarmotInboundManager refuses future-epoch messages outright, on both wire formats, with no queue and no replay, so a commit arriving before its recipient's welcome is dropped and that member never advances. EPOCH_RETENTION_WINDOW retains past epochs and does nothing for messages from ahead. Three options are set out with the per-invite correctness table, including the honest limit that the recommended one narrows the race without closing it. shared-key-derivation.md argues why the paths are not BIP32 -- no chain code exists, hardened derivation is impossible rather than unimplemented, and a FROST tweak takes the scalar as input so the chain code leaves the problem entirely. It records the x-only serialisation trap avoided by choosing the scalar directly, and states the rule that must not be broken: never reconstruct a derived key in the clear, because k = k' - t hands over the group key rather than one derived key. shared-key-ceremony.md covers the seven kinds, the three approval gates and why the coordinator's aggregations are deliberately not among them, faults as values rather than exceptions, and the transcript's idempotency-by-construction. It also writes down the invariant that produces no error when broken: pendingApproval must mirror the gates in advance, or the screen offers an approval that does nothing -- or none while the ritual sits still. Every factual claim was checked against the source rather than recalled, which turned up one correction worth having: there are two future-epoch refusals, for PrivateMessage and for Commit, so the drop covers both wire formats and not just one. Each document leads with the failure mode rather than the architecture, on the grounds that a failure is what sends somebody to docs in the first place, and each lists its known gaps -- including that none of this has run on a physical device. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
9f14679aac |
feat: let the coordinator open a #admins room keyed on the shared key
Once a ceremony completes, the shared-key screen offers its coordinator a Marmot
room named "<group> (#admins)" with every member of the ceremony in
MarmotGroupData.adminPubkeys. The room the ceremony ran in is NIP-17, where nobody
administers anything; this gives the same people a room where every one of them
can act, which is the shape a group that has just made a t-of-n key is asking for.
Built directly rather than through MarmotGroupData.bootstrap, which hardcodes a
single admin, and baked into the epoch-0 GroupContext so later invitees receive a
populated group from their welcome instead of chasing a bootstrap commit that
predates their membership.
## The id is derived, not random
Every other Marmot room mints `nostrGroupId` as RandomInstance.bytes(32). This one
derives it from the group's threshold key, settling the
`// TODO: Generate GID through frost...` already sitting in
SelectChatRoomTypeViewModel.
Derivation buys two things random cannot. Every member's device can compute the id
from a ceremony they all took part in, so the room is addressable without being
announced; and two members racing to create it arrive at the same id rather than
two rival rooms -- which is why createAdminGroup returns to the existing room
instead of minting a second one.
## Why the derivation is what it is
SharedKeyDerivation walks the path as successive FROST tweaks, one per index,
returning both the XonlyPublicKey and the TweakCache. The cache is not an
optimisation: a signing session created without the same tweaks aggregates to
signatures that verify against a different key, which is why the id is usable as
an identity later rather than only as a label.
It is not BIP32, and the doc comment argues that at length rather than leaving it
to be rediscovered. A BIP32 node is a key *and* a chain code; ChillDKG produces no
chain code. BIP32 wants one only because it computes the tweak scalar for you, and
a FROST tweak takes that scalar as an input -- so choosing it directly removes the
chain code from the problem rather than requiring one to be invented and agreed
forever. It also removes a trap: with x-only keys there is no single obvious
serP(K_par), and two devices picking different parity conventions would silently
derive different keys rather than fail.
Each scalar commits to the key being tweaked as well as the index, so steps cannot
be reordered or replayed at a different depth. Tests cover that, determinism
across calls, path and key sensitivity, and that the cache and the public key
agree.
Hardened derivation is not available here and never will be: it needs the parent
private key, which in a threshold group nobody has. That leaves the non-hardened
weakness -- k' = k + t with publicly computable t inverts -- so anyone learning one
derived private key recovers the group key and can sign with no quorum at all. The
rule that follows is stated at the top of the file: never reconstruct a derived key
in the clear.
## The path is recorded in the room
MIP-01's group data is a fixed TLS schema with no extension map, so a custom field
would emit bytes other Marmot clients cannot decode. The path rides in the
description instead, on its own line under a marker, so somebody rewriting the
rest of the description does not cost the group the record of how its key was
derived:
Admins of Ubuntu Collective.
Shared key path: m/9420/0/0
Worth storing although the path is currently a constant: it is what rebuilds the
TweakCache a signing session needs, and recomputing from the constant only holds
while the constant never changes. parsePath refuses hardened indices rather than
tolerating them -- such a path cannot have been walked here, so acting on one
would derive something other than what the room claims.
## Known limits
Members without a published MarmotKeyPackage cannot be invited; inviteAdmins
collects them and logs them, and the coordinator is not yet told.
Invites go one at a time, each advancing the MLS epoch, so the room is re-read
between them. That inherits a silent failure mode documented in
docs/marmot-membership.md: the first invite takes the deferred-welcome path even
though the group is still just its creator, and a commit reaching a member before
their welcome is dropped rather than queued. Not introduced here -- group creation
has always done this -- but more visible in a room whose whole membership is known
up front.
Nothing here has run on a device.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
b200916844 |
feat: show the group's key in full, with a copy button
The shared-key screen showed `thresholdPublicKey.take(16)` followed by an ellipsis. A 16-character prefix is enough to recognise a key you already know and not enough for the one thing this key is for. Members compare it out of band to confirm every device finished the ceremony on the same key. That is the check that catches a device which quietly ended up elsewhere -- and it cannot be done against a prefix, or from a screen the value cannot be copied off. Both halves of that were missing. The whole 66-character key now renders, wrapping rather than ellipsised, in a monospaced face so a character-by-character comparison lines up instead of drifting under proportional spacing. A FilledIconButton beside it copies the key via LocalClipboardManager, the same way ShareProfileScreen and the image viewers already do it. The "Key: " prefix became a label above so the key gets the full width. No copied-confirmation toast, matching ShareProfileScreen: Android shows its own clipboard notice on 13+, and a snackbar here would double up. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |