a56295b0e2a0185ee2dc6e7dd421db4ba070714f
22 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Kgothatso Ngako
|
30c732af67 |
Merge branch 'mantra' into claude/home-chat-previews-e1913f
Two conflicts, and both are the same collision: mantra took schema v15 while this branch was also calling its migration v15. **The version number.** mantra's v15 adds `ChatRoom.joinedGroupAt` with a manual MIGRATION_14_15, because half of what it does -- deleting the placeholder chat lines already written for messages sent before this device joined -- is not a shape Room generates. That is the older claim on the number and it keeps it. The index migration here becomes `AutoMigration(from = 15, to = 16)` and the database goes to v16, so a device that has already run v15 gets the index on top of it rather than the two fighting over one version. `15.json` is resolved to mantra's wholesale -- an add/add conflict between two unrelated schemas is not something to merge line by line -- and 16.json is regenerated from the build. Checked rather than assumed: 16.json differs from 15.json in exactly one place, `index_ChatMessage_chatRoomId`, and no table's fields, createSql or other indices move. **mantra's new membership lines needed handling here**, and nothing would have told me: `98f766f` added `TYPE_MEMBER_INVITED`, `TYPE_MEMBER_INVITE_SENT` and `TYPE_MEMBER_INVITE_FAILED`, which the transcript renders as system notices. The chat list preview dispatches on the same question the transcript does -- is this somebody's words -- and a type missing from that check falls through to the chat bubble branch. A room whose newest line was an invite would have previewed as "Alice: Invited Bob to the group", which reads as Alice having said it. Exactly the failure `ChatMessage.MEMBERSHIP_TYPES`' own comment warns about, one screen over from where it was written. So `MEMBERSHIP_TYPES` joins the ritual and chronicle sets in `lastChatMessagePreviewText`. They are not in the AUTHORED sets -- their content is a whole sentence with the invitee's name already in it -- so they stand alone, which is what the transcript does with them too. One new test, over all three types rather than a representative one, since the set is the thing being relied on. Nothing else needed reconciling. mantra's pre-join fix filters at indexing time and deletes the rows outright, so the last-message subquery sees fewer rows and needs no `memberSince` clause of its own to stay in step with the transcript. 550 jvm tests and 319 android unit tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
81965ba2d3 |
perf: index chat messages by their room
The query in the previous commit reads the newest line of every room the user is in, once per room, and Room re-runs the whole thing every time a message lands anywhere. Without an index on `ChatMessage.chatRoomId` each of those lookups is a scan of every message on the device: work that grows with the entire history rather than with the room, on the hot path of every arriving message. A device with ten rooms and a few thousand messages does tens of thousands of row reads to redraw a list whose visible change is one line of text. Room has wanted this index since the foreign key was declared and has said so on every build -- `chatRoomId column references a foreign key but it is not part of an index. This may trigger full table scans whenever parent table is modified` -- which is the same warning it still emits for a dozen other `chatRoomId` columns. Those stay as they are; this one now has a reader that makes it matter. **Schema v15, and Room writes the migration itself.** Adding an index changes no columns and moves no rows, which is one of the shapes `AutoMigration` handles without a spec, so this is an entry in the list rather than another manual migration alongside MIGRATION_13_14. The generated 15.json differs from 14.json in exactly one place, checked rather than assumed: `index_ChatMessage_chatRoomId` appears on ChatMessage, and no table's fields, createSql or other indices move at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
49c012bf8b |
fix: a member is not shown the messages sent before they were in the room
A joiner gets the MLS key schedule from their own epoch forward and nothing before it. The relay does not know that and hands them the whole room: negentropy syncs down every kind:445 the group ever published, `indexMarmotGroupEvent` read each one against the group, the outer layer refused, and every refusal wrote an `undecryptableOuterLayer` line. So the room a member had just been invited to opened on a screenful of "Undecryptable Message" above the conversation -- one per message the group had sent before they arrived, none of them ever readable, and the count only grows with how long the group had been talking. **The epoch of a kind:445 is inside the layer that will not decrypt**, so an event this device cannot read cannot be asked what epoch it is from. "From before we joined", "from an epoch we have not caught up to" and "from an epoch that fell out of the retention window" are indistinguishable from the outside, and only the first is permanent. What separates it is not the ciphertext but the clock: it was published before the group made the epoch we joined at. **`ChatRoom.joinedGroupAt` is that moment, written down.** The Welcome's `created_at`, which the inviter stamps as it mints the Welcome out of the Add commit that made us a member -- so it is the group's own account of when our epoch began, not this device's account of when it heard about it. A group this device created sets it to the room's creation; it was a member from epoch 0 and there is nothing behind it to hold back. Stored rather than read off `createdAt`, which today holds the same value in both paths. `createdAt` is row bookkeeping and this decides which of a group's messages a member is allowed to see at all; the two being equal is a coincidence of the current code, and hanging the second off the first makes a future change to when a room row is written into a change in what gets discarded. `memberSince` is `joinedGroupAt ?: createdAt`, so a room joined before the column existed gets the fix too -- and gets it from the value every path that sets the column would have written anyway. **`predatesMembership` draws the line strictly before**, and that is a judgement rather than a fact. Nostr stamps `created_at` in whole seconds, so the second the Welcome was minted holds both the commit that added us -- the last act of the epoch before ours, unreadable by construction -- and any message another member sent the instant they applied it. Only one of the two can be had. An unreadable event kept costs one refused decrypt; a readable event discarded is a message the member never sees. So the second is kept, and a room may still show a single placeholder for the commit that added its newest member. **Two places gate on it.** `indexMarmotGroupEvent` returns before touching the MLS group, so nothing is decrypted, no `MarmotGroupEvent` row is filed for ciphertext whose key this device never had, and no line is written. `reindexMarmotGroupEvents` partitions them out of the sweep entirely: a replay can say in advance that no pass will ever read them, so replaying them only spends a refused decrypt per sweep and reports every one as a failure on a room where nothing is wrong. `MarmotReindexSweep` is untouched apart from carrying the new count -- it decides how many times to go round, not what is worth going round for. **Schema v15, and the migration is the half that fixes devices already showing the bug.** Nothing rewrites a chat line that is already in the transcript, so fixing the write path alone would leave every member who joined a busy room opening it on the same run of placeholders forever. `MIGRATION_14_15` adds the column and deletes the lines: only the two types in `UNRESOLVED_MARMOT_TYPES`, and only where the group event behind them predates the room. Those lines say nothing by design -- they stand in for an event that was never read -- so removing one loses nothing, while every other line is the final word on its group event. The group events themselves stay; this is about what the room shows. The column is left null rather than backfilled from `createdAt`. Null already means "ask `createdAt`", and copying the value would turn a fallback into a claim this migration is in no position to make. It is manual rather than an `AutoMigration` only because of the delete: `ALTER TABLE ... ADD COLUMN` appends, which is where Room's own generated migration for a nullable addition puts one, and Room compares a table's columns by name rather than by position. **The reindex report stopped being true**, so it carries the number now. With the backlog held back, `unresolved` falls to zero and the screen said "Nothing to reindex - 30 event(s) all read" about a room where 27 of them were never this device's to read. `MarmotReindexReport.predatingMembership` is reported alongside `stored`, and the detail screen names it: "3 event(s) all read - 27 from before you joined". A member invited into an old room is the ordinary case, not an anomaly to bury in a total. Seventeen tests. `ChatRoomMembershipWindowTest` holds the boundary, including the same-second case and both directions of the `createdAt` fallback. `JoinedGroupAtMigrationJvmTest` runs the migration's own SQL against v14's three tables and covers what it must not take as carefully as what it must: a placeholder for an event from *after* the join is left to be recovered, a message that was read is left alone however old it is, a line with no group event behind it is out of reach of the rule, and two rooms joined at different times are each measured against their own join. `MarmotPreJoinIndexingJvmTest` drives the DAO against a room with no MLS state, which is what separates "left alone because it predates the join" from "tried and failed". 520 jvm tests and 302 android unit tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
ea11e8b233 |
refactor: call it a chronicle, and keep "archive" for what a user does to a chat
Archiving a chat is an ordinary thing a user will want to do to a conversation, and it is not this. This is the group's signed record, handed to a member who joined after the work was done so their room stops being empty. Two unrelated meanings of one word in one app is a bug waiting to be written, and `ChatRoom.archiveRequestedAt` is exactly where they would have met: a column on the chat row, named for the thing that is not the chat. So the whole feature is Chronicle now -- `press.mantra.compose.nostr.chronicle`, `ChronicleEvent` (30327), `ChronicleRequestEvent` (30328), the three tags, `ChronicleManager`, `docs/member-chronicle.md`. The kind numbers do not move; only the words do. **The wire tags move too**, `archiveId` -> `chronicleId` and `archivePage` -> `chroniclePage`, which is free exactly once. Both kinds are new and there is no old build to stay compatible with -- the design note says so in as many words -- so the alternative was carrying the old spelling on the wire forever to save a rename that costs nothing today. The recipient tag stays `p`; it was never ours. **Schema v14, because two things had the old word written into stored data.** `ChatRoom.archiveRequestedAt` becomes `chronicleRequestedAt`, renamed rather than dropped and re-added: while it is set it is the only record that a device with an empty room has already asked the group for its history, and a device that lost it mid-flight would ask again on its next launch, and the one after that. The three `ChatMessage.messageType` strings become their `chronicle*` spellings, rewritten rather than left to a legacy constant the way `dkgApprovalNeeded` was. These lines cannot be regenerated -- a chronicle is announced once, when it is requested, sent and applied -- and an unrecognised type is not skipped by the transcript. It renders as an ordinary chat bubble, so "Caught up on 12 items" would come back attributed to a member as something they said. `MIGRATION_13_14` does both, because Room can rename a column and cannot rewrite rows in the same breath. `ALTER TABLE ... RENAME COLUMN` needs SQLite 3.25, which `getRoomDatabase` guarantees by pinning `BundledSQLiteDriver`, and the column is in no index, no foreign key, and there is not a view or trigger in the database -- so nothing has to move with it. Five tests hold the two halves apart: the value survives, the column keeps its position, a room that never asked still reads as never having asked, the three types are rewritten, and every other type is left alone. **`isArchivable` is `isChroniclable`**, on the "recyclable" pattern, and it keeps its job unchanged: the allowlist that stands between a replayed `GroupKeyStateEvent` and the apply path. No behaviour change beyond the rename. 797 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
8f9e4de82e |
feat: keep what the group signed, and the path it signed as
A quorum signing something is the most expensive thing this app does and, until now, the least recorded. `FrostSigningManager.complete` verified the signature, handed the event to `ChatMessage.applyInnerEvent`, and let it go. What survived was whatever the row it became happened to keep -- an artifact keeps its `signature` and `publicKey`, a translation contributor list keeps nothing at all because that arm is still a TODO, and a kind this build has no arm for keeps nothing anywhere. The signature is the group's statement; the rows are one reading of it. `GroupSignedEvent` is where the statement itself now lives, at schema v13 behind an `AutoMigration(12, 13)`. **The columns are `NostrEvent`'s, not a summary of one.** `id`, `publicKey`, `kind`, `tags`, `content`, `signature` and the event's own `created_at` as `createdAt`, so what is stored is an event rather than a description of one. That is what makes `verifies()` answerable from the row alone: it delegates to `GroupKeyStateEvent.isSignedByRoom`, which asks whether the author is the room, whether the id is the hash of the fields sitting next to it, and whether the signature checks out. No ceremony, no key state and no path have to be on hand first -- which is exactly the position a member added after the ceremony is in. **The derivation path is the point of the exercise.** `publicKey` is the group's threshold key walked to `derivationPath`, and for a room that walk is also the room id -- see docs/shared-key-derivation.md, where those are one value. Without the path there is no way back from a signature to the ceremony behind it: a threshold key alone does not say which of a group's rooms signed, and a room id alone cannot be walked backwards. `GroupKeyState` records the path for the room; this records it for the event, so an event stays checkable after the room's state is gone or was never known. Null means the untweaked threshold key, the same meaning it carries on `FrostSigningSession.derivationPath`, which is where the signing path is copied from -- resolved from the room by `signingPath`, never from a proposer. **Two writers, and both file only what they have already checked.** `FrostSigningManager.recordSignedEvents` files a whole batch in one write, after every item's signature has verified and before any of them is applied -- a session's events are one decision by one quorum, so half a batch on file is a state no reader should have to reason about. `ArchiveManager.applyPage` files each payload it accepts, after the allowlist and `GroupKeyStateEvent.isSignedByRoom`, reading the room's path once per page from `GroupKeyState` rather than once per payload. Neither failure is the caller's: recording throws are logged and swallowed, because a ceremony that succeeded must not be reported as failed over a row this device could not write down. **The archive half is what makes a recipient more than a dead end.** A member handed their history used to end up holding the rows and none of the events -- able to read the group's work, unable to prove any of it, and unable to build a page for the next member to arrive. Now the events land too. **`record` merges rather than overwrites, and that direction is deliberate.** The same event reaches a device twice by design: once when the session that made it completes, once from any archive page carrying it. The second arrival is the poorer one -- an archive knows no session, and on a member who joined after the ceremony no derivation path either -- so the incoming row fills gaps and never empties them. The event's own fields are not merged because they cannot disagree: the id is the hash of them, so two rows under one id either hold the same event or one of them is not the event it claims to be. **Every `Mantra*` row points back at it.** `groupSignedEventId` on all twelve entities that carry `marmotGroupEventId`, stamped by `ChatMessage.applyInnerEvent` through a new defaulted parameter. On a group-signed row it is the only provenance there is: both Marmot ids are null, because there is no group event and no inner event behind one -- a signed event authored by the threshold key cannot travel as an inner event at all, since the outbound pipeline re-authors rumors as their sender and would strip the signature off. The column is only set when the record actually landed, so a row never points at an event that is not there. **`ArchiveManager`'s own doc said something that is no longer true.** It opened with "signed events are not stored as events", stated as present-tense fact and load-bearing for the paragraph under it. Corrected there and noted at the head of the same section in docs/member-archive.md, which is a phase history and so gets a note rather than a rewrite. Assembly still rebuilds payloads from rows via `toXEvent()` and the round-trip gate still holds it up: a room whose work predates v13 has no events on file, and rebuilding is the only way to reach it. Reading assembled events from the table is worth doing once that fallback can be dropped. **Two things this deliberately does not touch.** `ChatMessage` gets no such column -- it is not a `Mantra*` row and already carries `frostSigningSessionId` for the lines that need to name a session. `MantraTranslationChunkProposal` has a `marmotGroupEventId` but is not a `@Database` entity and nothing in `composeApp/src` references it, so it was left as the dead code it is rather than grown a column. Rows are not backfilled by the migration. The events they came from are gone, and minting an id for one would point a row at a signature nobody can produce; null reads as "this device does not hold the event behind this row", which is true of every row written before today. 490 jvm tests and 297 android unit tests pass. `GroupSignedEventDaoJvmTest` is eight cases against a real 2-of-3 quorum rather than a stub signature, because a fake one would satisfy every column assertion and prove nothing -- it covers the round trip, the path walking back to the row's own author, the merge in both directions, batch ordering, and a row edited after the fact no longer verifying. `SignedGroupKeyStateTest` adds the end-to-end claim over two devices: a batch of three signed in one session lands as three events on both, each at `m/9420/0/0` that neither device was told and both derived from the room they stand in. `ArchiveApplyJvmTest` asserts the receiver ends up holding the events and not only the rows, and that the four forgeries in its adversarial page become no signed-event rows either -- a forgery filed there is one the recipient goes on to hand to everybody else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
873203e4b4 |
feat: let a member with none of the group's work ask for it
Phase 5 of docs/member-archive.md, and the half that makes the whole thing reliable. A device opening a room it holds no signed work for asks the group; any member holding the work answers, addressed to whoever asked. **A request cannot lose the race a push loses.** Pushing an archive at an invitee is an application message in the epoch the add created, and one that overtakes the Welcome is dropped rather than deferred -- silently, while the inviter sees a success. That is marmot-membership.md's failure mode arriving in a new costume. Sending a request cannot lose it, because being able to send one is the proof it was won: a device that can put an application message into the room has processed its Welcome and is at the group's epoch. It also covers three things no invite-time push reaches, and they are answered by one rule because they are indistinguishable from inside the database: a member added after the work, a reinstall whose invite is long past, and a second device that was never invited at all. Hence the crude condition -- no dialects and no artifacts -- rather than anything that tries to tell them apart. **Anyone may answer and nobody is elected to.** A duplicate answer costs bandwidth and nothing else: pages are idempotent and every member who is not the named recipient ignores them. So the stand-down that would avoid the waste is an optimisation to add later rather than a correctness gap to close now. A member with nothing signed answers nothing at all, which is the honest reply from one still catching up themselves, and beats an empty archive that looks like an answer. **Queued with no chat line**, the way a signing message travels. Broadcast does not depend on one -- `encryptAndSendMarmotInnerEvent` inserts its `BroadcastNostrEventRequest` unconditionally, which is what let the FROST rounds travel with no transcript -- and an archive that filed a line per page would put a row of envelopes in the room's history. One line per archive is the right number and it is not writable from here, since the pages are indistinguishable from each other at this point. **Schema 11 -> 12**: `ChatRoom.archiveRequestedAt`, nullable, so Room generates the migration. It stops a device asking again on every launch while an answer is in flight. Rooms written before it read back null, meaning "never asked", which is true of all of them and harmless. Cleared as soon as an archive applies anything -- not when a sender's page count claims the archive was complete. A page count is the sender's word about the transfer rather than about the group's record, so a member who left work out must not get the last word on whether to ask again. A partial answer is followed by another request rather than by silence. The trigger is opening the room, through `ChatRepository` rather than from the view model into the database. Cheap to call every time: it stops at a room that already holds work and at one still waiting. A failure is not reported, because nothing acknowledges a request and the next open asks again. Eight tests: a device with nothing asks and does not ask twice, a device with work does not ask, answering queues pages addressed to the asker with every archivable kind in them and no chat line, a member with nothing signed answers nothing, a member does not answer themselves, an applied archive clears the stamp so a partial answer can be followed up, and the whole round trip -- ask, answer, apply -- leaves the joiner holding the sender's rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
4de87edf12 |
fix: tell one proposal's transcript lines from another's
A room with two proposals open showed "Review" on both, and then dropped it from both the moment either one was decided. The second proposal was still waiting on the reader, still had a decision in it, and had nowhere left to be reached from. Two proposals at once is not a corner case any more: a chapter and the translation scaffolding beside it are proposed as separate sessions, on purpose, and they run at the same time. Both write the same line types into the same stretch of transcript. `answeredRequests` matched a request against any later line of the fulfilling type, and `settledRequests` against any later ending. That reads a room signing one thing at a time exactly right -- the nonce after the request is the answer to it, because there is nothing else it could be an answer to -- and a room signing two things at once exactly wrong. Nothing else on the row could separate them: same type, same room, same minute, and `ChatMessage` carried no session. So the session goes on the row. `ChatMessage.frostSigningSessionId` is nullable, added as schema v11 through `AutoMigration(10, 11)`, and stamped by `FrostSigningManager.announce` -- the one place every FROST line is written, so there is no line that can be forgotten. Both rules read it when both rows have one and fall back to the clock when either does not. The fallback is not a compromise, it is the right reading of the rows it applies to. A line written before this column has no session and never will, and the rooms that wrote those lines could not run two sessions at once, so the clock is the whole truth there. A ceremony line falls back too and always will: a room runs one ritual at a time, and a DKG step is either taken or still waited on. **This reverses a call `FrostSigningRoute` argued for.** Its note said a chat row carrying a session id was "a poor trade for a lookup the screen can do". That was right when the lookup could only be wrong about which of one session it meant. The batch work made two sessions ordinary, and the lookup and the rules both became guesses at the same moment. A column on the table every message uses is the cost; two proposals, one of them unreachable, was the alternative. **Tests.** Three in TranscriptRequestStateTest for what the column buys: a nonce answers its own session's request and not the other's, one session completing settles nothing in the other, and a line naming no session is still read by the clock. TranslationBatchProposalJvmTest proves the other half against a real two-session proposal -- every FROST line the manager writes names its own session, and neither session's lines are attributed to the other. The rule is tested on rows and the stamping is tested on a database, because a rule that is right about rows nothing writes correctly is worth nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
dff41d417d |
feat(frost): move a signing session's per-event columns onto FrostSigningItem
Phase 1 of docs/frost-batch-signing.md, which is added here as the plan the next phases follow. Schema only: a session still signs exactly one event, the wire is byte-identical, and every existing test passes on the moved columns. ## What moved, and why it had to A batch of k events is k independent FROST instances sharing a signer set, not one signature over k messages. That is forced rather than chosen: a Schnorr partial signature is `s = k + e·x` with `e = H(R‖P‖m)`, so two messages under one nonce R give two equations in one unknown and the secret share falls out. So the five columns that enter that equation -- unsignedEventJson, eventId, nonceRandom, aggregatedNonce, signature -- move to a child table keyed (sessionId, itemIndex). What stays on FrostSigningSession is everything outside it: the ceremony, the threshold, the derivation path, the signer set, and the one approval. itemIndex is protocol rather than presentation -- nonces and partial signatures are joined positionally against it -- so getItems() orders by it and nothing re-sorts. Spelled itemIndex rather than index to keep hand-written queries free of backticks. No itemCount column. The count is a COUNT(*), for the same reason signerIds is derived from the ceremony's participant order rather than stored: a denormalised count is one more thing that can disagree with the rows. ## Migration 9 -> 10 Manual, not auto: Room can create the table and drop the columns but cannot copy between them, and the copy is the whole point. A session in flight at upgrade holds its nonce seed and the aggregate it is already signing against, and neither can be regenerated -- losing either makes the next pass derive a different nonce for the same message and publish a second partial signature over it, which is the extraction case. Both are copied verbatim into item 0, so an in-flight session resumes as though nothing happened. Removing the columns uses ALTER TABLE DROP COLUMN rather than the usual create-copy-drop-rename rebuild. FrostSignerMessage and FrostSigningItem both reference FrostSigningSession(id) ON DELETE CASCADE, and DROP TABLE fires cascades -- with foreign keys enforced the rebuild would delete every signer message and every item just written. Whether it does depends on Room disabling foreign keys around migrations, which is not worth depending on when DROP COLUMN cannot go wrong. It needs SQLite 3.35 and unindexed, unconstrained columns; these five qualify, and getRoomDatabase pins BundledSQLiteDriver on every platform. ## Invariants established here for the phases that follow - signerIds and every item's aggregatedNonce are one write-once unit, applied by applyAggregate() -- items first in one transaction, then the session, so "some items aggregated" is unreachable and signerIds != null stays the gate. - Signatures likewise, via applySignatures(); isSigned() counts rows instead of reading a flag. - complete() verifies every signature before applying any event, so a batch is all-or-nothing rather than half-filed. - itemsOver() gives each item its own 32 bytes of seed. Independent seeds mean an off-by-one in index handling produces a session that fails to aggregate rather than one that signs two messages under a single nonce. signedEvent() and isAwaitingApproval() now take the item(s) rather than the session, which propagates to the repository, the view model and the screen. advance() reads items.first() and Phase 2 turns that into a loop. ## Tests - FrostSigningSessionDaoJvmTest: index ordering, single-item read, upsert replacing rather than accumulating, signed-item counting, cascade delete. - FrostSigningItemMigrationJvmTest (new): the backfill against a real v9 database, asserting the seed and aggregate values survive -- not merely that a row appeared -- plus the exact column lists Room will check at open time. - 338 jvmTest and 217 testDebugUnitTest pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
a8f6638325 |
feat: sign a room's key state into being, at the room's own key
Two changes that turned out to be one. A room's key state stops being something its creator announces and becomes something the group signs, and every FROST signature moves from the group's root threshold key to the key derived at the room's own path -- which is the room's id. The second is what makes the first worth having: a key state is now signed by the very key it names. Supersedes the announcement introduced in |
||
|
Kgothatso Ngako
|
bcdfd2ec94 |
Merge branch 'mantra' into claude/marmot-direct-message-type-7a0473
Twenty-two commits had landed on mantra since this branch left it, several of them in the same files. Merged this way round so mantra stayed untouched until the result compiled and its tests passed. The migration had to be renumbered, and this is the conflict that mattered. mantra is at database version 7 and already has its own 5.json -- for MarmotInnerEvent.payloadEventId, nothing to do with direct messages. This branch had also written a 5.json, for a different schema. Resolved by restoring mantra's 5.json untouched and moving the direct message columns to an AutoMigration(7, 8) with a regenerated 8.json. Taking either 5.json over the other would have left every device validating a migration chain against a schema it was never built from; keeping version = 5 would have made a v7 install refuse to open at all. The regenerated 8.json is two ADD COLUMNs and nothing else, same as before. fromGroupEventResult was restructured on mantra: the kind switch moved into applyInnerEvent, and a SubmissionEvent envelope now wraps nip30303 payloads. Took that structure and re-applied the direct message branch ahead of it rather than inside it -- a gift wrap is not a nip30303 payload to apply, and what happens to it depends only on whether this device's key opens it, so it does not belong in a function about applying submissions. The isUserMessage fix was re-applied to the eight call sites mantra's version has, up from the six it had here. ChatMessageListViewModel and ChatRoomMessagingScreen took mantra's versions with the composer state, the two renderings and the reply action layered back on. docs/README.md keeps both new rows and mantra's closing note about the skipped-keys document. 108 tests pass, up from 50 here and 83 on mantra. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
a909108300 |
feat: announce which key a room signs with, instead of rederiving it
A signer holds a different secret share under every ceremony it took part in, and signing with the wrong one produces a partial signature that cannot aggregate. Nothing said which was which: FrostSigningManager found a room's key by walking every ceremony this device holds a share for and rederiving each one's room id until one matched. That search can only find rooms derived at the one path the constant names. SharedKeyDerivation.parsePath was written to lift that limit and was never called, so a room derived anywhere else was invisible to signing. So the coordinator now says it. GroupKeyStateEvent (kind 30326) carries the threshold public key, the ceremony that made it and the path the room's id came from, posted into the room as its first application message and filed as a GroupKeyState row. completedKey reads that row first and follows it to the share. Nothing secret travels. Every member of the room can read the event, so a share on it would be each member holding everyone else's -- a 1-of-n key wearing a t-of-n's clothes. The event names the ceremony; the share stays in DkgSession.secretShare on the device that generated it. The coordinator is untrusted, as everywhere else in the ceremony, so a state is verified rather than believed: the room's id *is* the threshold key derived at the path, and one that does not rederive its own room is dropped. That is the same guarantee the rederivation gave, kept rather than traded for a lookup. The old scan stays behind it for rooms that predate the table. Announced after the members are added, which is the only order that works -- adding them commits a new epoch and MLS will not let a member read what was encrypted before the one they joined at. A member invited later still misses it and falls back to the scan, which is where every member was before this existed. Replacement is this app's job. These are rumors inside a Marmot group event, so no relay applies the 3xxxx rule, and the DAO keeps the newest announcement per room so a backfill cannot walk a room backwards. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
b4ac65f5c9 |
feat: sign a nostr event with the group's shared key
A ceremony leaves every member holding a share of a t-of-n key and no way
to use it. This is the other half: a session that turns an unsigned nostr
event into one signed by the group.
The shape is ChillDkgRitualManager's, deliberately. The member who
proposes coordinates, protocol messages travel as gift-wrapped rumors on
the same NIP-17 pipeline chat messages use, each inbound message is
persisted and then the session is asked whether it can move, and every
step is recomputed from stored inputs so a device killed mid-round
resumes on the next message. Anyone who has read that manager can read
this one.
proposer --[ 30320 proposal ]-> everyone the unsigned event
signer --[ 30321 nonce ]-> everyone this device's public nonce
proposer --[ 30322 signer set ]-> everyone who signs, and their aggregated nonce
signer --[ 30323 partial ]-> everyone this device's partial signature
proposer --[ 30324 signature ]-> everyone the finished 64-byte signature
anyone --[ 30325 failure ]-> everyone abandon + blame
Three things are genuinely different, and each is why this is a separate
manager rather than another branch of that one.
**It does not need everybody.** A DKG cannot finish until every member
takes part; that is what makes the key. Signing needs t, and waiting for
n would throw away the property the group ran a ceremony to get. So the
coordinator waits for the threshold to be reachable, picks a set and says
who is in it. Members left out do nothing and stall nothing.
**Restart-safety is forced rather than chosen.** SecretNonce cannot be
serialised and refuses to be used twice, so storing the randomness it
derives from and regenerating on demand is the only way a session
survives the app closing. That is safe for exactly one reason: a session
signs one message and cannot be made to sign another. Two rules hold it
in place and both are load-bearing rather than tidy:
- the event id is written at creation, and a proposal that disagrees
with it is refused rather than applied;
- the aggregated nonce and signer set are write-once. A coordinator
that sends a second, different set is ignored. Obeying it would mean
two partial signatures over one secret nonce against two challenges,
which is precisely how a secret share is extracted. The session
stalls; the share does not.
**One approval, not three.** A DKG asks three times because each step
publishes something different and commits the member to something
different. Here every step serves one decision -- sign this event or do
not -- and the event is fixed before the member is asked, so a second
prompt would be the same question twice. Declining is broadcast rather
than silent: a t-of-n group can sign without you, but only if it knows.
Two things are checked rather than trusted, both because the coordinator
is untrusted by construction: the event id is recomputed from the
proposal's own fields, so a proposer cannot have the group sign one thing
while showing them another; and the finished signature is verified before
the session is called complete, so a bad aggregate is a failure here
rather than a rejection at every relay it reaches.
Signer ids are derived, not stored: a member's FROST id is their index in
the bytewise sort of the ceremony's host keys, the same ordering ChillDKG
hashed into the session identity and the same one the public shares are
in. Deriving means signing cannot disagree with the ceremony that made
the key.
DkgSession gains publicShares, kept because FROST validates each signer's
secret share against its public one. A ceremony finished before this
column reads back null and signing runs without that check rather than
refusing.
The tests run the same calls in the same order against real FROST and
assert the aggregate verifies as a nostr signature. That path was written
from reading the library rather than from a working example, so it is the
part most likely to be subtly wrong -- and wired up wrong it fails
silently, on every device.
Kinds start at 30320 with a gap. The DKG runs 30310-30316 and the
nip30303 document kinds run 30300 up; those two already collide at 30310
and 30311, and SubmissionEvent sits on 30312, which is also the DKG's
round-1 kind. They are kept apart today only by riding different
transports, which is luck. Signing shares a transport and rooms with the
DKG, so it starts clear of both.
No UI yet: this is the session logic, reachable through proposeSigning,
approve and decline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
fcc28de931 |
Revert "fix: hold a payload whose parent has not arrived instead of losing the event"
This reverts commit
|
||
|
Kgothatso Ngako
|
d7aac49cf1 |
fix: hold a payload whose parent has not arrived instead of losing the event
A receiver hit `FOREIGN KEY constraint failed` on an artifact submission and lost the whole group event. The artifact referenced a dialect the receiver did not have, MantraArtifact.dialectId is a foreign key, and SQLite answers a violated constraint by aborting -- which rolled back the entire transaction the inbound pipeline runs in. Gone with it: the NostrEvent, the MarmotGroupEvent, the submission's MarmotInnerEvent holding the payload verbatim, and the transcript line. Nothing retries, so the artifact stayed lost even once the dialect turned up. Every nip30303 entity is a child of another and the schema enforces all of it -- artifact→dialect, version→artifact, chapter→version, chunk→chapter, translations→both of theirs -- so this was every branch, not one. And submissions make arriving before your parent ordinary rather than exotic. That is the point of them: an admin submits a backlog in whatever order they hold it, and a member who joined last week can be sent what the group was told last month. Both produce payloads whose parents are not here yet, and both were losing data. So check the parents before inserting. A payload that arrives early is held on the submission row -- awaitingEventId names what it waits for -- and applied when that arrives. Releasing one can release another, a version freeing its chapters and those freeing their chunks, so it walks outward until nothing more comes unstuck. A payload with a second parent still missing is re-pointed at that one rather than retried on every arrival. Nothing is written to the transcript while a payload is held. Nobody has said anything yet; the line appears when it is applied, in the position its own timestamp gives it. Two things fall out of the shape: parentRefsOf is pure and separate from the lookups, because the mapping is the part that can silently drift from the schema and there is no database harness in commonTest to catch it. ParentRefsTest pins one case per kind. Which table an id lives in is carried as the kind of event that would have created it, so there is no second enum to keep in step. applyInnerEvent takes ids rather than a GroupEvent, since replay happens long after that object is gone. A released payload is recorded as not ours: we hold the parents of anything we wrote, having written those too. Also reconstructs a held bare nip30303 event from its own columns rather than parsing its content as an event -- only submissions carry an event there, and reading both that way would have stranded every bare one permanently. Verified: the v5→v6 migration runs clean on the receiver's real populated database. The hold path itself still needs a fresh submission from a sender to exercise end to end. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
ce77b77240 |
feat: apply the nip30303 event a submission carries, keeping its author
Teach the receiving side to open an envelope before anything starts
sending one. In that order a client that has this can already handle
submissions from a client that does not yet send them; the reverse would
turn every artifact, dialect and chapter into an "unsupported" row for
anyone who had not updated.
Despite the name, MarmotInboundManager does not dispatch on inner-event
kinds -- it decrypts MLS and hands back a GroupEventResult. The kind
dispatch has always lived in ChatMessage.fromGroupEventResult, so that
is where support for a new kind goes.
The `when (event.kind)` body becomes applyInnerEvent, which takes the
event to apply separately from how it arrived:
event the nip30303 event, written by whoever wrote
it -- possibly nobody in this group
marmotInnerEventId the row the group actually delivered
senderPublicKey the member who delivered it
createdAt when they did
For a plain nip30303 event those all come from the one event, which is
exactly the old behaviour. For a submission they come from the envelope
while `event` is the payload. Entity rows take their author from the
payload via fromXEvent, so the chat line says who added something and
the row says who wrote it -- the point of the envelope, made real at the
only place it can be.
createdAt deliberately follows the envelope rather than the payload: a
submitted archive translation can be years old, and sorting the group's
transcript by when the source was written would file "X added a
translation" somewhere nobody will scroll to.
The stored MarmotInnerEvent stays the outer event -- that is what the
group sent -- and gains payloadEventId naming what it carries. The
payload is not given a row of its own: it is recoverable from the
submission's content, and a second row with a null marmotGroupEventId
would look to the outbound pipeline like something waiting to be sent.
Nullable column, so AutoMigration(4, 5) is all it needs; rumors queued
before this read back null, which is correct, since none of them were
submissions.
Two submissions are stored but not applied, because there is nothing in
them to make a row from: one whose payload will not parse, and one
carrying another submission. Both surface as "unsupported" rather than
disappearing.
The unsupported fallback also stops attributing to groupEvent.pubKey,
which is the ephemeral key every kind:445 is signed with and so names
nobody.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
296011dcd5 |
feat: give a queued message somewhere to say who it is private to
Two nullable columns and the v5 migration that adds them, ahead of the code that fills them, so the schema lands on its own and can be reverted on its own. MarmotInnerEvent.directMessageRecipientPublicKey is the outbound signal. The notary reads a queued row and has no other way to know a message is meant for one member rather than the room -- the plaintext is identical either way -- so this is what routes it into the gift wrap path. Inbound rows leave it null on purpose: the recipient is on the wrap's `p` tag, which is where every member reads it from, so a second copy on the row would be a second thing that can disagree. ChatMessage.directMessageRecipientPublicKey is what the transcript reads. Both lines a direct message can produce need it -- the one its two parties see, and the "sent a private message to Bob" line everybody else gets -- and holding it on the row keeps the view model off a join for a fact it already has to render. MarmotInnerEvent hand-writes equals and hashCode over every field, so both are extended too. A field missing from those is not a compile error and not a test failure; it is two rows that differ comparing equal, which surfaces much later as an upsert that does nothing. Room generates the migration -- verified as two ADD COLUMNs with no table rebuild, so nothing is copied and nothing can be dropped: ALTER TABLE `ChatMessage` ADD COLUMN `directMessageRecipientPublicKey` TEXT DEFAULT NULL ALTER TABLE `MarmotInnerEvent` ADD COLUMN `directMessageRecipientPublicKey` TEXT DEFAULT NULL Rows written before this come back null, which reads as "not a direct message" -- the only answer that is true of all of them. v5 is an AutoMigration entry rather than a hand-written Migration like MIGRATION_3_4 next to it, because that one rewrote data without changing shape and this one changes shape without touching data. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
cae50ce359 |
feat: hold the ritual until its owner approves each step
The ChillDKG ritual ran entirely on its own. `acceptProposal` published this
device's host key the moment a PROPOSAL arrived from a relay, and `advance`
published rounds 1 and 2 as soon as their inputs landed. Receiving a nostr event
was therefore enough to enrol the owner of a phone in a group's permanent signing
quorum, without anything having been shown to them first.
Nothing of this device's own now goes out before its owner says so. Three
approvals, because each publishes something different and commits the member to
something different:
host key joins the ceremony, and fixes n. A member who joins and then stops
answering does not merely fail to help -- the ritual cannot finish
without every member, so they hold it open for everybody.
round 1 contributes to the key itself. The member's own secret material
starts shaping a key they will be expected to help sign with.
round 2 confirms the coordinator's combined result matches what this device
sent. A check rather than a formality: it is what stops a coordinator
substituting a key the members never contributed to.
The coordinator's two aggregations are deliberately not gated. They relay other
members' already-published messages and disclose nothing of the coordinator's own,
so an approval there would stall the whole group on one person's attention without
protecting anybody. The member who opens a ceremony is auto-approved for the host
key alone -- starting one is already the act of agreeing to be in it -- and is
still asked for rounds 1 and 2, which publish key material.
Each gate returns rather than throwing. The ritual is not failing, it is waiting
on a person; everything already received stays stored, so it resumes the moment
they approve. `pendingApproval` mirrors those gates exactly and has to keep doing
so: if the two disagree the screen offers an approval that does nothing, or none
while the ritual sits still.
Schema v2 -> v3 adds four nullable columns to DkgSession -- three approval
timestamps and `approvalRequestedThrough` -- so Room generates the migration. A
ritual already in flight comes back with all three null, which reads as "not
approved yet" and simply asks, rather than silently continuing.
## Being asked
Three screens rather than one parameterised by step, because each is making a
different case and the copy is the substance of the screen, not decoration around
it. They share a scaffold for one reason that is not cosmetic: a screen opened for
one step can go stale -- a redelivery carries the ritual forward, or the member
approves on another device -- so it re-checks the pending step before offering a
button, and `approve` checks again in the manager and ignores a mismatch.
"Not now" does not refuse on the member's behalf. There is no "no" in ChillDKG
short of abandoning the ceremony, and quietly leaving is what a member who is not
ready actually wants; abandoning stays on the ritual screen where the consequence
can be spelled out.
A chat line announces each request, written once per step and guarded by
`approvalRequestedThrough` -- `advance` runs on every arriving message and would
otherwise ask again on each one. It is the one ritual line that asks rather than
reports, so it is the one that is not quiet: primary tint, a Review affordance,
and a tap through to the ritual screen, whose bottom bar routes to the step the
ceremony is actually waiting on.
## Telling the steps apart
The request started as a single message type, which meant one icon for all three
and no way to tell "join the ceremony" from "confirm the key". The type is the
only thing a transcript keeps -- a line drawn days later has no session to ask
what was being requested -- so the step moved into it, one type per step, and
every stage now carries its own icon.
MIGRATION_3_4 rewrites the rows already written. They cannot regenerate: a request
is announced once, so a ceremony already in flight would keep its undifferentiated
icons forever. It changes no schema at all -- the version bump exists only to give
a data rewrite somewhere to run, which is why it is a manual migration on the
builder rather than another AutoMigration. Rows it cannot match keep the old type,
which the renderer still recognises.
An answered request shows a checkmark where Review was. Whether it was answered
comes from the transcript rather than the session: approving is the only thing
that causes the step to be published, and publishing writes an authored line, so a
matching line at or after the request means done. That keeps a room that has run
more than one ceremony correct -- ChatMessage has no session id to disambiguate
with -- and needs no DkgRepository in the message list. The comparison is on
createdAt rather than list position, because the list is ORDER BY createdAt DESC
with reverseLayout, where index arithmetic runs backwards.
Compiles and assembles; the ordering test still passes. No ritual has been run on
a device.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
3995cdefc6 |
feat: run a ChillDKG ritual over a NIP-17 group
Robust groups can now generate a FROST threshold key together. The
group's members are the participants, the room's creator is the
coordinator, and the whole protocol travels as gift-wrapped rumors on
the chat the group already has -- so there is no second transport to
build, operate or debug.
This is what the quorum has been reaching for since it was introduced.
Until now "t of n must approve" had no key to approve anything with;
ChillDKG produces one that no single member holds.
## Transport: seven rumor kinds (nostr/dkg/)
coordinator --[ 30310 proposal ]-> everyone
participant --[ 30311 host key ]-> everyone
participant --[ 30312 round 1 ]-> everyone ParticipantMsg1
coordinator --[ 30313 coord round 1 ]-> everyone CoordinatorMsg1
participant --[ 30314 round 2 ]-> everyone ParticipantMsg2
coordinator --[ 30315 certificate ]-> everyone CoordinatorMsg2
anyone --[ 30316 failure ]-> everyone abort + reason
These only ever exist inside a NIP-17 gift wrap, so no relay sees them
unencrypted and the replaceable semantics normally implied by the 3xxxx
range never apply -- which is why they can sit next to the app's other
private kinds (30300-30309) without meaning anything different.
Every message is addressed to the whole group, even the two the protocol
only needs the coordinator to read. NIP-17 wraps per recipient anyway,
ChillDKG treats the coordinator as untrusted by construction, and having
every member observe the ritual is what makes a progress UI possible
without a side channel.
`DkgSessionIdTag` is on every message: a group may abandon an attempt and
start another, and a straggler from the dead one must be dropped rather
than mixed into the live session. `DkgThresholdTag` rides the proposal so
every participant validates the same SessionParams -- disagreement on `t`
fails the session instead of quietly producing a weaker key.
## Persistence: inputs, not state (database/model/Dkg*, schema v2)
DkgSession deliberately stores no protocol state. Reading EncPedPop
confirms randomness enters the participant steps only through the passed
`random`/`auxRand` arguments (`simplSeed = taggedHash("encpedpop seed",
seed + random + encContext)`), so every ChillDkg step is a pure function
of inputs. Keeping the two 32-byte randoms plus the received messages is
therefore enough to recompute any intermediate state on demand, and the
opaque ParticipantState/CoordinatorState objects -- which have no
serialization API -- never need to be persisted at all.
That is not a micro-optimisation. A DKG cannot finish unless all n
members take part, and chat users close apps mid-round; recomputation is
what lets a ritual resume instead of forcing the group to start over.
DkgParticipantMessage is keyed (sessionId, participantPublicKey, kind) so
a redelivered message overwrites rather than accumulates -- relays
redeliver, and a duplicated round-1 message would hand the coordinator a
participant list of the wrong length.
Database goes to version 2 with an AutoMigration: v2 only adds tables, so
Room generates it. Schema 2.json is exported alongside.
## Driving it (managers/ChillDkgRitualManager.kt)
State machine driven entirely by arriving messages: persist, then ask
whether the ritual can move. Because every step is recomputable there is
no long-lived session in memory to lose, and processing is idempotent --
a redelivered message re-runs a step that has already been taken and
changes nothing.
The coordinator is a participant too, so it records its own outbound
messages locally: its round-1 message has to be in its own aggregation
alongside everyone else's. Being the room's creator buys it no authority
here -- ChillDKG's coordinator relays but cannot learn secrets or bias
the key -- only the job of aggregating.
Two decisions worth knowing:
* Host keys are DERIVED, not reused. `sha256("mantra/chilldkg/host-key/v1"
|| nostrSeckey)`. Reusing the nostr identity key directly was the
simpler option, but one secret serving two protocols means a flaw in
either reaches the other. Deriving from the same seed keeps it
recoverable from the wallet backup, which matters because ChillDKG
needs the host secret key to recover a session's outputs and asking
chat users to back up a second secret is how keys get lost.
* Participant order is a bytewise sort of the host public keys. ChillDKG
fails outright if participants disagree on ordering, and a sort is the
only order every device can derive independently from the same set.
Any ChillDkg exception ends the session for this device and is broadcast
as a 30316 so the rest of the group stops waiting, rather than leaving
every member on a spinner that will never resolve.
## Inbound (database/dao/NostrDao.kt)
One branch on the existing decrypted-gift-wrap dispatch, beside the
kind-14 and WelcomeEvent branches, handing ritual kinds to the manager.
## UI (ui/.../DkgRitualScreen.kt + view model, state, route)
Reached from chat room detail via "Shared Key", shown only for rooms with
no MLS state -- i.e. the NIP-17/robust ones. An MLS room has a single
admin and no group key to share, so the entry point would be a lie there.
The screen is a ladder of rounds with real counts ("3 of 5") rather than
a spinner. The unusual thing about a DKG, and the thing the UI has to get
across, is that it needs *everyone* at once; a count says who it is
waiting on, an indeterminate spinner says nothing. The coordinator gets
the start button, everyone else is told who they are waiting for, and a
failed ritual states plainly that no key was created and it is safe to
run again.
DkgSession.threshold finally gives the quorum somewhere to live. The
value chosen during group creation is still not persisted on ChatRoom,
so this screen re-asks with the same majority default rather than
inventing a different one; there is a TODO where that gap closes.
Verified:
./gradlew :composeApp:compileCommonMainKotlinMetadata
./gradlew :composeApp:compileDebugKotlinAndroid
Not runtime-verified: exercising a DKG needs several devices exchanging
live messages, and the library's own vector suite needs JDK 21.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
3a830d77d6 | Add chat room id on mantra tables | ||
|
Kgothatso Ngako
|
3640a77413 | Add more associations | ||
|
Kgothatso Ngako
|
03c3e56b26 | Bug fix for translationChapter primary key | ||
|
Kgothatso Ngako
|
6cdddee424 | Rename TorchDatabase to MantraDatabase |