87ef129de5931e1e032f52ad21320ebf6e412efd
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Kgothatso Ngako
|
87ef129de5 |
fix: archive the translation that stands, not every draft of it
Follow-up to reading the archive out of `GroupSignedEvent` rather than rebuilding it from rows. The two sources do not hold the same thing, and one place where they differ reaches the archive. `ChatMessage.applyInnerEvent` supersedes a translation chunk: retranslating a passage changes the text and so the event id, so the arm drops the row it replaces -- newest by the timestamp the group signed at, id breaking a tie. The record does not, and should not: a signature is the group's statement and discarding one is not that table's business. So a passage translated three times leaves one row and three events. While the archive was rebuilt from rows that difference was invisible, because a sender simply had nothing but the group's current answer to each passage. Read from the record it is not: measured on a seeded room, one retranslation leaves one row and puts **two** payloads in the archive, and it compounds -- every draft a group ever signed would travel in every archive it ever sends, for as long as the room exists. **The rule is the applying arm's, restated rather than approximated.** An archive that shipped one translation as current while the recipient settled on another would have both validly signed and nothing downstream to notice they disagree, so `currentTranslationsOnly` groups by `(translationChapterId, chunkId)` and keeps the maximum by `(createdAt, id)` -- the same comparison, spelled the same way. Dropping the drafts is safe precisely *because* the recipient applies that rule too. This is not what keeps them correct; it is what stops them being sent work they would discard on arrival. Grouped per passage rather than per chapter, or retranslating one passage would take every other passage's translation with it. A translation naming no passage is left alone rather than lumped in with the rest: it is unappliable either way, and letting one stand in for a whole passage would let a malformed event suppress a good one. Three tests, and all three fail if the filter is removed: a retranslated passage leaves one row, two recorded events and one payload; three translations signed in the same second settle on the same id the row keeps, which is what pins the tiebreak to the applying arm's; and two passages each keep their own, which is what a group-by-chapter mistake would fail. Two documentation corrections alongside it. docs/member-archive.md said "a retranslated passage archives once" as a property of the rows, which stopped being true the moment the record became the source -- it now says what makes it true again. And `GroupSignedEvent.verifies()` said a false means the row's columns have drifted from the event they came from. That is the reading worth acting on and it is not the only one: a room never derived from its group's key signs as the bare threshold key rather than as its own id, so a perfectly good event there fails and cannot be made to pass, the key it would need being absent from the row and unreachable from one. 498 jvm tests and 297 android unit tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
a69d80d38f |
feat: archive the events the group signed, not rebuilds of its rows
`assemble` read the archive out of `Mantra*` rows, rebuilding each payload with `toXEvent()` and standing or falling on that rebuild being byte-identical to what was signed. It had to: nothing kept the events. `GroupSignedEvent` keeps them now, so `signedEventsOf` reads the record first and rebuilds only what the record does not hold. **The rebuild stays, as the fallback, keyed by id.** A room whose work predates v13 has no events on file, and dropping the walk would silently empty its archive -- the failure mode being that a member asks for the history, a member answers, and nobody notices the answer was blank. So both sources are read and unioned by event id, which is also what a half-upgraded room needs: older work only the rows remember, newer work on file, and neither half complete on its own. The fallback can go once no install still carries pre-v13 work, and `ArchiveRoundTripTest` is what holds it up until then. **The allowlist does real work on the way out now, and this is the part that would have bitten.** The rebuild could only ever produce document kinds, because those are the only rows it walks. The record holds every kind the group has ever signed -- and every room signs a `GroupKeyStateEvent` as its first act, so one is on file in every room that has signed anything at all. `ArchiveEvent.build` refuses a non-archivable kind with `require`, so an unfiltered read does not quietly ship a key state: it throws, and the room's entire archive fails on the one event every room has. `signedEventsOf` therefore filters on `isArchivable` before anything else, which is the same rule `applyPage` applies on the way in. Removing that one line fails two tests with exactly that exception, which is how I know they are load-bearing rather than passing for the reason I expected. **An artifact whose initial version row is missing now archives.** The rebuild has to recover the version label from that row -- `fromArtifactEvent` drops it, so it is not on the artifact -- and logs and gives up without it, which is a hole in the archive for any device that applied half a batch. Read from the record there is nothing to recover: the label never left the event. That is the case that makes the record the better source rather than merely the faster one, and it has a test of its own. **One verify filter over both sources**, because the rule is per event and not per source: nothing leaves that the recipient could not check for themselves. A drop still means different things on each side -- a member's own rumor sitting in the same table as the group's work, versus a row that has drifted from the event it recorded -- and the comment now says so, since the log line cannot. **Ordering is unchanged where it matters and looser where it does not.** `inApplyOrder` is a stable sort by dependency rank, so the union only affects order *within* a rank: a room holding some work both ways can order two chapters differently from a member holding one way only. Pages are idempotent and applied payload by payload, and two members already differed by the order their rows were written in, so this costs nothing. `rebuiltEventsOf` still runs on every archive even where it contributes nothing, because there is no way to tell a complete record from a partial one without doing the walk, and it is a handful of indexed queries against a room's own rows. 495 jvm tests and 297 android unit tests pass. Five new cases in `ArchiveAssemblyJvmTest`, which seeds through the real inbound path and now records the same batch the way `FrostSigningManager.complete` does: payloads compared byte-for-byte against what was signed, work held both ways travelling exactly once, a genuinely room-signed key state left behind, a signed kind the archive has no arm for left behind, and the artifact the rebuild has to leave out archiving from the record. The existing assembly and end-to-end tests seed without recording, so they go on covering the rebuild fallback unchanged -- which is why they all still pass, and why that is evidence rather than luck. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
8f9e4de82e |
feat: keep what the group signed, and the path it signed as
A quorum signing something is the most expensive thing this app does and, until now, the least recorded. `FrostSigningManager.complete` verified the signature, handed the event to `ChatMessage.applyInnerEvent`, and let it go. What survived was whatever the row it became happened to keep -- an artifact keeps its `signature` and `publicKey`, a translation contributor list keeps nothing at all because that arm is still a TODO, and a kind this build has no arm for keeps nothing anywhere. The signature is the group's statement; the rows are one reading of it. `GroupSignedEvent` is where the statement itself now lives, at schema v13 behind an `AutoMigration(12, 13)`. **The columns are `NostrEvent`'s, not a summary of one.** `id`, `publicKey`, `kind`, `tags`, `content`, `signature` and the event's own `created_at` as `createdAt`, so what is stored is an event rather than a description of one. That is what makes `verifies()` answerable from the row alone: it delegates to `GroupKeyStateEvent.isSignedByRoom`, which asks whether the author is the room, whether the id is the hash of the fields sitting next to it, and whether the signature checks out. No ceremony, no key state and no path have to be on hand first -- which is exactly the position a member added after the ceremony is in. **The derivation path is the point of the exercise.** `publicKey` is the group's threshold key walked to `derivationPath`, and for a room that walk is also the room id -- see docs/shared-key-derivation.md, where those are one value. Without the path there is no way back from a signature to the ceremony behind it: a threshold key alone does not say which of a group's rooms signed, and a room id alone cannot be walked backwards. `GroupKeyState` records the path for the room; this records it for the event, so an event stays checkable after the room's state is gone or was never known. Null means the untweaked threshold key, the same meaning it carries on `FrostSigningSession.derivationPath`, which is where the signing path is copied from -- resolved from the room by `signingPath`, never from a proposer. **Two writers, and both file only what they have already checked.** `FrostSigningManager.recordSignedEvents` files a whole batch in one write, after every item's signature has verified and before any of them is applied -- a session's events are one decision by one quorum, so half a batch on file is a state no reader should have to reason about. `ArchiveManager.applyPage` files each payload it accepts, after the allowlist and `GroupKeyStateEvent.isSignedByRoom`, reading the room's path once per page from `GroupKeyState` rather than once per payload. Neither failure is the caller's: recording throws are logged and swallowed, because a ceremony that succeeded must not be reported as failed over a row this device could not write down. **The archive half is what makes a recipient more than a dead end.** A member handed their history used to end up holding the rows and none of the events -- able to read the group's work, unable to prove any of it, and unable to build a page for the next member to arrive. Now the events land too. **`record` merges rather than overwrites, and that direction is deliberate.** The same event reaches a device twice by design: once when the session that made it completes, once from any archive page carrying it. The second arrival is the poorer one -- an archive knows no session, and on a member who joined after the ceremony no derivation path either -- so the incoming row fills gaps and never empties them. The event's own fields are not merged because they cannot disagree: the id is the hash of them, so two rows under one id either hold the same event or one of them is not the event it claims to be. **Every `Mantra*` row points back at it.** `groupSignedEventId` on all twelve entities that carry `marmotGroupEventId`, stamped by `ChatMessage.applyInnerEvent` through a new defaulted parameter. On a group-signed row it is the only provenance there is: both Marmot ids are null, because there is no group event and no inner event behind one -- a signed event authored by the threshold key cannot travel as an inner event at all, since the outbound pipeline re-authors rumors as their sender and would strip the signature off. The column is only set when the record actually landed, so a row never points at an event that is not there. **`ArchiveManager`'s own doc said something that is no longer true.** It opened with "signed events are not stored as events", stated as present-tense fact and load-bearing for the paragraph under it. Corrected there and noted at the head of the same section in docs/member-archive.md, which is a phase history and so gets a note rather than a rewrite. Assembly still rebuilds payloads from rows via `toXEvent()` and the round-trip gate still holds it up: a room whose work predates v13 has no events on file, and rebuilding is the only way to reach it. Reading assembled events from the table is worth doing once that fallback can be dropped. **Two things this deliberately does not touch.** `ChatMessage` gets no such column -- it is not a `Mantra*` row and already carries `frostSigningSessionId` for the lines that need to name a session. `MantraTranslationChunkProposal` has a `marmotGroupEventId` but is not a `@Database` entity and nothing in `composeApp/src` references it, so it was left as the dead code it is rather than grown a column. Rows are not backfilled by the migration. The events they came from are gone, and minting an id for one would point a row at a signature nobody can produce; null reads as "this device does not hold the event behind this row", which is true of every row written before today. 490 jvm tests and 297 android unit tests pass. `GroupSignedEventDaoJvmTest` is eight cases against a real 2-of-3 quorum rather than a stub signature, because a fake one would satisfy every column assertion and prove nothing -- it covers the round trip, the path walking back to the row's own author, the merge in both directions, batch ordering, and a row edited after the fact no longer verifying. `SignedGroupKeyStateTest` adds the end-to-end claim over two devices: a batch of three signed in one session lands as three events on both, each at `m/9420/0/0` that neither device was told and both derived from the room they stand in. `ArchiveApplyJvmTest` asserts the receiver ends up holding the events and not only the rows, and that the four forgeries in its adversarial page become no signed-event rows either -- a forgery filed there is one the recipient goes on to hand to everybody else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
47aa79ebc7 |
feat: archive the translated text too, now that the group signs it
The merge brought in two commits that close the gap this feature was written around, so the allowlist grows from six kinds to eight. `feat: sign an artifact's first version with it, not derive it after` makes the version the second item of the artifact's own signing batch. `feat: ask the group to sign a chunk's translation, not just save it` puts a quorum behind the prose. Both were done for their own reasons and neither was about the archive, but they are exactly what the archive was missing: an archive can only carry what its recipient can check, so a derived version and a member-authored translation could not travel. A new member got the whole structure and none of the words. **30301 and 30309 do not go on the end of the list.** The order is the foreign keys: a version sits between its artifact and the chapters hanging off it, and a translated chunk hangs off both a source chunk and a translation chapter, so that one really is last. **`toArtifactVersionEvent` had the bug this predicted it would.** It emitted [artifactId, alt] where `build` emits [alt, artifactId], so the id did not round-trip -- the same fault fixed on `MantraArtifact.toArtifactEvent` in Phase 3, in the second of the three unused rebuilds, and for the same reason: nothing had ever called it, so the "tag order matches build" claim in its comment was never checked. `toTranslationChunkEvent` was already correct. Both now have a round-trip case, which is what makes the difference between a rebuild that is right and one that has not been contradicted yet. **`signedEventsOf` walks two steps further**, emitting each version and the translation chunks under each translation chapter. A retranslated passage archives once: the arm that applies a translation chunk drops the one it supersedes -- newest by the timestamp the group signed at, id breaking a tie -- so what a sender holds, and therefore what travels, is the group's current answer to each passage rather than its drafts. **The seeds had to change with it.** Both database tests derived the artifact's first version by applying the artifact, which is exactly what stopped happening; they now sign it through `ArtifactVersionEvent.initialVersionOf`, the way the batch does. That also removes the one exception in the end-to-end assertion: every archived row is now authored by the room and carries a signature, where the artifact version used to have to be excused for having neither. 480 tests pass. The plan's Phase 3 table, its built-vs-plan table and its "what this does not do" section are updated -- what an archive cannot do is down from two things to one, and the remaining one is that it still cannot make its recipient able to sign. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
f3984e838c |
test: prove the catch-up row by row, and say what an old build makes of a page
Phases 8 and 9 of docs/member-archive.md. The tests ran in the phases where the code they cover first existed -- the way the batch-signing note's did -- so this is what was missing from them, plus the rollout note, plus the plan marked built. **Compared row by row, not by count.** The end-to-end test asserted the two databases held the same *number* of artifacts, chapters and chunks. That is not the claim: two databases can hold the same counts and disagree about every row, and a rebuild that lost the group's signature -- or re-authored a row as whoever sent it -- would pass a count and fail the only thing an archive is for. It now compares `(id, author, signature)` per row across every archived kind, and then asserts each one is authored by the room and carries a signature. The artifact version is the one exception, and it has to be: nobody signs it, it is derived from the signed artifact on arrival. Which is exactly why it is not archived, and why a chapter's foreign key survives without it. **An old build does not ignore an archive page, it renders it.** Phase 9's first draft said an old build "files it as unsupported, exactly as it does today for anything it does not know" -- true, and it reads better than it lives. An unsupported row's content is `event.toJson()` and it renders as an ordinary chat bubble, so every member on an old build sees each archive page as a raw-JSON bubble of up to `MAX_PAGE_BYTES`, once per page. Nothing breaks and nothing is lost, but a group mid-upgrade gets a genuinely unpleasant transcript, and that is worth knowing before the first archive goes out. So the rollout rule is stated rather than implied: the receiving half ships safely on its own -- phases 1-4 send nothing -- and no member starts sending until every member understands kind 30327. The mitigation if that ever proves unacceptable is the one the appendix rejects for other reasons, and it is named there so the trade can be weighed rather than rediscovered. **The plan is marked built**, with a table of the five places the implementation chose differently from the plan and why: nine archivable kinds became six, a count cap that could never fire, queueing moved a phase later, a re-read that was never needed, and the rollout note above. Phase 8 also records the three tests that were not in the first draft, each written because something passed for the wrong reason -- a cap that could not fire, an out-of-order test on an archive that was never out of order, and a sweep whose "still missing" count included failures a later pass had already fixed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
a315b86918 |
feat: say in the transcript that a member is being caught up
Phase 7 of docs/member-archive.md, in part. Three chat types -- `TYPE_ARCHIVE_REQUESTED`, `TYPE_ARCHIVE_SENT`, `TYPE_ARCHIVE_RECEIVED` -- so a room that fills itself in explains itself once. Without this the archive is entirely silent by design: it files no line per applied payload, because `ChatMessage` has an `autoGenerate` primary key and every payload would mint a fresh row on every pass of the sweep. The result was a member joining a working group and watching a room populate with no account of where any of it came from, which is worse than the noise it avoided. **One line per archive, not per page.** The received line is written when the request stamp is cleared, which is as close as this can get: an archive's pages are not distinguishable from each other at apply time, and clearing the stamp is exactly the moment a catch-up stops being pending. There is a test that delivers a payload per page, backwards, so the sweep runs repeatedly over many pages, and asserts the transcript holds two lines. **A push behind a Welcome writes nothing**, because the room was never asked. It lands before the member has opened the room, and "caught up on work you have not seen yet" is a line about nothing. Also tested. **The received line names no sender.** An archive can be assembled from pages sent by more than one member, so attributing the catch-up to one would be a guess dressed as a fact. The sent line does name its recipient, written into the content the way the invite line writes one -- which does not follow a rename, and is the accepted cost for a line about something that happened once. **Content is whole sentences**, so these stay out of the AUTHORED sets and nothing prefixes a name to them. And they are added to `ARCHIVE_TYPES` with a matching arm in the transcript, because the failure mode for a missed set is silent: the line renders as a chat bubble, looking exactly like a member having said "Caught up on 12 items". Icons per type rather than the `PanTool` fallback. **Two items from this phase are deliberately not done**, rather than written without the app in front of me: the banner saying a room is catching up, and a "Send history" action on the member row. The first is UI state plumbed through a view model into a layout and the transcript line covers the same ground; the second is a convenience, since both real paths are already automatic. Both are written up in the plan as outstanding, along with the thing this phase was also meant to say and does not: that an archive does not make its recipient able to sign, and does not carry the translated text. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
6d8866c2b0 |
feat: offer a new member the group's work alongside their welcome
Phase 6 of docs/member-archive.md, and deliberately the phase after the one that makes it unnecessary. `deliveryWelcome` now queues an archive for the member being invited, so in the ordinary case they have the group's signed work before they think to ask for it. **This is a latency optimisation, not the mechanism.** A page queued behind the Welcome is not delivered after it: they are different transports -- a relay-borne gift wrap and a kind:445 -- with no ordering between them, and a page that overtakes the Welcome is from an epoch ahead of the invitee's, so `MarmotInboundManager` drops it outright rather than deferring it. Nothing retries and the inviter sees a success. That is the failure in docs/marmot-membership.md wearing new clothes, and the only thing that closes it is the invitee asking once they are demonstrably in the group, which Phase 5 already does on their first open of the room. So nothing here reports failure to the inviter. A push that does not land is the ordinary case the pull exists for, and it sits inside `deliveryWelcome`'s own catch alongside the Welcome it rides behind. A room with nothing signed queues nothing and still invites. **One call, two occasions.** `ArchiveManager.answer` becomes `sendTo`: answering a request and pushing behind a Welcome are the same operation and differ only in who decided, so it is named for what it does rather than for either occasion. Also corrects the plan. Phase 6 claimed the room had to be re-read between the invite and the assembly, for the same reason sequential invites re-read it. It does not -- that rule is about the MLS snapshot a commit is built on, and this runs downstream of the commit over `Mantra*` rows, which no commit touches. Two tests against a real `deliveryWelcome`: inviting into a room with signed work queues exactly one archive page addressed to the invitee, and inviting into a room with none queues no page while still writing the Welcome's gift wrap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
ed4410b972 |
feat: apply an archive a member is sent, and sweep what arrived too early
Phase 4 of docs/member-archive.md, and the half where the security lives. A member who holds no share, took part in no signing session and cannot decrypt a word of the room's history now ends up with the same rows as everybody else -- and gets there without trusting whoever sent them. **Intercepted in `fromGroupEventResult`, not in `applyInnerEvent`.** An archive is neither a document nor a submission, and deciding whether to act on one needs the active key, which `applyInnerEvent` has no business knowing. That is the same reason the gift wrap above it is handled there, so it sits next to it. **Verification per payload, framing per page.** A forged payload costs itself and nothing else -- the rule `MarmotInboundManager` already uses for a forged direct message, and for the same reason: this runs inside the inbound transaction and one bad event must not take the room down with it. Refusing the whole page would also let a single forgery deny an entire archive. The page's own framing stays all-or-nothing, because a page that will not parse has lost the thing that says what it contains. **The allowlist runs before the signature check, and it is not a formality.** Verification admits an event to the apply path on the strength of the group's signature, which makes every kind the group has ever signed replayable by any member at any time. There is a test that puts a genuine, still-verifying `GroupKeyStateEvent` in a hand-rolled page -- `ArchiveEvent.build` refuses to make one, which is the outbound half of the same rule -- and asserts the receiver's key state does not move. **The chat line is dropped, deliberately.** `ChatMessage` has an `autoGenerate` primary key, so there is no id to dedupe on and every applied payload would mint a new row: a synthetic transcript dated now, and another one on every pass of the sweep. The archive restores the work. The conversation is forward secret and stays gone. **A device that is not the named recipient does nothing.** It can read the page -- it is an ordinary group message, and it is the group's own history -- but it already holds the work, and re-applying would rewrite every one of its rows to point at an archive page rather than at the event that introduced it. That is also what bounds the sweep: only the member being caught up ever builds the list. **The sweep needs no table.** Pages arrive over relays in no order, so page 3 can land before page 2 and its chunks have no chapter to hang off. Those throw a foreign key violation and would be lost -- except the inbound path already stores every inner event it decrypts, so re-reading them is the same shape `FrostSigningManager.replayStoredMessages` has, for the same reason: nothing was lost, it just had nowhere to go at the time. Two things about the loop, the second found by a test: Progress is measured by *failures falling*, not by rows written. "Repeat while a pass applied something" does not terminate, because every write is an upsert and succeeds forever. What strictly decreases is the count that threw. And the result is the last pass rather than the sum of them. Accumulating counts a payload once per pass it survived and reports failures a later pass went on to fix, so `failed > 0` stops meaning "still missing" -- which is the only question a caller asks it. Caught by strengthening the out-of-order test to assert that the page completing an archive leaves nothing behind, rather than only that the rows matched: without that, the test passed while reporting fourteen failures on a fully converged database. Seven tests over two real databases with the pages carried by hand. The one that matters puts four forgeries in a page beside one honest dialect -- the room's id as author with a made-up signature, a real quorum of another group, an event edited after signing, and a member's own rumor, which is what everything on the wire looks like today -- and asserts the receiver ends with exactly the honest one. The rest: a full catch-up matches the sender row for row with the group's signature intact, pages delivered backwards converge and are asserted to have really failed first so the test cannot pass for the wrong reason, an archive files no chat lines, a bystander applies none of it, and applying the same archive twice changes nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
acff66a22e |
feat: rebuild the group's signed record out of the rows it left behind
Phase 3 of docs/member-archive.md. `ArchiveManager.assemble` walks a room's rows, rebuilds each into the event the group signed, drops anything it cannot prove, and cuts the rest into pages. Nothing sends one yet. **The gate found a real bug, which is why it was the gate.** Signed events are not stored as events -- `FrostSigningManager.complete` applies one and what survives is a `Mantra*` row -- so an archive has to rebuild them with `toXEvent()` and stands or falls on that being byte-identical to what was signed. Every `toXEvent()` in the codebase turned out to be unused in production, written for exactly this and never called, so the "tag order matches build so the event id round-trips" comments on them were claims nothing had ever checked. One was wrong. `MantraArtifact.toArtifactEvent` put the alt tag last where `ArtifactEvent.build` puts it first, and left out the version metadata tag altogether -- because that tag is not on the artifact row at all. `fromArtifactEvent` reads the artifact's own fields and drops the version label, which `applyInnerEvent` has by then turned into the artifact's first `MantraArtifactVersion`. So the label is now a parameter, read off the initial version: the one whose `createdAt` is the artifact's, since `initialVersionOf` derives it from the same event. Neither fault would have surfaced as an error. Both produce a well-formed artifact whose id no longer matches its fields, which every receiver drops as a forgery, silently, one kind at a time. `ArchiveRoundTripTest` now signs each archivable kind with a real quorum, files it as a row, rebuilds it and asserts the signature still covers what comes out -- plus the negative case, that rebuilding with the wrong version label fails as a forgery rather than as a mistake, which is why the assembler reads the label rather than defaulting it. **The allowlist narrows from nine kinds to six, and this is the finding to read.** Only six of the thirteen nip30303 kinds ever reach a signing session; the rest travel as member rumors, vouched for by the MLS frame they arrived in and by nothing that survives leaving it. An artifact version is derived rather than signed -- which is fine, because applying the archived artifact derives it again and the chapters hanging off it keep their foreign key. Nothing builds a `TranslationEvent` at all. The contributor lists have no arm in `applyInnerEvent` that writes a row. And `TranslationChunkEvent` -- **the translated text itself** -- is submitted by `MantraDao.saveTranslation` as its author's rumor, because a translation is one member's work rather than a group decision. So an archive restores everything a translation hangs on and not the translation: a new member gets the dialects, the artifacts, the chapters, the source chunks, which translations exist and their chapter scaffolding, and none of the prose. That is a real limit rather than a detail, so it is written into the allowlist's own doc comment, into the plan's "what this does not do", and into a test named after it -- with the three ways out sketched and none of them taken here, because the cheapest gives up the property the rest of this rests on and the best is a product decision about whether translating is an act of the group or of a member. **Nothing unverifiable leaves.** Every rebuilt event is checked with `isSignedByRoom` against the same room id the recipient will use. Not politeness -- the receiver checks anyway -- but so the page count says what will actually arrive: a row from a member's rumor is dropped here rather than by the recipient. **Walked down the tree, not queried per kind.** Only dialects and artifacts have a by-room query and the rest hang off a parent, and the walk is also what puts an artifact's version label within reach. Order is settled afterwards by `inApplyOrder` rather than by the walk, since the walk groups by artifact and the foreign keys are by kind. **Paging is greedy against both caps**, because they bind different archives: a room of one-line dialects hits the count first and a room of chapters hits the bytes. An event too large for a page of its own is dropped with a log rather than failing the archive -- a chapter nobody can archive is a hole, a member who gets nothing is a bigger one. Assembling only; queueing moved to Phase 5, where the thing that decides when to send lives. That keeps this testable against a real database with no outbound path in the way. Seven tests over a real in-memory database seeded through `applyInnerEvent` itself, so what is archived is what a member's device really holds rather than rows built to suit the test: every payload verifies, all six kinds appear exactly as often as they were signed, the whole archive is in dependency order end to end, a member's unsigned dialect sitting in the same room is left out, an empty room archives nothing without failing, and two archives of identical rows do not share an id -- which is what stops two members answering one request from having their pages counted towards each other's total. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kgothatso Ngako
|
6e04f6c7af |
feat: give the group's signed record an envelope it can travel in
Phase 2 of docs/member-archive.md. Two kinds, three tags, a codec and two caps.
Nothing sends or applies one yet -- that is phases 3 and 4 -- so this changes no
behaviour at all.
`ArchiveEvent` (30327) carries a page of the group's signed events, each whole,
keeping its own id, author and signature so the receiver checks it rather than
believing it. `ArchiveRequestEvent` (30328) is how a device with none of it asks.
**Why not one `SubmissionEvent` per event.** The envelope fits and the meaning
does not. A submission is an *act* -- this member is putting this event in front
of this group -- and an archive asserts nothing; it re-delivers what the group
already agreed. On one kind a four-hundred-event backfill is indistinguishable
from four hundred new submissions and every device has to guess which it is
reading. It would also be one inner event and one kind:445 per payload where a
page is one, and the submission arm of `applyInnerEvent` files a chat line per
payload, which an archive must not.
**Why 3032x and not 30313.** 30313 is free beside the nip30303 document kinds
and is not used, on `FrostSigningEvents`' own advice: the DKG's 30310-30316
already overlap that range and are told apart only by living in NIP-17 gift wraps
instead, which it calls "an accident of routing rather than a decision, and the
next family added should not rely on it." This is that next family. 30327 is also
the right neighbourhood on the merits, next to `GroupKeyStateEvent` at 30326 --
an archive is a statement about the record rather than a document kind.
**One list is the apply order and the allowlist both**, because a separate
allowlist is one more thing that can disagree with the order it is applied in.
The order is Room's rather than nostr's: every archivable kind has a foreign key
on the one before it, and kind order is not dependency order -- a dialect (30304)
has to land before an artifact (30300), and a translation chapter (30308) hangs
off a translation artifact version (30306) which hangs off an artifact version
(30301). So it is a list, not a `sortedBy { kind }`, and there is a test that
fails if anybody makes it one.
It is an allowlist first. Verification admits an event to the apply path on the
strength of the group's signature, which makes every kind the group has ever
signed replayable by any member at any time. A `GroupKeyStateEvent` is
group-signed and passes verification perfectly, so an archive carrying an old one
is a validly signed statement about what the room signs with, replayed by whoever
kept a copy. Nothing but this list stops it. The contributor-list kinds (30305,
30307, 30310) are left out on the same principle from the other side:
`applyInnerEvent` has no arm that writes a row for any of them, so archiving them
would cost bytes and restore nothing.
**All-or-nothing parsing, per-payload verification.** These are not in tension;
they answer different questions. A page that will not parse has lost its framing,
and one silently shortened by an element would report a complete archive on its
page count while holding less than it says. A payload whose signature does not
verify is a well-framed page with one bad event in it, and costing its honest
neighbours would let a single forgery deny an entire archive.
**The count cap was 256 and 256 can never fire.** An event carries 64 characters
of id, 64 of pubkey and 128 of signature before it says anything, so the floor is
about 370 bytes and a 64 KB page cannot hold much past 170 of them -- the byte
cap always binds first and the count cap is a check that never runs. Found by
writing the test that a page at exactly the cap still decodes, which failed. Now
128, where both bind something: the count stops a page of many small payloads,
the bytes stop a page of few large ones. That test is what fails if somebody
later raises one number without the other, and the doc comment says they have to
move together.
**The `p` tag is a hint, not access control**, and `ArchiveRecipientTag` says so
where it is defined. The page is an ordinary group message and every member can
read it, which is right, because it is their own history going back to them. What
it decides is who *acts*: a device that is not named applies nothing, since it
already holds the work and re-applying would rewrite every one of its rows to
point at an archive page rather than at the event that introduced it.
`ArchivePageTag` refuses an index outside its own count rather than clamping it.
The pair is how a receiver decides it has everything, so a repaired one would let
a truncated archive read as complete.
Twenty-one tests over the codec, both caps, the allowlist, the order and the
tags. Also corrects the phase-2 section of the plan, which still said 256.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Kgothatso Ngako
|
3ee1676a04 |
docs: plan handing a new member the group's signed history
A member added after the work was done sees none of it, and nothing in the app will ever show it to them. Two independent reasons, and the second is the one that surprises people. MLS gives no history: a Welcome carries the ratchet tree at the current epoch, not the transcript, and `MarmotInboundManager` drops anything from an epoch it holds no keys for. That is forward secrecy working rather than a gap to close. But group-signed events never travel at all. `FrostSigningManager.complete` says so in as many words -- a signed event authored by the threshold key cannot go out as an inner event, because the outbound pipeline would re-author it as its sender and strip the group's signature off -- so every device *derives* the finished event from its own `FrostSigningItem` rows. A member who was not in the session has no items, and no later message carries the event. So the second problem does not follow from the first and is not fixed by fixing it: even a member who could decrypt the whole back-transcript would still hold nothing an artifact, chapter or chunk could be built from. Which makes an archive not a convenience but the only path, and fixes the line the design has to hold: **it carries what the group signed, never the chat.** Restoring the chat would undo forward secrecy on purpose, and a signed event is the only thing a new member can check for themselves. **The property the whole plan rests on is already true.** A room's id *is* the group's threshold key derived at the room's path -- `GroupKeyState.verifies` and `FrostSigningManager.signingPath` hold that invariant from their own ends -- so `isSignedByGroup`'s three checks collapse to `event.pubKey == chatRoomId`, an id check and a signature verify. No key state row, no threshold key, no path, no lookup. A member who can name the room can verify its signatures, which is exactly the position a new member is in, and it means the sender of an archive does not have to be trusted at all. **Two guards the plan makes non-negotiable.** Nothing on the inbound nip30303 path verifies a signature today, and that is currently correct: rumors carry an empty sig and are authenticated by the MLS frame, so nothing on the wire has ever claimed group authorship. An archive is the first thing that does, so the verify is the feature's entire security rather than hardening on top of it. And verification turns "group-signed" into an admission ticket for the apply path, which is a wider door than it looks: a `GroupKeyStateEvent` is group-signed and would pass perfectly, so an archive could replay a genuine old one and re-point what the room signs with. The archive therefore carries an allowlist of document kinds, checked outbound and independently inbound -- the same shape, and the same reasoning, as the cap on `k` in frost-batch-signing.md. **Push and pull, in that order of appearance and the reverse order of importance.** Pushing an archive after the Welcome is what the question asked for, and on its own it fails the way marmot-membership.md describes: it is an application message in the epoch the add created, so one that beats the Welcome there is dropped rather than deferred, silently, while the inviter sees a success. So the joiner asks instead -- a request is proof it has processed its Welcome, and it covers the reinstall and the second device, which no invite-time push can. The push stays as a latency optimisation, deliberately phased after the thing that makes it safe. Nine phases: the verifier, the events, assembling an archive, applying one and the sweep that lets pages arrive out of order, the request, the push, UI, the cross-device tests, and rollout. The sweep needs no new table -- the inbound path already stores every inner event it decrypts, so it is the shape `FrostSigningManager.replayStoredMessages` already has. Also written down, because it is the first thing this will be reported as a bug for: an archive lets a new member *read* everything and does not let them sign anything. `proposeSigningBatch` wants a secret share and a place in the ceremony, and a group that re-runs its ceremony derives a different room rather than re-keying this one. Closing that needs share resharing, which is a great deal more work than this and is the thing to build after it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |