From acff66a22eff0153cf6d887645768d8ae69e3694 Mon Sep 17 00:00:00 2001 From: Kgothatso Ngako Date: Sun, 6 Sep 2026 14:12:29 +0200 Subject: [PATCH] feat: rebuild the group's signed record out of the rows it left behind Phase 3 of docs/member-archive.md. `ArchiveManager.assemble` walks a room's rows, rebuilds each into the event the group signed, drops anything it cannot prove, and cuts the rest into pages. Nothing sends one yet. **The gate found a real bug, which is why it was the gate.** Signed events are not stored as events -- `FrostSigningManager.complete` applies one and what survives is a `Mantra*` row -- so an archive has to rebuild them with `toXEvent()` and stands or falls on that being byte-identical to what was signed. Every `toXEvent()` in the codebase turned out to be unused in production, written for exactly this and never called, so the "tag order matches build so the event id round-trips" comments on them were claims nothing had ever checked. One was wrong. `MantraArtifact.toArtifactEvent` put the alt tag last where `ArtifactEvent.build` puts it first, and left out the version metadata tag altogether -- because that tag is not on the artifact row at all. `fromArtifactEvent` reads the artifact's own fields and drops the version label, which `applyInnerEvent` has by then turned into the artifact's first `MantraArtifactVersion`. So the label is now a parameter, read off the initial version: the one whose `createdAt` is the artifact's, since `initialVersionOf` derives it from the same event. Neither fault would have surfaced as an error. Both produce a well-formed artifact whose id no longer matches its fields, which every receiver drops as a forgery, silently, one kind at a time. `ArchiveRoundTripTest` now signs each archivable kind with a real quorum, files it as a row, rebuilds it and asserts the signature still covers what comes out -- plus the negative case, that rebuilding with the wrong version label fails as a forgery rather than as a mistake, which is why the assembler reads the label rather than defaulting it. **The allowlist narrows from nine kinds to six, and this is the finding to read.** Only six of the thirteen nip30303 kinds ever reach a signing session; the rest travel as member rumors, vouched for by the MLS frame they arrived in and by nothing that survives leaving it. An artifact version is derived rather than signed -- which is fine, because applying the archived artifact derives it again and the chapters hanging off it keep their foreign key. Nothing builds a `TranslationEvent` at all. The contributor lists have no arm in `applyInnerEvent` that writes a row. And `TranslationChunkEvent` -- **the translated text itself** -- is submitted by `MantraDao.saveTranslation` as its author's rumor, because a translation is one member's work rather than a group decision. So an archive restores everything a translation hangs on and not the translation: a new member gets the dialects, the artifacts, the chapters, the source chunks, which translations exist and their chapter scaffolding, and none of the prose. That is a real limit rather than a detail, so it is written into the allowlist's own doc comment, into the plan's "what this does not do", and into a test named after it -- with the three ways out sketched and none of them taken here, because the cheapest gives up the property the rest of this rests on and the best is a product decision about whether translating is an act of the group or of a member. **Nothing unverifiable leaves.** Every rebuilt event is checked with `isSignedByRoom` against the same room id the recipient will use. Not politeness -- the receiver checks anyway -- but so the page count says what will actually arrive: a row from a member's rumor is dropped here rather than by the recipient. **Walked down the tree, not queried per kind.** Only dialects and artifacts have a by-room query and the rest hang off a parent, and the walk is also what puts an artifact's version label within reach. Order is settled afterwards by `inApplyOrder` rather than by the walk, since the walk groups by artifact and the foreign keys are by kind. **Paging is greedy against both caps**, because they bind different archives: a room of one-line dialects hits the count first and a room of chapters hits the bytes. An event too large for a page of its own is dropped with a log rather than failing the archive -- a chapter nobody can archive is a hole, a member who gets nothing is a bigger one. Assembling only; queueing moved to Phase 5, where the thing that decides when to send lives. That keeps this testable against a real database with no outbound path in the way. Seven tests over a real in-memory database seeded through `applyInnerEvent` itself, so what is archived is what a member's device really holds rather than rows built to suit the test: every payload verifies, all six kinds appear exactly as often as they were signed, the whole archive is in dependency order end to end, a member's unsigned dialect sitting in the same room is left out, an empty room archives nothing without failing, and two archives of identical rows do not share an id -- which is what stops two members answering one request from having their pages counted towards each other's total. Co-Authored-By: Claude Opus 5 --- .../compose/database/model/MantraArtifact.kt | 27 +- .../mantra/compose/managers/ArchiveManager.kt | 212 ++++++++++ .../compose/nostr/archive/ArchiveEvent.kt | 40 +- .../compose/nostr/archive/ArchiveEventTest.kt | 53 ++- .../nostr/archive/ArchiveRoundTripTest.kt | 351 ++++++++++++++++ .../managers/ArchiveAssemblyJvmTest.kt | 386 ++++++++++++++++++ docs/member-archive.md | 71 +++- 7 files changed, 1107 insertions(+), 33 deletions(-) create mode 100644 composeApp/src/commonMain/kotlin/press/mantra/compose/managers/ArchiveManager.kt create mode 100644 composeApp/src/commonTest/kotlin/press/mantra/compose/nostr/archive/ArchiveRoundTripTest.kt create mode 100644 composeApp/src/jvmTest/kotlin/press/mantra/compose/managers/ArchiveAssemblyJvmTest.kt diff --git a/composeApp/src/commonMain/kotlin/press/mantra/compose/database/model/MantraArtifact.kt b/composeApp/src/commonMain/kotlin/press/mantra/compose/database/model/MantraArtifact.kt index 995ec3fe..fe83eab5 100644 --- a/composeApp/src/commonMain/kotlin/press/mantra/compose/database/model/MantraArtifact.kt +++ b/composeApp/src/commonMain/kotlin/press/mantra/compose/database/model/MantraArtifact.kt @@ -11,6 +11,7 @@ import com.vitorpamplona.quartz.nip31Alts.AltTag import press.mantra.compose.database.model.traits.OptionalNostrEventEntity import press.mantra.compose.database.model.traits.TimestampedEntity import press.mantra.compose.nostr.nip30303.ArtifactEvent +import press.mantra.compose.nostr.nip30303.tags.ArtifactVersionMetadataTag import press.mantra.compose.nostr.nip30303.tags.DialectIdTag import press.mantra.compose.nostr.nip30303.tags.LicenseTag import press.mantra.compose.nostr.nip30303.tags.UrlTag @@ -65,12 +66,34 @@ data class MantraArtifact( override val createdAt: Instant = Clock.System.now(), override val updatedAt: Instant = createdAt ): OptionalNostrEventEntity, TimestampedEntity { - fun toArtifactEvent(): ArtifactEvent { + /** + * The event the group signed, rebuilt from this row. + * + * [versionLabel] is a parameter because it is not on this row and cannot be. + * `fromArtifactEvent` reads the artifact's own fields and drops the version + * metadata, which `ChatMessage.applyInnerEvent` has by then turned into the + * artifact's first `MantraArtifactVersion`. It was still part of what the + * group put its signature to, so a rebuild without it hashes to a different + * id and produces an event the signature does not cover -- which a receiver + * reads as a forgery, silently. Recover it from the initial version: the one + * whose `createdAt` is this artifact's, since `initialVersionOf` derives it + * from the same event. + * + * Tag order matches [ArtifactEvent.build] exactly -- alt first -- for the + * same reason, and `ArchiveRoundTripTest` is what says so. Both of those were + * wrong here until an archive needed to rebuild an artifact and nothing had + * ever called this. + */ + fun toArtifactEvent(versionLabel: String): ArtifactEvent { return ArtifactEvent( id = id, pubKey = publicKey, createdAt = createdAt.epochSeconds, + // Tag order matches ArtifactEvent.build so the event id round-trips. tags = TagArrayBuilder() + .addUnique( + AltTag.assemble(ArtifactEvent.ALT_DESCRIPTION) + ) .addUnique(UrlTag.assemble(url)) .addUnique(VisibilityTag.assemble(visibility)) .addUnique(LicenseTag.assemble(license)) @@ -78,7 +101,7 @@ data class MantraArtifact( DialectIdTag.assemble(dialectId) ) .addUnique( - AltTag.assemble(ArtifactEvent.ALT_DESCRIPTION) + ArtifactVersionMetadataTag.assemble(versionLabel) ) .build(), content = name, diff --git a/composeApp/src/commonMain/kotlin/press/mantra/compose/managers/ArchiveManager.kt b/composeApp/src/commonMain/kotlin/press/mantra/compose/managers/ArchiveManager.kt new file mode 100644 index 00000000..5c8171ae --- /dev/null +++ b/composeApp/src/commonMain/kotlin/press/mantra/compose/managers/ArchiveManager.kt @@ -0,0 +1,212 @@ +package press.mantra.compose.managers + +import co.touchlab.kermit.Logger +import com.vitorpamplona.quartz.nip01Core.core.Event +import com.vitorpamplona.quartz.nip01Core.core.HexKey +import com.vitorpamplona.quartz.nip01Core.signers.EventTemplate +import com.vitorpamplona.quartz.utils.RandomInstance +import com.vitorpamplona.quartz.utils.TimeUtils +import press.mantra.compose.database.MantraDatabase +import press.mantra.compose.extensions.toHex +import press.mantra.compose.nostr.archive.ArchiveEvent +import press.mantra.compose.nostr.frost.GroupKeyStateEvent + +/** + * Building the group's signed record into pages a member who lacks it can apply. + * + * See docs/member-archive.md. The short of it: a member added after the work was + * done has none of it and never will, because a group-signed event is applied + * locally by each device that took part and never goes on the wire. This is how + * it gets to them. + * + * ### The rows are the archive + * + * Signed events are not stored as events. `FrostSigningManager.complete` applies + * one and what survives is a `Mantra*` row, so every payload here is rebuilt with + * `toXEvent()` and stands or falls on that rebuild being byte-identical to what + * was signed. `ArchiveRoundTripTest` is what says it is, per kind, against a real + * quorum -- and it found two faults in the artifact's rebuild the first time it + * ran, both of which would have shipped payloads that every receiver drops as + * forgeries without a word. + * + * ### Nothing unverifiable leaves + * + * Every rebuilt event is checked with [GroupKeyStateEvent.isSignedByRoom] before + * it is packed, against the same room id the recipient will check it with. That + * is not politeness towards the receiver, who checks anyway. It is what keeps an + * archive honest about its own size: a row that came from a member's rumor + * cannot be archived, and dropping it here rather than letting the recipient + * drop it means the page count says what will actually arrive. + */ +object ArchiveManager { + private const val TAG = "ArchiveManager" + + private val logger = Logger.withTag(TAG) + + /** + * Every group-signed event this device holds for [chatRoomId], paged and + * addressed to [recipient]. + * + * Empty when there is nothing to send -- a room with no shared key, a room + * whose work is all member rumors, or a device that is itself behind. An + * empty result is not an error and the caller should not report one: the + * honest answer to "send them the history" in a room with no signed history + * is nothing. + */ + suspend fun assemble( + database: MantraDatabase, + chatRoomId: String, + recipient: HexKey, + archiveId: String = RandomInstance.bytes(32).toHex(), + createdAt: Long = TimeUtils.now(), + ): List> { + val rebuilt = signedEventsOf(database, chatRoomId) + + val verified = rebuilt.filter { GroupKeyStateEvent.isSignedByRoom(it, chatRoomId) } + if (verified.size != rebuilt.size) { + // Expected rather than alarming: an artifact version is derived + // rather than signed, and a translation is its author's rumor. What + // would be worth looking at is this dropping something the group + // really did sign, which reads as a broken `toXEvent` rather than as + // a missing signature. + logger.d( + "Leaving ${rebuilt.size - verified.size} of ${rebuilt.size} row(s) out of " + + "$chatRoomId's archive: nothing verifiably signed by the room" + ) + } + + val pages = paginate(ArchiveEvent.inApplyOrder(verified)) + if (pages.isEmpty()) { + logger.i("Nothing signed to archive for room $chatRoomId") + return emptyList() + } + + logger.i( + "Archiving ${verified.size} event(s) for $chatRoomId as $archiveId, " + + "${pages.size} page(s) for ${recipient.take(8)}" + ) + + return pages.mapIndexed { index, payloads -> + ArchiveEvent.build( + payloads = payloads, + archiveId = archiveId, + index = index, + count = pages.size, + recipient = recipient, + createdAt = createdAt, + ) + } + } + + /** + * The room's rows, rebuilt into the events they came from. + * + * Walked down the tree rather than queried per kind, because only dialects + * and artifacts have a by-room query and the rest hang off a parent. The walk + * is also what makes an artifact's version label reachable: it is not on the + * artifact row -- see `MantraArtifact.toArtifactEvent` -- and the version it + * went into is one step away here. + * + * Order does not matter at this point; [ArchiveEvent.inApplyOrder] settles it + * afterwards. What matters is that nothing is missed, so this returns + * everything and the verify filter above decides what can travel. + */ + private suspend fun signedEventsOf( + database: MantraDatabase, + chatRoomId: String, + ): List = buildList { + database.mantraDialectDao().getDialectsByChatRoomId(chatRoomId).forEach { + add(it.toDialectEvent()) + } + + database.mantraArtifactDao().getArtifactsByChatRoomId(chatRoomId).forEach { artifact -> + val versions = database.mantraArtifactVersionDao() + .getArtifactVersionsByArtifactId(artifact.id) + + // The version an artifact starts life with carries the label the + // artifact was signed with, and `initialVersionOf` gives it the + // artifact's own timestamp. A row with none of those is an artifact + // this device cannot rebuild, which is a gap in the archive rather + // than a reason to abandon it. + val versionLabel = versions + .firstOrNull { it.createdAt == artifact.createdAt } + ?.versionLabel + + if (versionLabel == null) { + logger.w("Artifact ${artifact.id} has no initial version; leaving it out") + } else { + add(artifact.toArtifactEvent(versionLabel = versionLabel)) + } + + versions.forEach { version -> + database.mantraChapterDao() + .getChaptersByArtifactVersionId(version.id) + .forEach { chapter -> + add(chapter.toChapterEvent()) + + database.mantraChunkDao() + .getChunksByChapterId(chapter.id) + .forEach { add(it.toChunkEvent()) } + } + + database.mantraTranslationArtifactVersionDao() + .getTranslationsByArtifactVersionId(version.id) + .forEach { translationVersion -> + add(translationVersion.toTranslationArtifactVersionEvent()) + + database.mantraTranslationChapterDao() + .getTranslationChaptersByTranslationArtifactVersionId(translationVersion.id) + .forEach { add(it.toTranslationChapterEvent()) } + } + } + } + } + + /** + * [events] cut into pages that fit, keeping the order they arrive in. + * + * Greedy: fill a page until the next event would cross either cap. Both are + * checked because they bind different archives -- a room of one-line dialects + * hits the count first and a room of chapters hits the bytes. + * + * An event too large to share a page with anything is given one of its own. + * One too large for even that is dropped with a log rather than failing the + * archive: a chapter nobody can archive is a hole, and a member who gets + * nothing at all is a bigger one. + */ + private fun paginate(events: List): List> { + val pages = mutableListOf>() + var page = mutableListOf() + // The brackets an empty page already costs. + var bytes = 2 + + events.forEach { event -> + // Plus the comma this event needs if it is not first on its page. + val size = event.toJson().encodeToByteArray().size + 1 + + if (2 + size > ArchiveEvent.MAX_PAGE_BYTES) { + logger.w( + "Event ${event.id} is $size bytes and will not fit a " + + "${ArchiveEvent.MAX_PAGE_BYTES}-byte page; leaving it out of the archive" + ) + return@forEach + } + + val full = page.isNotEmpty() && + (bytes + size > ArchiveEvent.MAX_PAGE_BYTES || page.size >= ArchiveEvent.MAX_PAGE_EVENTS) + + if (full) { + pages.add(page) + page = mutableListOf() + bytes = 2 + } + + page.add(event) + bytes += size + } + + if (page.isNotEmpty()) pages.add(page) + + return pages + } +} diff --git a/composeApp/src/commonMain/kotlin/press/mantra/compose/nostr/archive/ArchiveEvent.kt b/composeApp/src/commonMain/kotlin/press/mantra/compose/nostr/archive/ArchiveEvent.kt index 08a731c2..f15b68a8 100644 --- a/composeApp/src/commonMain/kotlin/press/mantra/compose/nostr/archive/ArchiveEvent.kt +++ b/composeApp/src/commonMain/kotlin/press/mantra/compose/nostr/archive/ArchiveEvent.kt @@ -15,14 +15,11 @@ import press.mantra.compose.nostr.archive.tags.ArchiveIdTag import press.mantra.compose.nostr.archive.tags.ArchivePageTag import press.mantra.compose.nostr.archive.tags.ArchiveRecipientTag import press.mantra.compose.nostr.nip30303.ArtifactEvent -import press.mantra.compose.nostr.nip30303.ArtifactVersionEvent import press.mantra.compose.nostr.nip30303.ChapterEvent import press.mantra.compose.nostr.nip30303.ChunkEvent import press.mantra.compose.nostr.nip30303.DialectEvent import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionEvent import press.mantra.compose.nostr.nip30303.TranslationChapterEvent -import press.mantra.compose.nostr.nip30303.TranslationChunkEvent -import press.mantra.compose.nostr.nip30303.TranslationEvent /** * 30327 @@ -165,22 +162,43 @@ class ArchiveEvent( * what the room signs with, replayed by whoever kept a copy. Nothing but * this list stops it. * - * Absent on purpose: the contributor-list kinds (30305, 30307, 30310). - * `ChatMessage.applyInnerEvent` has no arm that writes a row for any of - * them, so archiving them would cost bytes and restore nothing. They - * belong here on the day that changes, and the test that this list is a - * subset of what `applyInnerEvent` handles is what should catch it. + * ### What is missing from it, and why + * + * **Only kinds the group actually signs can be here**, because an archive + * that cannot be verified is one that has to be believed. Six of the + * thirteen nip30303 kinds reach a signing session; the rest travel as + * rumors -- empty signature, member author, vouched for by the MLS frame + * they arrived in and by nothing that survives leaving it. + * + * Left out for that reason, and this is a real limit rather than an + * oversight: + * + * - `ArtifactVersionEvent` (30301). Never signed. The version an + * artifact starts life with is *derived* from the signed artifact by + * `MantraArtifactVersion.initialVersionOf`, which the artifact's own + * arm in `applyInnerEvent` runs -- so archiving the artifact brings its + * version along and the chapters hanging off it keep their foreign + * key. Later versions go through `MantraDao.addArtifactVersion` as + * rumors, and no screen calls it today. + * - `TranslationChunkEvent` (30309) -- **the translated text itself**. + * `MantraDao.saveTranslation` submits it as a member's rumor, because + * a translation is one member's work rather than a group decision. + * So an archive restores everything a translation hangs on and not the + * translation. See docs/member-archive.md for what closing that would + * take; it is not a line in this list. + * - `TranslationEvent` (30311). Nothing builds one; the inbound arm + * exists and no producer does. + * - The contributor lists (30305, 30307, 30310). `applyInnerEvent` has + * no arm that writes a row for any of them, so archiving them would + * cost bytes and restore nothing. */ private val APPLY_ORDER: List = listOf( DialectEvent.KIND, ArtifactEvent.KIND, - ArtifactVersionEvent.KIND, ChapterEvent.KIND, ChunkEvent.KIND, TranslationArtifactVersionEvent.KIND, TranslationChapterEvent.KIND, - TranslationChunkEvent.KIND, - TranslationEvent.KIND, ) /** Every kind an archive may carry. */ diff --git a/composeApp/src/commonTest/kotlin/press/mantra/compose/nostr/archive/ArchiveEventTest.kt b/composeApp/src/commonTest/kotlin/press/mantra/compose/nostr/archive/ArchiveEventTest.kt index 20515041..fe011582 100644 --- a/composeApp/src/commonTest/kotlin/press/mantra/compose/nostr/archive/ArchiveEventTest.kt +++ b/composeApp/src/commonTest/kotlin/press/mantra/compose/nostr/archive/ArchiveEventTest.kt @@ -179,20 +179,38 @@ class ArchiveEventTest { // ---- What an archive may carry --------------------------------------- @Test - fun `the allowlist is every kind an archive can restore, and only those`() { + fun `the allowlist is every kind the group signs, and only those`() { listOf( DialectEvent.KIND, ArtifactEvent.KIND, - ArtifactVersionEvent.KIND, ChapterEvent.KIND, ChunkEvent.KIND, TranslationArtifactVersionEvent.KIND, TranslationChapterEvent.KIND, - TranslationChunkEvent.KIND, - TranslationEvent.KIND, ).forEach { assertTrue(ArchiveEvent.isArchivable(it), "kind $it should be archivable") } - assertEquals(9, ArchiveEvent.ARCHIVABLE_KINDS.size) + assertEquals(6, ArchiveEvent.ARCHIVABLE_KINDS.size) + } + + @Test + fun `the kinds nobody signs are left out, translations included`() { + // The limit worth knowing about before reading anything else here. An + // archive can only carry what the receiver can check, and six of the + // thirteen nip30303 kinds reach a signing session. The rest travel as + // rumors -- empty signature, member author -- vouched for by the MLS + // frame they arrived in and by nothing that survives leaving it. + + // The translated text. MantraDao.saveTranslation submits it as the + // member's own rumor, so an archive restores everything a translation + // hangs on and not the translation. + assertFalse(ArchiveEvent.isArchivable(TranslationChunkEvent.KIND)) + + // Never signed either: derived from the signed artifact on arrival, which + // is what keeps a chapter's foreign key satisfied without archiving it. + assertFalse(ArchiveEvent.isArchivable(ArtifactVersionEvent.KIND)) + + // Nothing builds one of these at all. + assertFalse(ArchiveEvent.isArchivable(TranslationEvent.KIND)) } @Test @@ -235,17 +253,18 @@ class ArchiveEventTest { // Every one of these is a foreign key in Room, so an archive applied the // other way round is a constraint violation rather than a wrong answer. assertTrue(rank(DialectEvent.KIND) < rank(ArtifactEvent.KIND)) - assertTrue(rank(ArtifactEvent.KIND) < rank(ArtifactVersionEvent.KIND)) - assertTrue(rank(ArtifactVersionEvent.KIND) < rank(ChapterEvent.KIND)) assertTrue(rank(ChapterEvent.KIND) < rank(ChunkEvent.KIND)) - assertTrue(rank(ArtifactVersionEvent.KIND) < rank(TranslationArtifactVersionEvent.KIND)) - assertTrue(rank(DialectEvent.KIND) < rank(TranslationArtifactVersionEvent.KIND)) assertTrue(rank(TranslationArtifactVersionEvent.KIND) < rank(TranslationChapterEvent.KIND)) assertTrue(rank(ChapterEvent.KIND) < rank(TranslationChapterEvent.KIND)) - assertTrue(rank(ChunkEvent.KIND) < rank(TranslationChunkEvent.KIND)) - assertTrue(rank(TranslationChapterEvent.KIND) < rank(TranslationChunkEvent.KIND)) - assertTrue(rank(TranslationChunkEvent.KIND) < rank(TranslationEvent.KIND)) - assertTrue(rank(TranslationArtifactVersionEvent.KIND) < rank(TranslationEvent.KIND)) + assertTrue(rank(DialectEvent.KIND) < rank(TranslationArtifactVersionEvent.KIND)) + + // A chapter and a translation artifact version both hang off an artifact + // *version*, which is not archived: it is derived from the signed + // artifact by the artifact's own arm in applyInnerEvent. So the artifact + // has to land before either of them, and the dependency runs through a + // kind that is not in this list at all. + assertTrue(rank(ArtifactEvent.KIND) < rank(ChapterEvent.KIND)) + assertTrue(rank(ArtifactEvent.KIND) < rank(TranslationArtifactVersionEvent.KIND)) // And the numbers really do disagree with the order, which is why this is // a list rather than a sortedBy { kind }. @@ -256,22 +275,22 @@ class ArchiveEventTest { @Test fun `sorting a page puts what is referenced before what refers to it`() { val jumbled = listOf( - payload(TranslationEvent.KIND), + payload(TranslationChapterEvent.KIND), payload(ChunkEvent.KIND), payload(DialectEvent.KIND), + payload(TranslationArtifactVersionEvent.KIND), payload(ChapterEvent.KIND), payload(ArtifactEvent.KIND), - payload(ArtifactVersionEvent.KIND), ) assertEquals( listOf( DialectEvent.KIND, ArtifactEvent.KIND, - ArtifactVersionEvent.KIND, ChapterEvent.KIND, ChunkEvent.KIND, - TranslationEvent.KIND, + TranslationArtifactVersionEvent.KIND, + TranslationChapterEvent.KIND, ), ArchiveEvent.inApplyOrder(jumbled).map { it.kind } ) diff --git a/composeApp/src/commonTest/kotlin/press/mantra/compose/nostr/archive/ArchiveRoundTripTest.kt b/composeApp/src/commonTest/kotlin/press/mantra/compose/nostr/archive/ArchiveRoundTripTest.kt new file mode 100644 index 00000000..78dd62f9 --- /dev/null +++ b/composeApp/src/commonTest/kotlin/press/mantra/compose/nostr/archive/ArchiveRoundTripTest.kt @@ -0,0 +1,351 @@ +package press.mantra.compose.nostr.archive + +import com.vitorpamplona.quartz.nip01Core.core.Event +import com.vitorpamplona.quartz.nip01Core.core.hexToByteArray +import com.vitorpamplona.quartz.nip01Core.crypto.EventHasher +import com.vitorpamplona.quartz.nip01Core.signers.EventTemplate +import fr.acinq.bitcoin.ByteVector +import fr.acinq.bitcoin.ByteVector32 +import fr.acinq.bitcoin.PrivateKey +import fr.acinq.bitcoin.crypto.frost.Frost +import fr.acinq.bitcoin.crypto.frost.IndividualNonce +import fr.acinq.bitcoin.crypto.frost.KeyMaterial +import fr.acinq.bitcoin.crypto.frost.SecretNonce +import fr.acinq.bitcoin.crypto.frost.Session +import fr.acinq.bitcoin.crypto.frost.TweakCache +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertFalse +import kotlin.test.assertNotNull +import kotlin.test.assertTrue +import press.mantra.compose.database.model.MantraArtifact +import press.mantra.compose.database.model.MantraChapter +import press.mantra.compose.database.model.MantraChunk +import press.mantra.compose.database.model.MantraDialect +import press.mantra.compose.database.model.MantraTranslationArtifactVersion +import press.mantra.compose.database.model.MantraTranslationChapter +import press.mantra.compose.extensions.toHex +import press.mantra.compose.managers.SharedKeyDerivation +import press.mantra.compose.nostr.frost.GroupKeyStateEvent +import press.mantra.compose.nostr.nip30303.ArtifactEvent +import press.mantra.compose.nostr.nip30303.ChapterEvent +import press.mantra.compose.nostr.nip30303.ChunkEvent +import press.mantra.compose.nostr.nip30303.DialectEvent +import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionEvent +import press.mantra.compose.nostr.nip30303.TranslationChapterEvent + +/** + * The assumption the whole archive rests on: a row can be turned back into the + * event the group signed. + * + * Signed events are not stored as events. `FrostSigningManager.complete` applies + * them and what survives is a `Mantra*` row, so an archive has to rebuild each + * one with `toXEvent()` and hope it comes out byte-identical. If it does not, + * the id changes, the signature no longer covers it, and every receiver drops + * the payload as a forgery -- silently, one kind at a time. + * + * That is what the round-trip note on each `toXEvent` claims and what nothing + * asserted until now. Every kind in `ArchiveEvent.ARCHIVABLE_KINDS` is checked + * here against a real FROST quorum, because a signature that verifies is the + * only evidence that the rebuild is faithful -- comparing fields would only + * prove the test and the code agree about which fields matter. + * + * A kind that fails here cannot be archived at all, whatever the allowlist says. + */ +class ArchiveRoundTripTest { + private val participants = 3 + private val threshold = 2 + + /** Stands in for a completed ceremony; nothing here is about the DKG. */ + private val keyMaterial: KeyMaterial = Frost.trustedDealerKeygen( + thresholdSecretKey = PrivateKey( + ByteVector32("1c0ffee0000000000000000000000000000000000000000000000000000000a1") + ), + nParticipants = participants, + threshold = threshold + ) + + private val room: SharedKeyDerivation.Derived = SharedKeyDerivation.derive( + thresholdPublicKey = keyMaterial.thresholdPublicKey.value.toHex(), + path = SharedKeyDerivation.MARMOT_ADMIN_GROUP_PATH + ) + + private val tweakCache: TweakCache = room.cache + + /** The group's identity, which is also the room's id. */ + private val chatRoomId = room.hex + + private val dialectId = "b".repeat(64) + private val artifactVersionId = "c".repeat(64) + private val chapterId = "d".repeat(64) + private val translationArtifactVersionId = "e".repeat(64) + + /** Exactly what `FrostSigningManager.unsignedEventOf` does. */ + private fun unsignedEventOf(template: EventTemplate<*>) = Event( + id = EventHasher.hashId( + pubKey = chatRoomId, + createdAt = template.createdAt, + kind = template.kind, + tags = template.tags, + content = template.content + ), + pubKey = chatRoomId, + createdAt = template.createdAt, + kind = template.kind, + tags = template.tags, + content = template.content, + sig = "" + ) + + /** A real quorum signing [eventId], in the manager's order. */ + private fun groupSignature(eventId: String): String { + val message = ByteVector(eventId.hexToByteArray()) + val signerIds = listOf(0, 1) + + val nonces = signerIds.map { signerId -> + SecretNonce.generate( + sessionRandom = ByteVector32("a".repeat(63) + "${signerId + 1}"), + secretShare = keyMaterial.secretShares[signerId], + publicShare = keyMaterial.publicShares[signerId], + tweakedThresholdPublicKey = tweakCache.tweakedPublicKey, + message = message, + extraInput = null + ) + } + + val signingSession = Session.create( + aggregatedNonce = IndividualNonce.aggregate(nonces.map { it.second }).right!!, + signerIds = signerIds.map { it.toUInt() }, + signerPublicShares = signerIds.map { keyMaterial.publicShares[it] }, + nParticipants = participants, + threshold = threshold, + tweakCache = tweakCache, + message = message + ) + + val partials = signerIds.mapIndexed { position, signerId -> + signingSession.sign( + nonces[position].first, + keyMaterial.secretShares[signerId], + signerId.toUInt() + ).right!! + } + + return signingSession.aggregateSigs(partials).right!!.toByteArray().toHex() + } + + /** A template, signed by the group, the way a finished session leaves one. */ + private fun signed(template: EventTemplate<*>): Event { + val unsigned = unsignedEventOf(template) + + return Event( + id = unsigned.id, + pubKey = unsigned.pubKey, + createdAt = unsigned.createdAt, + kind = unsigned.kind, + tags = unsigned.tags, + content = unsigned.content, + sig = groupSignature(unsigned.id) + ) + } + + /** + * The claim, for one kind: sign it, file it as a row, rebuild it from the + * row, and the group's signature still covers what comes out. + */ + private fun assertRoundTrips(template: EventTemplate<*>, rebuild: (Event) -> Event?) { + val signedEvent = signed(template) + assertTrue( + GroupKeyStateEvent.isSignedByRoom(signedEvent, chatRoomId), + "the harness itself must produce a verifiable event" + ) + + val rebuilt = assertNotNull( + rebuild(signedEvent), + "kind ${template.kind} did not survive being turned into a row" + ) + + // Asserted before the signature so a failure says which half broke: an id + // that moved means the tag order or a field changed in the rebuild, where + // a signature that fails on a matching id would mean something stranger. + assertEquals( + signedEvent.id, + rebuilt.id, + "kind ${template.kind} came back with a different id" + ) + assertEquals( + signedEvent.sig, + rebuilt.sig, + "kind ${template.kind} lost the group's signature" + ) + assertTrue( + GroupKeyStateEvent.isSignedByRoom(rebuilt, chatRoomId), + "kind ${template.kind} rebuilt into something the group's signature does not cover" + ) + } + + @Test + fun `a dialect survives the trip through a row`() { + assertRoundTrips( + DialectEvent.build( + name = "isiZulu", + country = "ZA", + language = "zu", + createdAt = 1_700_000_000L + ) + ) { event -> + MantraDialect.fromDialectEvent( + DialectEvent(event.id, event.pubKey, event.createdAt, event.tags, event.content, event.sig), + chatRoomId + )?.toDialectEvent() + } + } + + @Test + fun `an artifact survives the trip through a row`() { + // The one that did not, when this test was written. Two faults, and the + // second is the interesting one: `toArtifactEvent` put the alt tag last + // where `build` puts it first, and it left out the version metadata + // entirely -- because that tag is not on the row at all. It is consumed + // into the artifact's first MantraArtifactVersion and dropped. So the + // label has to come back from there, which is why it is a parameter. + assertRoundTrips( + ArtifactEvent.build( + name = "In Detention", + url = "https://example.com/in-detention", + visibility = "private", + license = "cc", + dialectId = dialectId, + versionLabel = "1.0", + createdAt = 1_700_000_000L + ) + ) { event -> + MantraArtifact.fromArtifactEvent( + ArtifactEvent(event.id, event.pubKey, event.createdAt, event.tags, event.content, event.sig), + chatRoomId + )?.toArtifactEvent(versionLabel = "1.0") + } + } + + @Test + fun `an artifact rebuilt with the wrong version label is not the one that was signed`() { + // What the parameter costs if a caller guesses. The label was part of + // what the group put its signature to, so a rebuild carrying a different + // one is a different event -- and it fails as a forgery rather than as a + // mistake, which is why the assembler reads it off the initial version + // rather than defaulting it. + val signedEvent = signed( + ArtifactEvent.build( + name = "In Detention", + url = "https://example.com/in-detention", + visibility = "private", + license = "cc", + dialectId = dialectId, + versionLabel = "1.0", + createdAt = 1_700_000_000L + ) + ) + + val rebuilt = MantraArtifact.fromArtifactEvent( + ArtifactEvent( + signedEvent.id, signedEvent.pubKey, signedEvent.createdAt, + signedEvent.tags, signedEvent.content, signedEvent.sig + ), + chatRoomId + )?.toArtifactEvent(versionLabel = "2.0") + + assertNotNull(rebuilt) + assertFalse(GroupKeyStateEvent.isSignedByRoom(rebuilt, chatRoomId)) + } + + @Test + fun `a chapter survives the trip through a row`() { + assertRoundTrips( + ChapterEvent.build( + artifactVersionId = artifactVersionId, + name = "Chapter One", + originalText = "He was a man of parts.", + index = 0, + wordCount = 6, + characterCount = 22, + createdAt = 1_700_000_000L + ) + ) { event -> + MantraChapter.fromChapterEvent( + ChapterEvent(event.id, event.pubKey, event.createdAt, event.tags, event.content, event.sig), + chatRoomId + )?.toChapterEvent() + } + } + + @Test + fun `a chunk survives the trip through a row`() { + assertRoundTrips( + ChunkEvent.build( + chapterId = chapterId, + text = "He was a man of parts.", + index = 0, + wordCount = 6, + characterCount = 22, + createdAt = 1_700_000_000L + ) + ) { event -> + MantraChunk.fromChunkEvent( + ChunkEvent(event.id, event.pubKey, event.createdAt, event.tags, event.content, event.sig), + chatRoomId + )?.toChunkEvent() + } + } + + @Test + fun `a translation artifact version survives the trip through a row`() { + assertRoundTrips( + TranslationArtifactVersionEvent.build( + artifactVersionId = artifactVersionId, + dialectId = dialectId, + name = "isiZulu", + visibility = "private", + license = "cc", + createdAt = 1_700_000_000L + ) + ) { event -> + MantraTranslationArtifactVersion.fromTranslationArtifactVersionEvent( + TranslationArtifactVersionEvent( + event.id, event.pubKey, event.createdAt, event.tags, event.content, event.sig + ), + chatRoomId + )?.toTranslationArtifactVersionEvent() + } + } + + @Test + fun `a translation chapter survives the trip through a row`() { + assertRoundTrips( + TranslationChapterEvent.build( + translationArtifactVersionId = translationArtifactVersionId, + chapterId = chapterId, + index = 0, + createdAt = 1_700_000_000L + ) + ) { event -> + MantraTranslationChapter.fromTranslationChapterEvent( + TranslationChapterEvent( + event.id, event.pubKey, event.createdAt, event.tags, event.content, event.sig + ), + chatRoomId + )?.toTranslationChapterEvent() + } + } + + @Test + fun `every archivable kind has a round-trip test above`() { + // The list and this file have to move together. A kind added to the + // allowlist without a case here is one nobody has checked can be rebuilt, + // and the failure is silent: the receiver drops it as a forgery. + assertEquals( + 6, + ArchiveEvent.ARCHIVABLE_KINDS.size, + "an archivable kind was added or removed; add or remove its round-trip case" + ) + } +} diff --git a/composeApp/src/jvmTest/kotlin/press/mantra/compose/managers/ArchiveAssemblyJvmTest.kt b/composeApp/src/jvmTest/kotlin/press/mantra/compose/managers/ArchiveAssemblyJvmTest.kt new file mode 100644 index 00000000..404f0849 --- /dev/null +++ b/composeApp/src/jvmTest/kotlin/press/mantra/compose/managers/ArchiveAssemblyJvmTest.kt @@ -0,0 +1,386 @@ +package press.mantra.compose.managers + +import androidx.room3.Room +import com.vitorpamplona.quartz.nip01Core.core.Event +import com.vitorpamplona.quartz.nip01Core.core.hexToByteArray +import com.vitorpamplona.quartz.nip01Core.crypto.EventHasher +import com.vitorpamplona.quartz.nip01Core.signers.EventTemplate +import fr.acinq.bitcoin.ByteVector +import fr.acinq.bitcoin.ByteVector32 +import fr.acinq.bitcoin.PrivateKey +import fr.acinq.bitcoin.crypto.frost.Frost +import fr.acinq.bitcoin.crypto.frost.IndividualNonce +import fr.acinq.bitcoin.crypto.frost.KeyMaterial +import fr.acinq.bitcoin.crypto.frost.SecretNonce +import fr.acinq.bitcoin.crypto.frost.Session +import kotlin.test.AfterTest +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertNotNull +import kotlin.test.assertTrue +import kotlin.time.Instant +import kotlinx.coroutines.runBlocking +import press.mantra.compose.database.MantraDatabase +import press.mantra.compose.database.builder.getRoomDatabase +import press.mantra.compose.database.model.ChatMessage +import press.mantra.compose.database.model.ChatRoom +import press.mantra.compose.database.model.MantraDialect +import press.mantra.compose.database.model.NostrEvent +import press.mantra.compose.database.model.Profile +import press.mantra.compose.extensions.toHex +import press.mantra.compose.nostr.archive.ArchiveEvent +import press.mantra.compose.nostr.frost.GroupKeyStateEvent +import press.mantra.compose.nostr.nip30303.ArtifactEvent +import press.mantra.compose.nostr.nip30303.ChapterEvent +import press.mantra.compose.nostr.nip30303.ChunkEvent +import press.mantra.compose.nostr.nip30303.DialectEvent +import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionEvent +import press.mantra.compose.nostr.nip30303.TranslationChapterEvent + +/** + * Assembling a room's archive out of the rows a device actually holds. + * + * The seed is the real inbound path -- `ChatMessage.applyInnerEvent`, the same + * call a finished signing session makes -- so what is archived here is what a + * member's database really contains rather than rows built to suit the test. + * That matters because the whole risk in this direction is a rebuild that + * differs from what was signed, and rows assembled by hand would hide it. + */ +class ArchiveAssemblyJvmTest { + private val db: MantraDatabase = getRoomDatabase(Room.inMemoryDatabaseBuilder()) + + @AfterTest + fun closeDb() = db.close() + + private val participants = 3 + private val threshold = 2 + + private val keyMaterial: KeyMaterial = Frost.trustedDealerKeygen( + thresholdSecretKey = PrivateKey( + ByteVector32("1c0ffee0000000000000000000000000000000000000000000000000000000a1") + ), + nParticipants = participants, + threshold = threshold + ) + + private val room: SharedKeyDerivation.Derived = SharedKeyDerivation.derive( + thresholdPublicKey = keyMaterial.thresholdPublicKey.value.toHex(), + path = SharedKeyDerivation.MARMOT_ADMIN_GROUP_PATH + ) + + /** The room's id and the key it signs as are one value. */ + private val chatRoomId = room.hex + + private val newMember = "9".repeat(64) + + private fun groupSignature(eventId: String): String { + val message = ByteVector(eventId.hexToByteArray()) + val signerIds = listOf(0, 1) + + val nonces = signerIds.map { signerId -> + SecretNonce.generate( + sessionRandom = ByteVector32("a".repeat(63) + "${signerId + 1}"), + secretShare = keyMaterial.secretShares[signerId], + publicShare = keyMaterial.publicShares[signerId], + tweakedThresholdPublicKey = room.cache.tweakedPublicKey, + message = message, + extraInput = null + ) + } + + val signingSession = Session.create( + aggregatedNonce = IndividualNonce.aggregate(nonces.map { it.second }).right!!, + signerIds = signerIds.map { it.toUInt() }, + signerPublicShares = signerIds.map { keyMaterial.publicShares[it] }, + nParticipants = participants, + threshold = threshold, + tweakCache = room.cache, + message = message + ) + + val partials = signerIds.mapIndexed { position, signerId -> + signingSession.sign( + nonces[position].first, + keyMaterial.secretShares[signerId], + signerId.toUInt() + ).right!! + } + + return signingSession.aggregateSigs(partials).right!!.toByteArray().toHex() + } + + /** A template as a finished session leaves it: authored and signed by the room. */ + private fun signed(template: EventTemplate<*>): Event { + val id = EventHasher.hashId( + pubKey = chatRoomId, + createdAt = template.createdAt, + kind = template.kind, + tags = template.tags, + content = template.content + ) + + return Event( + id = id, + pubKey = chatRoomId, + createdAt = template.createdAt, + kind = template.kind, + tags = template.tags, + content = template.content, + sig = groupSignature(id) + ) + } + + /** What `FrostSigningManager.applySignedEvent` does, through the same call. */ + private suspend fun apply(template: EventTemplate<*>): Event { + val event = signed(template) + + ChatMessage.applyInnerEvent( + database = db, + groupId = chatRoomId, + event = event, + marmotGroupEventId = null, + marmotInnerEventId = null, + senderPublicKey = chatRoomId, + isUserMessage = false, + createdAt = Instant.fromEpochSeconds(event.createdAt) + ) + + return event + } + + /** ChatRoom -> Profile -> NostrEvent, the foreign key chain a room hangs off. */ + private suspend fun seedRoom() { + val nostrEventId = "c".repeat(64) + db.nostrEventDao().upsert( + NostrEvent( + id = nostrEventId, + pubKey = newMember, + kind = 0, + tags = emptyArray(), + content = "{}", + sig = "0".repeat(128), + ) + ) + db.profileDao().upsert( + Profile(publicKey = newMember, userName = "member", nostrEventId = nostrEventId) + ) + db.chatRoomDao().upsert( + ChatRoom( + id = chatRoomId, + userPublicKey = newMember, + subject = "#admins", + description = null, + mlsGroupState = null, + ) + ) + } + + /** A room with one artifact, one chapter, two chunks and one translation. */ + private suspend fun seedSignedWork() { + seedRoom() + + val dialect = apply( + DialectEvent.build(name = "isiZulu", country = "ZA", language = "zu", createdAt = 1_700_000_000L) + ) + + val artifact = apply( + ArtifactEvent.build( + name = "In Detention", + url = "https://example.com/in-detention", + visibility = "private", + license = "cc", + dialectId = dialect.id, + versionLabel = "1.0", + createdAt = 1_700_000_010L + ) + ) + + // Derived from the artifact rather than signed, which is exactly why it + // is not archived and why its label has to be read back off this row. + val version = assertNotNull( + db.mantraArtifactVersionDao().getArtifactVersionsByArtifactId(artifact.id).firstOrNull(), + "applying the artifact should have derived its first version" + ) + + val chapter = apply( + ChapterEvent.build( + artifactVersionId = version.id, + name = "Chapter One", + originalText = "He was a man of parts. He had many.", + index = 0, + wordCount = 8, + characterCount = 35, + createdAt = 1_700_000_020L + ) + ) + + apply( + ChunkEvent.build( + chapterId = chapter.id, + text = "He was a man of parts.", + index = 0, + wordCount = 6, + characterCount = 22, + createdAt = 1_700_000_030L + ) + ) + apply( + ChunkEvent.build( + chapterId = chapter.id, + text = "He had many.", + index = 1, + wordCount = 3, + characterCount = 12, + createdAt = 1_700_000_031L + ) + ) + + val translationVersion = apply( + TranslationArtifactVersionEvent.build( + artifactVersionId = version.id, + dialectId = dialect.id, + name = "isiZulu", + visibility = "private", + license = "cc", + createdAt = 1_700_000_040L + ) + ) + + apply( + TranslationChapterEvent.build( + translationArtifactVersionId = translationVersion.id, + chapterId = chapter.id, + index = 0, + createdAt = 1_700_000_050L + ) + ) + } + + private suspend fun archivedPayloads(): List = + ArchiveManager.assemble(db, chatRoomId, newMember) + .flatMap { assertNotNull(ArchiveEvent.decodePage(it.content)) } + + @Test + fun `every payload in an archive is one the room signed`() = runBlocking { + seedSignedWork() + + val payloads = archivedPayloads() + + assertTrue(payloads.isNotEmpty(), "a room with signed work archives something") + payloads.forEach { + assertTrue( + GroupKeyStateEvent.isSignedByRoom(it, chatRoomId), + "kind ${it.kind} (${it.id.take(8)}) was archived without a signature the room can be checked for" + ) + } + } + + @Test + fun `an archive holds every kind the group signed, and each of them once`() = runBlocking { + seedSignedWork() + + val byKind = archivedPayloads().groupBy { it.kind } + + assertEquals(1, byKind[DialectEvent.KIND]?.size) + assertEquals(1, byKind[ArtifactEvent.KIND]?.size) + assertEquals(1, byKind[ChapterEvent.KIND]?.size) + assertEquals(2, byKind[ChunkEvent.KIND]?.size) + assertEquals(1, byKind[TranslationArtifactVersionEvent.KIND]?.size) + assertEquals(1, byKind[TranslationChapterEvent.KIND]?.size) + + assertEquals(ArchiveEvent.ARCHIVABLE_KINDS, byKind.keys) + } + + @Test + fun `an archive is in dependency order across its whole length`() = runBlocking { + seedSignedWork() + + val ranks = archivedPayloads().map { assertNotNull(ArchiveEvent.applyRank(it.kind)) } + + assertEquals(ranks.sorted(), ranks, "an archive out of order is a foreign key violation") + } + + @Test + fun `a member's own rumor is not part of the group's record`() = runBlocking { + seedSignedWork() + + // What `MantraDao.addDialect` writes: the member's key, no signature. It + // is real work in the room and the group never signed it, so it cannot be + // handed to somebody who has no way to check it. + db.mantraDialectDao().upsert( + MantraDialect( + id = "f".repeat(64), + publicKey = newMember, + name = "seSotho", + country = "ZA", + language = "st", + signature = "", + chatRoomId = chatRoomId, + createdAt = Instant.fromEpochSeconds(1_700_000_060L), + ) + ) + + assertEquals( + 2, + db.mantraDialectDao().getDialectsByChatRoomId(chatRoomId).size, + "the rumor should be on disk" + ) + assertEquals( + 1, + archivedPayloads().count { it.kind == DialectEvent.KIND }, + "the unsigned dialect should not have been archived" + ) + } + + @Test + fun `a room with nothing signed archives nothing, and says so rather than failing`() = runBlocking { + seedRoom() + + assertEquals(emptyList(), ArchiveManager.assemble(db, chatRoomId, newMember)) + } + + @Test + fun `an archive of one page still says it is one of one`() = runBlocking { + seedSignedWork() + + val pages = ArchiveManager.assemble(db, chatRoomId, newMember) + + assertEquals(1, pages.size, "this much work fits one page") + + val page = ArchiveEvent( + id = "d".repeat(64), + pubKey = chatRoomId, + createdAt = pages.single().createdAt, + tags = pages.single().tags, + content = pages.single().content, + sig = "" + ) + + assertEquals(0, page.page()?.index) + assertEquals(1, page.page()?.count) + assertEquals(newMember, page.recipient()) + assertNotNull(page.archiveId()) + } + + @Test + fun `two archives of the same rows do not share an id`() = runBlocking { + seedSignedWork() + + fun idOf(template: EventTemplate) = ArchiveEvent( + id = "d".repeat(64), + pubKey = chatRoomId, + createdAt = template.createdAt, + tags = template.tags, + content = template.content, + sig = "" + ).archiveId() + + // Two members answering one request is the ordinary case, and their page + // counts differ whenever their databases do. Sharing an id would let one + // archive's pages be counted towards the other's total. + assertTrue( + idOf(ArchiveManager.assemble(db, chatRoomId, newMember).single()) != + idOf(ArchiveManager.assemble(db, chatRoomId, newMember).single()) + ) + } +} diff --git a/docs/member-archive.md b/docs/member-archive.md index ba0ecedc..73e9a1aa 100644 --- a/docs/member-archive.md +++ b/docs/member-archive.md @@ -265,9 +265,13 @@ array containing a non-event is refused whole. `ArchiveManager.assemble(database, chatRoomId, recipient): List>` -Read every group-signed event this device holds for the room, order it, pack it -into pages, and queue each page as a `MarmotInnerEvent` -- the ordinary outbound -path, nothing new. +Read every group-signed event this device holds for the room, order it, and pack +it into pages. + +**Assembling only.** Queueing each page as a `MarmotInnerEvent` moved to Phase 5, +where the thing that decides *when* to send one lives. Splitting them keeps this +phase testable against a real database with no outbound path in the way, and +keeps the decision about transcript lines next to the decision about triggers. ### Where the events come from @@ -283,6 +287,44 @@ still verifies, that entity cannot be archived at all, and it is better to find out in an afternoon than in Phase 8. A round-trip test per kind, over rows produced by a real signing session, is the gate on the rest of this work. +**It was right to run it first.** Every `toXEvent()` in the codebase turned out +to be unused in production -- written for exactly this and never called, so the +"tag order matches build so the event id round-trips" comments on them were +claims nothing had checked. One was wrong. `MantraArtifact.toArtifactEvent` put +the alt tag last where `ArtifactEvent.build` puts it first, *and* left out the +version metadata tag entirely -- because that tag is not on the artifact row at +all. `fromArtifactEvent` reads the artifact's own fields and drops the version +label, which `applyInnerEvent` has by then turned into the artifact's first +`MantraArtifactVersion`. So the label comes back as a parameter, read off the +initial version -- the one whose `createdAt` is the artifact's, since +`initialVersionOf` derives it from the same event. + +Neither fault would have shown up as an error. Both produce a well-formed +artifact whose id no longer matches its fields, which every receiver drops as a +forgery, silently, one kind at a time. + +### What is actually archivable, which is less than it looks + +Only six of the thirteen nip30303 kinds ever reach a signing session. The rest +travel as rumors -- empty signature, member author, vouched for by the MLS frame +they arrived in and by nothing that survives leaving it -- so they cannot be put +in front of somebody who has no way to check them. + +| kind | | why | +|---|---|---| +| 30304 Dialect, 30300 Artifact, 30302 Chapter, 30303 Chunk, 30306 TranslationArtifactVersion, 30308 TranslationChapter | archivable | proposed through `proposeSigning`/`proposeSigningBatch` | +| 30301 ArtifactVersion | no | never signed; derived from the signed artifact on arrival, which is what keeps a chapter's foreign key satisfied without archiving it | +| 30309 TranslationChunk | no | **the translated text itself.** `MantraDao.saveTranslation` submits it as the member's own rumor | +| 30311 Translation | no | nothing builds one; the inbound arm exists and no producer does | +| 30305, 30307, 30310 contributor lists | no | `applyInnerEvent` has no arm that writes a row for any of them | + +The second row of "no" is the one that matters and it is not a detail: **an +archive restores everything a translation hangs on and not the translation.** A +new member gets the dialects, the artifacts, the chapters, the source chunks, +which translations exist and their chapter scaffolding -- the whole structure, +enough to start translating -- and none of the translated text. See +[What this does not do](#what-this-does-not-do). + ### Ordering Room enforces the shape, so an archive out of order is a foreign key violation @@ -574,6 +616,29 @@ general answer for the reason [frost-batch-signing.md](./frost-batch-signing.md#appendix--what-was-considered-and-rejected) gives for manifests. Worth revisiting once anything depends on completeness. +**The translated text is not archived.** The largest gap, and it follows from the +same rule everything else here follows from. A translation chunk is submitted by +`MantraDao.saveTranslation` as the member's own rumor, because a translation is +one member's work rather than a group decision -- so it carries no signature, and +an archive carrying it would be asking its recipient to believe whoever sent it. +A new member therefore receives the entire structure and none of the prose. + +Three ways out, in increasing order of how much they cost: + + - **Send them anyway, marked unverified**, and let the reader see which rows + came with a group signature and which came on one member's word. Cheap, and it + gives up the property that makes the rest of this safe, so it needs its own + screen language rather than a quiet inclusion. + - **Corroborate.** Every member's archive is an independent copy, so a + translation two members' archives agree on is a claim two devices make. That + is a real strengthening and it needs a second archive to compare against, + which the request path already makes ordinary. + - **Sign them.** The group already puts a quorum behind a chapter and its + chunks; putting one behind a translation would make it archivable like + everything else. It is also a product decision about whether translating is an + act of the group or of a member, which is not a decision this document gets to + make. + **The chat is gone and stays gone.** By design, restated here because it is the first thing a new member will notice and the archive is what makes them expect otherwise.