mantra/mantra-kmp
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat: apply an archive a member is sent, and sweep what arrived too early

Browse Source 
Phase 4 of docs/member-archive.md, and the half where the security lives. A
member who holds no share, took part in no signing session and cannot decrypt a
word of the room's history now ends up with the same rows as everybody else --
and gets there without trusting whoever sent them.

**Intercepted in `fromGroupEventResult`, not in `applyInnerEvent`.** An archive
is neither a document nor a submission, and deciding whether to act on one needs
the active key, which `applyInnerEvent` has no business knowing. That is the same
reason the gift wrap above it is handled there, so it sits next to it.

**Verification per payload, framing per page.** A forged payload costs itself and
nothing else -- the rule `MarmotInboundManager` already uses for a forged direct
message, and for the same reason: this runs inside the inbound transaction and
one bad event must not take the room down with it. Refusing the whole page would
also let a single forgery deny an entire archive. The page's own framing stays
all-or-nothing, because a page that will not parse has lost the thing that says
what it contains.

**The allowlist runs before the signature check, and it is not a formality.**
Verification admits an event to the apply path on the strength of the group's
signature, which makes every kind the group has ever signed replayable by any
member at any time. There is a test that puts a genuine, still-verifying
`GroupKeyStateEvent` in a hand-rolled page -- `ArchiveEvent.build` refuses to make
one, which is the outbound half of the same rule -- and asserts the receiver's key
state does not move.

**The chat line is dropped, deliberately.** `ChatMessage` has an `autoGenerate`
primary key, so there is no id to dedupe on and every applied payload would mint
a new row: a synthetic transcript dated now, and another one on every pass of the
sweep. The archive restores the work. The conversation is forward secret and
stays gone.

**A device that is not the named recipient does nothing.** It can read the page --
it is an ordinary group message, and it is the group's own history -- but it
already holds the work, and re-applying would rewrite every one of its rows to
point at an archive page rather than at the event that introduced it. That is
also what bounds the sweep: only the member being caught up ever builds the list.

**The sweep needs no table.** Pages arrive over relays in no order, so page 3 can
land before page 2 and its chunks have no chapter to hang off. Those throw a
foreign key violation and would be lost -- except the inbound path already stores
every inner event it decrypts, so re-reading them is the same shape
`FrostSigningManager.replayStoredMessages` has, for the same reason: nothing was
lost, it just had nowhere to go at the time.

Two things about the loop, the second found by a test:

Progress is measured by *failures falling*, not by rows written. "Repeat while a
pass applied something" does not terminate, because every write is an upsert and
succeeds forever. What strictly decreases is the count that threw.

And the result is the last pass rather than the sum of them. Accumulating counts
a payload once per pass it survived and reports failures a later pass went on to
fix, so `failed > 0` stops meaning "still missing" -- which is the only question
a caller asks it. Caught by strengthening the out-of-order test to assert that
the page completing an archive leaves nothing behind, rather than only that the
rows matched: without that, the test passed while reporting fourteen failures on
a fully converged database.

Seven tests over two real databases with the pages carried by hand. The one that
matters puts four forgeries in a page beside one honest dialect -- the room's id
as author with a made-up signature, a real quorum of another group, an event
edited after signing, and a member's own rumor, which is what everything on the
wire looks like today -- and asserts the receiver ends with exactly the honest
one. The rest: a full catch-up matches the sender row for row with the group's
signature intact, pages delivered backwards converge and are asserted to have
really failed first so the test cannot pass for the wrong reason, an archive
files no chat lines, a bystander applies none of it, and applying the same
archive twice changes nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kgothatso Ngako
2026-09-06 14:19:15 +02:00
parent acff66a22e
commit ed4410b972
4 changed files with 853 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
composeApp/src/commonMain/kotlin/press/mantra/compose/database/model/ChatMessage.kt
View File

@@ -10,6 +10,7 @@ import press.mantra.compose.database.model.traits.SoftDeletableEntity
import press.mantra.compose.database.model.traits.TimestampedEntity
import press.mantra.compose.database.model.traits.UserViewableEntity
import press.mantra.compose.exceptions.MarmotUnprocessableInnerEventException
import press.mantra.compose.managers.ArchiveManager
import press.mantra.compose.managers.GroupKeyStateManager
import press.mantra.compose.extensions.toHex
import com.vitorpamplona.quartz.marmot.GroupEventResult
@@ -25,6 +26,7 @@ import press.mantra.compose.nostr.nip30303.ArtifactVersionEvent
import press.mantra.compose.nostr.nip30303.ChapterEvent
import press.mantra.compose.nostr.nip30303.ChunkEvent
import press.mantra.compose.nostr.nip30303.DialectEvent
import press.mantra.compose.nostr.archive.ArchiveEvent
import press.mantra.compose.nostr.nip30303.SubmissionEvent
import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionContributorListEvent
import press.mantra.compose.nostr.nip30303.TranslationArtifactVersionEvent
@@ -613,6 +615,28 @@ data class ChatMessage(
)
}
// An archive is neither a document nor a submission: it is a
// bundle of documents addressed to one member who is missing
// them. Intercepted here rather than in applyInnerEvent for
// the same reason the gift wrap above is -- deciding whether
// to act needs the active key, which applyInnerEvent has no
// business knowing.
if (event.kind == ArchiveEvent.KIND) {
ArchiveManager.receive(
database = database,
chatRoomId = groupEventResult.groupId,
userPublicKey = activeKeyPair.pubKey.toHex(),
page = event,
)
// No chat line, and not for want of one worth writing.
// One line per archive is right; one per page is not, and
// the pages of an archive are not distinguishable from
// each other here. That is Phase 7's, and until then a
// silent catch-up beats a transcript full of envelopes.
return null
}
// A submission whose payload will not parse, or which carries
// another submission, is kept but not applied: there is nothing
// here we can turn into a row.

222
composeApp/src/commonMain/kotlin/press/mantra/compose/managers/ArchiveManager.kt
View File

@@ -7,6 +7,8 @@ import com.vitorpamplona.quartz.nip01Core.signers.EventTemplate
import com.vitorpamplona.quartz.utils.RandomInstance
import com.vitorpamplona.quartz.utils.TimeUtils
import press.mantra.compose.database.MantraDatabase
import press.mantra.compose.database.model.ChatMessage
import press.mantra.compose.database.model.MarmotInnerEvent
import press.mantra.compose.extensions.toHex
import press.mantra.compose.nostr.archive.ArchiveEvent
import press.mantra.compose.nostr.frost.GroupKeyStateEvent
@@ -98,6 +100,226 @@ object ArchiveManager {
}
}
// ---- Receiving ------------------------------------------------------
/**
* What one pass over a room's stored pages did.
*
* [skipped] and [failed] are kept apart because only one of them is worth
* trying again. A payload the allowlist or the signature refused will be
* refused identically forever; a payload that threw was probably missing a
* row it depends on, and the next page may bring it.
*/
data class Outcome(
val applied: Int = 0,
val skipped: Int = 0,
val failed: Int = 0,
) {
operator fun plus(other: Outcome) = Outcome(
applied = applied + other.applied,
skipped = skipped + other.skipped,
failed = failed + other.failed,
)
}
/**
* Take delivery of an archive page for [userPublicKey].
*
* The page is already on disk by the time this runs -- the inbound path
* stores every inner event it decrypts before dispatching on kind -- so this
* does not apply the arriving page as such. It sweeps every page the room
* has, which covers the new one and any that arrived before the rows they
* depend on.
*
* **A device that is not the named recipient does nothing.** The page is an
* ordinary group message and it can read it; it has no reason to. It already
* holds the work, and re-applying would rewrite every one of its rows to
* point at an archive page rather than at the event that introduced it. That
* is also what keeps the sweep bounded: only the member being caught up ever
* builds the list.
*/
suspend fun receive(
database: MantraDatabase,
chatRoomId: String,
userPublicKey: HexKey,
page: Event,
): Outcome {
val recipient = ArchiveEvent(
id = page.id,
pubKey = page.pubKey,
createdAt = page.createdAt,
tags = page.tags,
content = page.content,
sig = page.sig,
).recipient()
if (!recipient.equals(userPublicKey, ignoreCase = true)) {
logger.d("Archive page ${page.id.take(8)} is for ${recipient?.take(8)}; not applying")
return Outcome()
}
return sweep(database, chatRoomId, userPublicKey)
}
/**
* Apply every archive page this room holds for [userPublicKey], repeatedly,
* until a pass stops making progress.
*
* Pages arrive over relays in no order, so page 3 can land before page 2 and
* its chunks have no chapter to hang off yet. Those payloads throw a foreign
* key violation and would be lost -- unless something runs them again.
*
* **No table is needed for that**, because the inbound path already stores
* every inner event it decrypts. This is the same shape
* `FrostSigningManager.replayStoredMessages` has, and for the same reason:
* nothing was actually lost, it just had nowhere to go at the time.
*
* **Progress is measured by failures falling, not by rows written.** Every
* write here is an upsert keyed on the event id, so "applied something" is
* true on every pass forever and would not terminate. A pass that fails fewer
* payloads than the last one learned something; a pass that does not is as
* far as this archive gets, and the rest is a hole to be filled by another
* page or another archive.
*
* **The answer is the last pass, not the sum of them.** Accumulating would
* count a payload once per pass it survived and report failures the next pass
* went on to fix, so `failed > 0` would stop meaning "still missing". The
* final pass is the settled state: what these pages can apply, what they will
* never apply, and what is still waiting on something that has not arrived.
*/
suspend fun sweep(
database: MantraDatabase,
chatRoomId: String,
userPublicKey: HexKey,
): Outcome {
val pages = database.marmotInnerEventDao()
.getByChatRoomAndKinds(chatRoomId, listOf(ArchiveEvent.KIND))
.filter { addressedTo(it, userPublicKey) }
if (pages.isEmpty()) return Outcome()
var pass = Outcome()
var previousFailures = Int.MAX_VALUE
while (true) {
pass = Outcome()
pages.forEach { pass += applyPage(database, chatRoomId, it) }
if (pass.failed == 0 || pass.failed >= previousFailures) {
if (pass.failed > 0) {
logger.w(
"Stopped sweeping $chatRoomId with ${pass.failed} payload(s) still " +
"unapplied: nothing in the pages it holds satisfies them"
)
}
break
}
previousFailures = pass.failed
}
logger.i(
"Swept ${pages.size} archive page(s) for $chatRoomId: " +
"${pass.applied} applied, ${pass.skipped} skipped, ${pass.failed} left"
)
return pass
}
private fun addressedTo(stored: MarmotInnerEvent, userPublicKey: HexKey): Boolean =
ArchiveEvent(
id = stored.id,
pubKey = stored.publicKey,
createdAt = stored.createdAt.epochSeconds,
tags = stored.tags,
content = stored.content,
sig = "",
).recipient().equals(userPublicKey, ignoreCase = true)
/**
* One page, checked payload by payload and applied in dependency order.
*
* **Per payload, not per page.** A forged payload sitting beside honest ones
* costs itself and nothing else -- the same rule `MarmotInboundManager` uses
* for a forged direct message, and for the same reason: this runs inside the
* inbound transaction, and one bad event must not take the room down with it.
* Refusing the whole page would also let a single forgery deny an entire
* archive.
*
* The page's own framing is still all-or-nothing; see
* [ArchiveEvent.decodePage] for why those two are not in tension.
*/
private suspend fun applyPage(
database: MantraDatabase,
chatRoomId: String,
stored: MarmotInnerEvent,
): Outcome {
val payloads = ArchiveEvent.decodePage(stored.content)
if (payloads == null) {
logger.w("Archive page ${stored.id.take(8)} does not read as a page; dropping it")
return Outcome()
}
var outcome = Outcome()
ArchiveEvent.inApplyOrder(payloads).forEach { payload ->
// The allowlist first, and it is not a formality. Verification admits
// an event to the apply path on the strength of the group's
// signature, which makes every kind the group has ever signed
// replayable by any member at any time -- a GroupKeyStateEvent from
// an earlier epoch passes the signature check perfectly.
if (!ArchiveEvent.isArchivable(payload.kind)) {
logger.w(
"Archive page ${stored.id.take(8)} carries kind ${payload.kind}, " +
"which an archive may not deliver; dropping ${payload.id.take(8)}"
)
outcome += Outcome(skipped = 1)
return@forEach
}
if (!GroupKeyStateEvent.isSignedByRoom(payload, chatRoomId)) {
logger.w(
"Archive page ${stored.id.take(8)} carries ${payload.id.take(8)}, " +
"which room $chatRoomId did not sign; dropping it"
)
outcome += Outcome(skipped = 1)
return@forEach
}
outcome += try {
// The chat line this returns is deliberately dropped rather than
// filed. ChatMessage has an autoGenerate primary key, so there is
// no id to dedupe on and every applied payload would mint a new
// row -- giving the recipient a synthetic transcript dated now,
// and another one on every pass of the sweep. The archive
// restores the work; the conversation is forward secret and stays
// gone.
ChatMessage.applyInnerEvent(
database = database,
groupId = chatRoomId,
event = payload,
marmotGroupEventId = stored.marmotGroupEventId,
marmotInnerEventId = stored.id,
senderPublicKey = payload.pubKey,
isUserMessage = false,
createdAt = stored.createdAt,
)
Outcome(applied = 1)
} catch (error: Throwable) {
// Almost always a foreign key: this payload names a row that is
// in a page which has not arrived yet. Retryable, which is what
// the sweep is for, so it is counted rather than logged loudly.
logger.d(
"Could not apply ${payload.id.take(8)} from archive page " +
"${stored.id.take(8)} yet: ${error.message}"
)
Outcome(failed = 1)
}
}
return outcome
}
/**
* The room's rows, rebuilt into the events they came from.
*