feat(marmot): send the canonical app payload, and an envelope for direct messages (v2 Phase 2)

What goes into MLS is now the Marmot app event the spec defines: six members and
no `sig`. The rumor serialised as an Event carried `"sig":""`, which a conformant
decoder refuses outright -- a payload with a signature would be a standalone relay
event, republishable as a signed statement by its author. The id does not move,
because NIP-01 never hashed the signature, so every row filed under an inner
event's id stays valid; and the build already in the field reads the new shape,
because Event.fromJsonOrNull defaults a missing signature. This switch alone needs
no gating and is made everywhere at once. The `encoding` tag comes off the
application kind:445 while here -- transports/nostr.md forbids it on every shape.
docs/marmot-v2.md, Phase 2.

A direct message could not make that switch as it was. Its payload is a NIP-59
wrap signed by a throwaway key, which a conformant peer refuses twice over: for
the sig member and for an author that is not the MLS sender. In a current-profile
room the wrap now travels inside an envelope -- an unsigned app event of kind
30330 authored by the sender, one `p` tag for the recipient, the wrap's JSON as
content. The envelope passes the author check like any payload, so the carve-out
in mip03Rejection that admitted bare wraps is confined to legacy rooms by a new
`legacyRoom` flag: in a current-profile room a bare kind:1059 is refused as every
other conformant client refuses it. Every property the throwaway key was chosen
for survives -- the envelope is unsigned and so a claim rather than a proof, the
wrap inside is still signed by nobody, and the seal-to-leaf binding in classify
still decides who wrote the words. The line for a message this device cannot read
now hangs off the row the payload was filed under (the envelope's id in a
current-profile room), which ChatMessage's foreign key requires.

The envelope is gated, and the gate is the room's profile: an old build renders
it as a raw-JSON bubble, so a legacy room keeps sending the bare wrap, and a
current-profile room -- which admits only leaves carrying an identity proof,
which only a build past Phase 3 can make -- never holds an old build. That is the
flag for every wire change after this one.

A conformant decoder bounds the whole app payload at 64 KiB, and a chronicle page
rides inside one as JSON-escaped content. The new bounds test caught what the
arithmetic missed: dense JSON payloads escape to half again their size, so a 48
KiB page became a 70 KiB app event. The packer now measures a page by
ChronicleEvent.escapedByteSize -- the bytes it will occupy inside the envelope --
against MAX_PAGE_BYTES, lowered from 64 to 48 KiB, and MAX_PAGE_EVENTS drops from
128 to 96 so the count cap still binds under it, as the comment on the two caps
insists. The receiver's raw-bytes check is unchanged and now holds by
construction.

Tests: MarmotMip03CarveOutTest becomes the test that the exemption is confined to
legacy rooms and that an envelope passes the ordinary check; MarmotDirectMessageTest
round-trips the envelope through the strict decoder and opens the wrap inside;
MarmotPayloadBoundsTest asserts the largest chronicle pages and an envelope are
conformant app events, including the dense-content page measured the way the
packer measures it. The ingest test's tamper case flips a decoded byte rather
than the base64 tail, which could change nothing but padding. Full JVM suite:
1,148 tests pass. The direct-message and chronicle notes record the changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kgothatso NgakoandClaude Opus 5 committed 2026-09-13 22:32:30 +02:00
1 parent a68cad06a4
commit 3cdce6a908
12 files changed
+495 -67

No files matched your search

@@ -28,8 +28,8 @@ import to.curare.compose.nostr.MarmotDelivery
import to.curare.compose.nostr.MarmotDirectMessage
import to.curare.compose.nostr.Relays
import co.touchlab.kermit.Logger
import com.vitorpamplona.quartz.marmot.foundation.appEvents.MarmotAppEvent
import com.vitorpamplona.quartz.marmot.mip00KeyPackages.KeyPackageUtils
import com.vitorpamplona.quartz.marmot.mip00KeyPackages.tags.EncodingTag
import com.vitorpamplona.quartz.marmot.mip01Groups.MarmotGroupData
import com.vitorpamplona.quartz.marmot.mip02Welcome.WelcomeEvent
import com.vitorpamplona.quartz.marmot.mip03GroupMessages.GroupEvent
@@ -741,12 +741,20 @@ abstract class MarmotOutboundDao(
marmotInnerEvent: MarmotInnerEvent,
nostrSignerSync: NostrSignerSync
): GroupEvent {
// A direct message is gift wrapped for one member before it goes into MLS, so
// what the group carries is a kind:1059 whose id is not this row's. Everything
// downstream -- MLS encrypt, outer ChaCha20, kind:445, persistence, broadcast --
// is the same either way. See MarmotDirectMessage.
val innerEvent: Event = marmotInnerEvent.directMessageRecipientPublicKey?.let { recipient ->
MarmotDirectMessage.wrap(
// The room's profile decides the shape of what goes into MLS. See
// docs/marmot-v2.md, Phase 2: it is the feature flag for every wire change
// an old build cannot read, because a current-profile room admits only
// leaves that carry an identity proof, and only a build past Phase 3 can
// make one.
val isCurrentProfileRoom = mlsGroup.currentGroupState().isCurrentProfile
val plainTextBytes: ByteArray = marmotInnerEvent.directMessageRecipientPublicKey?.let { recipient ->
// A direct message is gift wrapped for one member before it goes into
// MLS, so what the group carries is a kind:1059 whose id is not this
// row's. Everything downstream -- MLS encrypt, outer ChaCha20, kind:445,
// persistence, broadcast -- is the same either way. See
// MarmotDirectMessage.
val wrap = MarmotDirectMessage.wrap(
signer = nostrSignerSync,
recipientPublicKey = recipient,
kind = marmotInnerEvent.kind,
@@ -754,19 +762,44 @@ abstract class MarmotOutboundDao(
tags = marmotInnerEvent.tags,
content = marmotInnerEvent.content
)
} ?: RumorAssembler.assembleRumor(
pubKey = nostrSignerSync.pubKey,
ev = EventTemplate(
createdAt = marmotInnerEvent.createdAt.epochSeconds,
content = marmotInnerEvent.content,
tags = marmotInnerEvent.tags,
kind = marmotInnerEvent.kind
if (isCurrentProfileRoom) {
// Inside an unsigned envelope authored by the sender, which is what
// a conformant peer accepts. The bare wrap is signed and names a
// throwaway key, and fails their author check twice over.
MarmotDirectMessage.envelope(
senderPublicKey = nostrSignerSync.pubKey,
recipientPublicKey = recipient,
wrap = wrap,
createdAt = marmotInnerEvent.createdAt.epochSeconds,
).encodeToPayload()
} else {
// The bare wrap, signature and all: the one shape every build in a
// legacy room reads, through the carve-out in mip03Rejection.
wrap.toJson().encodeToByteArray()
}
} ?: run {
val rumor = RumorAssembler.assembleRumor<Event>(
pubKey = nostrSignerSync.pubKey,
ev = EventTemplate(
createdAt = marmotInnerEvent.createdAt.epochSeconds,
content = marmotInnerEvent.content,
tags = marmotInnerEvent.tags,
kind = marmotInnerEvent.kind
)
)
)
logger.d("InnerEvent: ${innerEvent.toJson()}")
// The canonical Marmot app event: six members and no `sig`. The rumor
// serialised as an Event carries `"sig":""`, which a conformant decoder
// refuses outright -- a payload with a signature would be a standalone
// relay event, republishable as a signed statement by its author. The
// id does not move, because NIP-01 never hashed the signature, so every
// row filed under this event's id stays valid, and every build already
// in the field reads the shape: Event.fromJsonOrNull defaults a missing
// signature. This switch alone needs no gating.
MarmotAppEvent.fromEvent(rumor).encodeToPayload()
}
val plainTextBytes = innerEvent.toJson().encodeToByteArray()
logger.d("plainTextBytes: ${plainTextBytes.toHex()}")
// Step 1: MLS encrypt
@@ -787,15 +820,16 @@ abstract class MarmotOutboundDao(
val createdAt = TimeUtils.now()
val expirationTime = disappearingExpiration(mlsGroup, createdAt)
// No `encoding` tag. `transports/nostr.md`: a sender MUST NOT add one to any
// event shape -- a receiver that switched decoders on it could be steered
// into a different parse of the same bytes -- and the commit template below
// never carried one either.
val template = GroupEvent.build(
encryptedContentBase64 = encryptedContent,
nostrGroupId = localChatRoom.chatRoom.id,
createdAt = createdAt,
) {
if (expirationTime != null) expiration(expirationTime)
addUnique(
EncodingTag.assemble()
)
}
// Step 4: Sign with a fresh ephemeral keypair
@@ -661,6 +661,12 @@ data class ChatMessage(
*
* The decision is [MarmotDirectMessage.classify]'s, which is where it is tested;
* this turns it into rows. See docs/marmot-direct-messages.md.
*
* [filedAs] is the id of the `MarmotInnerEvent` row the caller wrote for the
* payload as it arrived -- the wrap's own id in a legacy room, the envelope's in
* a current-profile one. The line for a message this device cannot read has to
* hang off a row that exists, and `ChatMessage.marmotInnerEventId` is a foreign
* key that says so.
*/
private suspend fun directMessage(
database: CurareDatabase,
@@ -668,6 +674,7 @@ data class ChatMessage(
groupEvent: GroupEvent,
chatRoomId: String,
wrap: Event,
filedAs: HexKey,
senderIdentity: HexKey?
): ChatMessage? {
val recipientPublicKey = wrap.tags.firstTagValue("p")
@@ -700,7 +707,7 @@ data class ChatMessage(
// id every time, so this is what keeps a second delivery from becoming
// a second line. ChatMessage.id is autogenerated and would take a
// duplicate.
if (database.chatMessageDao().getChatMessagesByMarmotInnerEventId(wrap.id) != null) {
if (database.chatMessageDao().getChatMessagesByMarmotInnerEventId(filedAs) != null) {
return null
}
@@ -708,7 +715,7 @@ data class ChatMessage(
giftWrapPayloadId = null,
messageType = TYPE_DIRECT_MESSAGE,
marmotGroupEventId = groupEvent.id,
marmotInnerEventId = wrap.id,
marmotInnerEventId = filedAs,
senderPublicKey = senderIdentity ?: wrap.pubKey,
directMessageRecipientPublicKey = recipientPublicKey,
isUserMessage = false,
@@ -821,7 +828,11 @@ data class ChatMessage(
// A direct message is not a nip30303 payload to apply. It is
// gift wrapped for one member, and what this device does with it
// depends only on whether its key opens it.
// depends only on whether its key opens it. A legacy room carries
// the wrap itself; a current-profile room carries it inside an
// envelope authored by the sender -- see MarmotDirectMessage. The
// row above was filed under the payload's own id either way, and
// that is the id the line for a message we cannot read hangs off.
if (event.kind == GiftWrapEvent.KIND) {
return directMessage(
database = database,
@@ -829,6 +840,24 @@ data class ChatMessage(
groupEvent = groupEvent,
chatRoomId = groupEventResult.groupId,
wrap = event,
filedAs = event.id,
senderIdentity = senderIdentity
)
}
if (event.kind == MarmotDirectMessage.ENVELOPE_KIND) {
val wrap = MarmotDirectMessage.wrapInside(event)
if (wrap == null) {
logger.e("Dropping direct message envelope ${event.id}: it holds no wrap")
return null
}
return directMessage(
database = database,
activeKeyPair = activeKeyPair,
groupEvent = groupEvent,
chatRoomId = groupEventResult.groupId,
wrap = wrap,
filedAs = event.id,
senderIdentity = senderIdentity
)
}
@@ -824,6 +824,12 @@ object ChronicleManager {
* checked because they bind different chronicles -- a room of one-line dialects
* hits the count first and a room of chapters hits the bytes.
*
* The bytes are the page's *escaped* bytes -- what it will occupy as the
* `content` of the Marmot app event that carries it, which is what a
* conformant decoder bounds. Dense JSON payloads grow by half on the way in,
* and a page measured raw was over that bound while under this one. See
* [ChronicleEvent.escapedByteSize].
*
* An event too large to share a page with anything is given one of its own.
* One too large for even that is dropped with a log rather than failing the
* chronicle: a chapter nobody can chronicle is a hole, and a member who gets
@@ -837,7 +843,7 @@ object ChronicleManager {
events.forEach { event ->
// Plus the comma this event needs if it is not first on its page.
val size = event.toJson().encodeToByteArray().size + 1
val size = ChronicleEvent.escapedByteSize(event.toJson()) + 1
if (2 + size > ChronicleEvent.MAX_PAGE_BYTES) {
logger.w(
@@ -66,21 +66,25 @@ object MarmotInboundManager {
/**
* Why an inner application event may not be accepted from this sender, or null.
*
* MIP-03: an inner event's `pubkey` MUST equal the MLS sender's credential identity,
* so a member cannot mint events claiming a different author.
* An inner event's `pubkey` MUST equal the MLS sender's credential identity, so a
* member cannot mint events claiming a different author. MIP-03 said so and
* `foundation/application-messages.md` says the same.
*
* A gift wrap is exempt. Its pubkey is a throwaway key by construction -- see
* [MarmotDirectMessage] -- so there is no author field to compare and the check
* cannot be applied as written. The authorship claim moves inside, to the signed
* kind:13 seal, which [MarmotDirectMessage.classify] binds to this same identity
* before a word is rendered: a verified signature bound to an MLS leaf, rather than a
* plaintext field compared to one.
* In a legacy room a gift wrap is exempt. Its pubkey is a throwaway key by
* construction -- see [MarmotDirectMessage] -- so there is no author field to
* compare and the check cannot be applied as written. The authorship claim moves
* inside, to the signed kind:13 seal, which [MarmotDirectMessage.classify] binds to
* this same identity before a word is rendered: a verified signature bound to an
* MLS leaf, rather than a plaintext field compared to one.
*
* The exemption is on kind:1059 alone. Widening it to any other kind, or dropping the
* kind guard, hands every member the ability to publish events as anybody.
*
* Other Marmot clients do not have this carve-out and drop these messages as
* impersonation. See docs/marmot-direct-messages.md.
* The exemption is on kind:1059 alone, and on legacy rooms alone. Widening it to any
* other kind, or dropping the kind guard, hands every member the ability to publish
* events as anybody. A current-profile room never carries a bare wrap -- a direct
* message there travels inside an envelope authored by the sender, which passes the
* check like any other payload -- so a bare wrap arriving in one is what every other
* conformant client says it is: an author that is not the sender, refused. Other
* Marmot clients have no carve-out at all and drop legacy wraps as impersonation.
* See docs/marmot-direct-messages.md and docs/marmot-v2.md, Phase 2.
*
* [senderIdentity] is required rather than merely compared: every sender-derived
* field downstream reads from it -- who a message is from, and whether it is ours --
@@ -90,12 +94,14 @@ object MarmotInboundManager {
innerEventKind: Int,
innerEventPubKey: HexKey,
senderIdentity: HexKey?,
legacyRoom: Boolean,
): String? {
if (senderIdentity == null) {
return "No MLS credential identity for the sender leaf"
}
if (innerEventKind != GiftWrapEvent.KIND && innerEventPubKey != senderIdentity) {
val bareWrapInLegacyRoom = legacyRoom && innerEventKind == GiftWrapEvent.KIND
if (!bareWrapInLegacyRoom && innerEventPubKey != senderIdentity) {
return "MIP-03: inner event pubkey ($innerEventPubKey) does not match MLS sender identity ($senderIdentity)"
}
@@ -332,7 +338,12 @@ object MarmotInboundManager {
if (innerAuthor != null) {
val senderIdentity = mlsGroup.memberIdentityHex(decrypted.senderLeafIndex)
mip03Rejection(innerAuthor.first, innerAuthor.second, senderIdentity)?.let { reason ->
mip03Rejection(
innerEventKind = innerAuthor.first,
innerEventPubKey = innerAuthor.second,
senderIdentity = senderIdentity,
legacyRoom = !mlsGroup.currentGroupState().isCurrentProfile,
)?.let { reason ->
return GroupEventResult.Error(groupId, reason)
}
}
@@ -1,5 +1,6 @@
package to.curare.compose.nostr
import com.vitorpamplona.quartz.marmot.foundation.appEvents.MarmotAppEvent
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip01Core.core.HexKey
import com.vitorpamplona.quartz.nip01Core.core.Kind
@@ -20,10 +21,22 @@ import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent
* A one-to-one message carried inside a Marmot group, as the application payload of a
* kind:445 group event. See docs/marmot-direct-messages.md.
*
* The payload is a stock NIP-59 gift wrap: a throwaway-keyed kind:1059 around a
* The message is a stock NIP-59 gift wrap: a throwaway-keyed kind:1059 around a
* sender-signed kind:13 seal around the kind:14 rumor that holds the words. Every member
* of the group decrypts the MLS layer and sees the wrap; only the recipient can open it.
*
* What carries the wrap depends on the room's profile. A legacy room carries the wrap
* itself as the payload -- signed by its throwaway key, with a `pubkey` that names
* nobody -- which is what every build before docs/marmot-v2.md Phase 2 sent and reads,
* and what the carve-out in `MarmotInboundManager.mip03Rejection` exists to admit. A
* current-profile room carries it inside an [envelope]: an unsigned Marmot app event of
* [ENVELOPE_KIND] whose `pubkey` is the sender's account and whose `content` is the
* wrap's JSON. A conformant decoder refuses a payload with a `sig` member and one whose
* author is not the MLS sender, and the wrap fails both; the envelope passes both, and
* keeps every property the wrap was chosen for -- see docs/marmot-direct-messages.md --
* because the envelope is unsigned and so a claim rather than a proof, and the wrap
* inside it is still signed by nobody.
*
* Everything here is a pure function of its arguments, with no database and no MLS state,
* which is what makes it testable — Room-backed code cannot be unit-tested in this project.
*
@@ -33,6 +46,17 @@ import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent
* message lives in `MarmotInnerEvent`, never in `GiftWrapPayload` or `GiftWrapMessage`.
*/
object MarmotDirectMessage {
/**
* The app kind of the envelope a current-profile room carries a direct message in.
*
* In the app's own range, beside the chronicle (30327), its request (30328) and the
* subgroup certificate (30329): a kind that exists only inside MLS and that no relay
* ever sees. Another client renders it as a kind it does not know, which is the
* "sent a private message" line and nothing more -- the same thing a bystander in
* this app sees.
*/
const val ENVELOPE_KIND: Kind = 30330
/**
* A wrap that opened, both layers of it.
*
@@ -176,6 +200,46 @@ object MarmotDirectMessage {
)
}
/**
* The envelope a current-profile room carries [wrap] in.
*
* Unsigned, by construction -- a `MarmotAppEvent` has no signature to give -- and
* authored by [senderPublicKey], which is the MLS sender's account and therefore
* what the standard author check compares against. The recipient rides in a `p` tag
* so a member can tell whose message it is without opening the content, exactly as
* the wrap's own `p` tag lets them today. [createdAt] is the wrap's, unfuzzed, for
* the reason [wrap] gives.
*/
fun envelope(
senderPublicKey: HexKey,
recipientPublicKey: HexKey,
wrap: Event,
createdAt: Long,
): MarmotAppEvent =
MarmotAppEvent.build(
pubKey = senderPublicKey,
kind = ENVELOPE_KIND,
content = wrap.toJson(),
createdAt = createdAt,
tags = arrayOf(arrayOf("p", recipientPublicKey)),
)
/**
* The wrap inside [envelope], or null when it is not an envelope or holds no wrap.
*
* Null rather than a throw for the same reason [open] returns null: the caller is
* inside inbound processing for a kind:445, and one malformed payload must not cost
* the rest.
*/
fun wrapInside(envelope: Event): Event? {
if (envelope.kind != ENVELOPE_KIND) return null
return try {
Event.fromJsonOrNull(envelope.content)?.takeIf { it.kind == GiftWrapEvent.KIND }
} catch (_: Exception) {
null
}
}
/**
* The rumor a direct message carries, unsigned and unencrypted.
*
@@ -113,8 +113,18 @@ class ChronicleEvent(
* Sized so the finished kind:445 clears the tightest relay limits with
* room for the MLS framing and the NIP-44 expansion that sit outside it;
* measure it against a real one rather than trusting the arithmetic.
*
* Under 64 KiB with room to spare, because that is the bound a conformant
* Marmot decoder puts on the *whole* app payload -- `MarmotJson.MAX_INPUT_BYTES`
* -- and this page rides inside one as its `content`, JSON-escaped and
* wrapped in the six members of the envelope. The packer measures the page
* as [escapedByteSize] -- the bytes it will occupy inside the envelope,
* which for dense JSON content is half again the raw page -- against this
* cap, so both the receiver's raw-bytes check and the decoder's bound hold
* by construction. At 64 KiB the raw page alone was over the bound, and an
* MDK peer dropped it without a word. See docs/marmot-v2.md, Phase 2.
*/
const val MAX_PAGE_BYTES = 64 * 1024
const val MAX_PAGE_BYTES = 48 * 1024
/**
* The most payloads one page may carry.
@@ -126,10 +136,10 @@ class ChronicleEvent(
* **Sized under what the byte cap allows, or it would never fire.** An
* event carries a 64-character id, a 64-character pubkey and a
* 128-character signature before it says anything at all, so the floor is
* around 370 bytes and [MAX_PAGE_BYTES] cannot hold much past 170 of
* around 370 bytes and [MAX_PAGE_BYTES] cannot hold much past 130 of
* them. A count cap above that ceiling is a check that cannot fail --
* which is how this was first written, and a test that asserts a page at
* exactly the cap decodes is what caught it. At 128 both caps bind
* exactly the cap decodes is what caught it. At 96 both caps bind
* something: the count stops a page of many small payloads, the bytes
* stop a page of few large ones.
*
@@ -141,7 +151,7 @@ class ChronicleEvent(
* item costs nonce generation, a FROST session and a native sign, where
* a chronicle payload costs a signature verify and an upsert.
*/
const val MAX_PAGE_EVENTS = 128
const val MAX_PAGE_EVENTS = 96
/**
* The kinds a chronicle may carry, in the order they have to be applied.
@@ -221,6 +231,34 @@ class ChronicleEvent(
fun inApplyOrder(payloads: List<Event>): List<Event> =
payloads.sortedBy { applyRank(it.kind) ?: Int.MAX_VALUE }
/**
* The bytes [json] occupies once it is the `content` of a Marmot app event --
* that is, once it is inside a JSON string.
*
* A page is JSON that travels as JSON: every quote in it becomes `\\"`, every
* backslash `\\\\`, and the payloads' own content, already escaped once inside
* the page, escapes again. Dense JSON content grows by half; content that is
* nothing but quotes doubles. The decoder's 64 KiB bound is on the *escaped*
* form, so the packer has to measure this and not the raw page -- see
* [MAX_PAGE_BYTES]. The escape set is NIP-01's, which is what
* `MarmotAppEvent.toJson` writes.
*/
fun escapedByteSize(json: String): Int {
var size = 0
for (ch in json) {
size += when {
ch == '"' || ch == '\\' || ch == '\n' || ch == '\r' || ch == '\t' ||
ch == '\u0008' || ch == '\u000C' -> 2
ch < ' ' -> 6
ch.code < 0x80 -> 1
ch.code < 0x800 -> 2
ch.isSurrogate() -> 2 // each half of a pair is two of the four bytes
else -> 3
}
}
return size
}
/**
* One page's payloads as its content: a JSON array of whole events.
*
@@ -4,48 +4,77 @@ import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent
import com.vitorpamplona.quartz.nip17Dm.messages.ChatMessageEvent
import com.vitorpamplona.quartz.nip59Giftwrap.seals.SealedRumorEvent
import com.vitorpamplona.quartz.nipC7Chats.ChatEvent
import to.curare.compose.nostr.MarmotDirectMessage
import kotlin.test.Test
import kotlin.test.assertNotNull
import kotlin.test.assertNull
import kotlin.test.assertTrue
/**
* The one hole deliberately left in MIP-03's author check, and its edges.
* The one hole deliberately left in the author check, and its edges.
*
* MIP-03 requires an inner application event's pubkey to equal the MLS sender's credential
* identity, which is what stops a member minting events attributed to somebody else. A
* gift wrap cannot satisfy it -- its pubkey is a throwaway key that names nobody -- so
* kind:1059 is exempt, and the authorship claim moves to the signed seal inside.
* An inner application event's pubkey must equal the MLS sender's credential identity,
* which is what stops a member minting events attributed to somebody else. In a legacy
* room a gift wrap cannot satisfy it -- its pubkey is a throwaway key that names nobody
* -- so kind:1059 is exempt there, and the authorship claim moves to the signed seal
* inside. In a current-profile room there is no exemption: a direct message travels
* inside an envelope the sender authors, and a bare wrap is refused like any other
* payload claiming an author it does not own.
*
* The exemption is the most dangerous line in this feature. Widened to another kind, or
* stripped of its kind guard, it hands every member of every group the ability to publish
* events as anybody, and nothing else in the pipeline would notice. These tests exist so
* that widening it fails here rather than in a group.
* The exemption is the most dangerous line in this feature. Widened to another kind,
* stripped of its kind guard, or extended to current-profile rooms, it hands every
* member of every group the ability to publish events as anybody, and nothing else in
* the pipeline would notice. These tests exist so that widening it fails here rather
* than in a group.
*/
class MarmotMip03CarveOutTest {
private val sender = "a".repeat(64)
private val somebodyElse = "b".repeat(64)
private val throwaway = "c".repeat(64)
private fun rejection(kind: Int, pubKey: String, identity: String?, legacyRoom: Boolean = true) =
MarmotInboundManager.mip03Rejection(
innerEventKind = kind,
innerEventPubKey = pubKey,
senderIdentity = identity,
legacyRoom = legacyRoom,
)
@Test
fun `an ordinary message from its own author is accepted`() {
assertNull(MarmotInboundManager.mip03Rejection(ChatEvent.KIND, sender, sender))
fun `an ordinary message from its own author is accepted in either room`() {
assertNull(rejection(ChatEvent.KIND, sender, sender, legacyRoom = true))
assertNull(rejection(ChatEvent.KIND, sender, sender, legacyRoom = false))
}
@Test
fun `an ordinary message claiming somebody else is rejected`() {
val reason = MarmotInboundManager.mip03Rejection(ChatEvent.KIND, somebodyElse, sender)
fun `an ordinary message claiming somebody else is rejected in either room`() {
listOf(true, false).forEach { legacy ->
val reason = rejection(ChatEvent.KIND, somebodyElse, sender, legacyRoom = legacy)
assertNotNull(reason)
assertTrue(reason.startsWith("MIP-03"), "expected the MIP-03 rejection, got: $reason")
assertNotNull(reason)
assertTrue(reason.startsWith("MIP-03"), "expected the MIP-03 rejection, got: $reason")
}
}
@Test
fun `a gift wrap is accepted despite naming nobody`() {
fun `in a legacy room a gift wrap is accepted despite naming nobody`() {
// The carve-out. The wrap's pubkey is a throwaway key and matches no member, which
// is the point of it; what authenticates the sender is the MLS frame, and what
// authenticates the author is the seal inside. See MarmotDirectMessage.classify.
assertNull(MarmotInboundManager.mip03Rejection(GiftWrapEvent.KIND, throwaway, sender))
assertNull(rejection(GiftWrapEvent.KIND, throwaway, sender, legacyRoom = true))
}
@Test
fun `in a current-profile room a bare gift wrap is an author that is not the sender`() {
// No carve-out. A direct message in a current-profile room is an envelope by the
// sender; a bare wrap there is what every other conformant client refuses it as.
assertNotNull(rejection(GiftWrapEvent.KIND, throwaway, sender, legacyRoom = false))
}
@Test
fun `an envelope passes the ordinary check because its author is the sender`() {
assertNull(rejection(MarmotDirectMessage.ENVELOPE_KIND, sender, sender, legacyRoom = false))
assertNotNull(rejection(MarmotDirectMessage.ENVELOPE_KIND, somebodyElse, sender, legacyRoom = false))
}
@Test
@@ -57,6 +86,7 @@ class MarmotMip03CarveOutTest {
ChatEvent.KIND,
ChatMessageEvent.KIND,
SealedRumorEvent.KIND,
MarmotDirectMessage.ENVELOPE_KIND,
GiftWrapEvent.KIND - 1,
GiftWrapEvent.KIND + 1,
0,
@@ -65,7 +95,7 @@ class MarmotMip03CarveOutTest {
kinds.forEach { kind ->
assertNotNull(
MarmotInboundManager.mip03Rejection(kind, somebodyElse, sender),
rejection(kind, somebodyElse, sender, legacyRoom = true),
"kind $kind was allowed to claim an author it does not own",
)
}
@@ -76,7 +106,7 @@ class MarmotMip03CarveOutTest {
// Required rather than merely compared: with no identity there is nothing to
// attribute a line to, and for a gift wrap there is nothing to check the seal
// against either. Both kinds must fail, not just the one that does a comparison.
assertNotNull(MarmotInboundManager.mip03Rejection(ChatEvent.KIND, sender, null))
assertNotNull(MarmotInboundManager.mip03Rejection(GiftWrapEvent.KIND, throwaway, null))
assertNotNull(rejection(ChatEvent.KIND, sender, null))
assertNotNull(rejection(GiftWrapEvent.KIND, throwaway, null))
}
}
@@ -1,5 +1,6 @@
package to.curare.compose.nostr
import com.vitorpamplona.quartz.marmot.foundation.appEvents.MarmotAppEvent
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip01Core.core.firstTagValue
import com.vitorpamplona.quartz.nip01Core.core.toHexKey
@@ -182,4 +183,55 @@ class MarmotDirectMessageTest {
assertNull(MarmotDirectMessage.open(notAWrap, bob))
}
// ---- The envelope, for current-profile rooms -----------------------------
@Test
fun `the envelope is unsigned, by the sender, and carries the wrap whole`() {
val wrap = aliceWrapsForBob()
val envelope = MarmotDirectMessage.envelope(alice.pubKey, bobPublicKey, wrap, at)
// Six members and no signature: what a conformant decoder accepts, and what a
// leak cannot republish as a signed statement by anybody.
val json = envelope.toJson()
assertTrue(!json.contains("\"sig\""), "the envelope carries a signature")
assertEquals(alice.pubKey, MarmotAppEvent.decode(json).pubKey)
assertEquals(MarmotDirectMessage.ENVELOPE_KIND, MarmotAppEvent.decode(json).kind)
assertEquals(bobPublicKey, envelope.tags.firstTagValue("p"))
// The wrap inside is the wrap, untouched, and Bob opens it as before.
val inside = assertNotNull(MarmotDirectMessage.wrapInside(Event.fromJsonOrNull(json)!!))
assertEquals(wrap.id, inside.id)
assertEquals(text, assertNotNull(MarmotDirectMessage.open(inside, bob)).rumor.content)
}
@Test
fun `the envelope's createdAt is the wrap's, unfuzzed`() {
val envelope = MarmotDirectMessage.envelope(alice.pubKey, bobPublicKey, aliceWrapsForBob(), at)
assertEquals(at, envelope.createdAt)
}
@Test
fun `something that is not an envelope holds no wrap`() {
val wrap = aliceWrapsForBob()
val wrongKind = Event(
id = "1".repeat(64), pubKey = alice.pubKey, createdAt = at,
kind = MarmotDirectMessage.ENVELOPE_KIND + 1, tags = emptyArray(), content = wrap.toJson(), sig = "",
)
val notAWrapInside = Event(
id = "2".repeat(64), pubKey = alice.pubKey, createdAt = at,
kind = MarmotDirectMessage.ENVELOPE_KIND, tags = emptyArray(), content = "not json", sig = "",
)
val sealInside = Event(
id = "3".repeat(64), pubKey = alice.pubKey, createdAt = at,
kind = MarmotDirectMessage.ENVELOPE_KIND, tags = emptyArray(),
content = alice.signNormal<Event>(createdAt = at, kind = SealedRumorEvent.KIND, tags = emptyArray(), content = "").toJson(),
sig = "",
)
assertNull(MarmotDirectMessage.wrapInside(wrongKind))
assertNull(MarmotDirectMessage.wrapInside(notAWrapInside))
assertNull(MarmotDirectMessage.wrapInside(sealInside))
}
}
@@ -0,0 +1,131 @@
package to.curare.compose.nostr
import com.vitorpamplona.quartz.marmot.foundation.appEvents.MarmotAppEvent
import com.vitorpamplona.quartz.marmot.foundation.appEvents.MarmotJson
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip01Core.core.toHexKey
import com.vitorpamplona.quartz.nip01Core.signers.EventTemplate
import com.vitorpamplona.quartz.nip01Core.crypto.KeyPair
import com.vitorpamplona.quartz.nip01Core.signers.NostrSignerSync
import com.vitorpamplona.quartz.nip01Core.tags.people.PTag
import com.vitorpamplona.quartz.nip17Dm.messages.ChatMessageEvent
import com.vitorpamplona.quartz.nip59Giftwrap.rumors.RumorAssembler
import to.curare.compose.nostr.chronicle.ChronicleEvent
import to.curare.compose.nostr.nip30303.ChunkEvent
import to.curare.compose.nostr.nip30303.DialectEvent
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertNotNull
import kotlin.test.assertTrue
/**
* The app's largest payloads fit inside what a conformant Marmot decoder will read.
*
* `MarmotJson` bounds an app payload at 64 KiB, sixteen levels of nesting and 64
* elements per collection, and refuses anything over before a parser sees it. The
* app's own rooms would still read an oversized page through the legacy fallback; an
* MDK peer drops it without a word. So the caps that decide the app's biggest
* payloads are asserted here against the decoder itself, at their maximum, so that
* raising one of them past the bound fails in a test rather than in a group.
*/
class MarmotPayloadBoundsTest {
private val author = "a".repeat(64)
private val recipient = "d".repeat(64)
private val at = 1_700_000_000L
private fun signedPayload(kind: Int, content: String, id: String) = Event(
id = id,
pubKey = "b".repeat(64),
createdAt = at,
kind = kind,
tags = arrayOf(arrayOf("alt", "test"), arrayOf("d", "x".repeat(64))),
content = content,
sig = "c".repeat(128),
)
/** The app event the outbound path builds for a chronicle page: the rumor, canonical. */
private fun chronicleAppEvent(payloads: List<Event>): MarmotAppEvent {
val template = ChronicleEvent.build(
payloads = payloads,
chronicleId = "e".repeat(64),
index = 0,
count = 1,
recipient = recipient,
createdAt = at,
)
// From the template's fields, as the outbound queue assembles every rumor from
// the row's: the typed template is not what goes through RumorAssembler.
val rumor = RumorAssembler.assembleRumor<Event>(
pubKey = author,
ev = EventTemplate(createdAt = template.createdAt, kind = template.kind, tags = template.tags, content = template.content),
)
return MarmotAppEvent.fromEvent(rumor)
}
private fun assertConformant(appEvent: MarmotAppEvent) {
val json = appEvent.toJson()
assertTrue(
MarmotJson.withinResourceBounds(json),
"${json.encodeToByteArray().size} bytes is outside the decoder's bounds",
)
assertEquals(appEvent.id, MarmotAppEvent.decode(json).id)
}
@Test
fun `a chronicle page of the most, smallest payloads is a conformant app event`() {
val payloads = (0 until ChronicleEvent.MAX_PAGE_EVENTS).map {
signedPayload(DialectEvent.KIND, content = "d", id = it.toString().padStart(64, '0'))
}
assertConformant(chronicleAppEvent(payloads))
}
/**
* The worst case for the byte bound: content that is itself dense JSON, so every
* quote in the page doubles when the page becomes the app event's `content`.
*/
@Test
fun `a chronicle page at the byte cap is a conformant app event after escaping`() {
val dense = buildString {
while (length < 4_096) append("{\"k\":\"v\",\"n\":[\"a\",\"b\"]}")
}
val payloads = (0 until ChronicleEvent.MAX_PAGE_EVENTS).map {
signedPayload(ChunkEvent.KIND, content = dense, id = it.toString().padStart(64, '0'))
}
// As many dense payloads as the byte cap admits, measured the way the packer
// measures them: by the bytes the page occupies once escaped into the app
// event's content. Measured raw, this page is under the cap and the app
// event is over the decoder's bound -- which is the bug the measure fixes.
val page = (payloads.size downTo 1).firstNotNullOf { n ->
payloads.take(n).takeIf {
ChronicleEvent.escapedByteSize(ChronicleEvent.encodePage(it)) <= ChronicleEvent.MAX_PAGE_BYTES
}
}
assertTrue(page.size < payloads.size, "the byte cap never bound; the page is not at its maximum")
assertTrue(
ChronicleEvent.encodePage(page + payloads[page.size]).encodeToByteArray().size <= ChronicleEvent.MAX_PAGE_BYTES,
"measured raw, one more payload would still have fit -- the escaped measure is not the binding one here",
)
assertConformant(chronicleAppEvent(page))
}
@Test
fun `a direct message envelope is a conformant app event`() {
val sender = NostrSignerSync(KeyPair())
val bob = KeyPair()
val wrap = MarmotDirectMessage.wrap(
signer = sender,
recipientPublicKey = bob.pubKey.toHexKey(),
kind = ChatMessageEvent.KIND,
createdAt = at,
tags = arrayOf(PTag.assemble(bob.pubKey.toHexKey(), null)),
content = "x".repeat(8_192),
)
val envelope = MarmotDirectMessage.envelope(sender.pubKey, bob.pubKey.toHexKey(), wrap, at)
assertConformant(envelope)
val inside = assertNotNull(MarmotDirectMessage.wrapInside(Event.fromJsonOrNull(envelope.toJson())!!))
assertEquals(wrap.id, inside.id)
}
}
@@ -24,6 +24,7 @@ import kotlin.test.assertEquals
import kotlin.test.assertNotNull
import kotlin.test.assertNull
import kotlin.test.assertTrue
import kotlin.io.encoding.Base64
/**
* A peer's kind:30443 through the sync funnel and into an invite, on both profiles.
@@ -155,9 +156,12 @@ class MarmotKeyPackageIngestJvmTest {
@Test
fun `a key package that does not verify is not stored`() = runBlocking {
val (event, _) = currentProfileKeyPackageEventFor(peer)
val tampered = event.asStoredNostrEvent().copy(
content = event.content.dropLast(4) + "AAAA",
)
// A byte of the KeyPackage's own signature -- its last field -- flipped after
// signing. Flipped in the decoded bytes rather than the base64 text, because
// rewriting the text's tail can change nothing but its padding.
val bytes = Base64.decode(event.content)
bytes[bytes.lastIndex] = (bytes[bytes.lastIndex].toInt() xor 0x01).toByte()
val tampered = event.asStoredNostrEvent().copy(content = Base64.encode(bytes))
store(tampered)
+22
View File
@@ -102,6 +102,28 @@ the first was asserted the wrong way round until one of them failed.
> divergence from the spec, and a reason to raise it upstream rather than carry it
> indefinitely.
### The envelope, in current-profile rooms
The carve-out is now confined to legacy rooms, and the upstream answer turned out
to be a shape rather than a rule change. A current-profile room ([marmot-v2.md](./marmot-v2.md),
Phase 2) carries the wrap *inside* an unsigned Marmot app event of
`MarmotDirectMessage.ENVELOPE_KIND` (30330), whose `pubkey` is the sender's account,
whose one tag is `["p", recipient]` and whose `content` is the wrap's JSON. The
envelope passes the author check like any other payload, so there is nothing to
exempt, and `mip03Rejection` takes a `legacyRoom` flag: in a current-profile room a
bare kind:1059 is refused as what every other conformant client refuses it as.
The properties above survive intact. The envelope is unsigned — a `MarmotAppEvent`
has no signature to give — so a lifted envelope is a *claim* of authorship and not a
proof; the wrap inside is still signed by nobody; the seal-to-leaf binding in
`classify` still decides who wrote the words. What a bystander sees is unchanged:
that a private message was sent, and to whom.
An old build renders the envelope as a raw-JSON bubble, which is why a legacy room
keeps sending the bare wrap: a current-profile room admits only leaves carrying an
identity proof, and only a build past Phase 3 can make one, so nobody in one runs
an old build. The room's profile is the flag.
### The sender cannot read their own messages back
The throwaway private key is discarded at send time, so nothing can reopen the wrap
+7
View File
@@ -272,6 +272,13 @@ a chronicle is the second place in this protocol where a remote party decides ho
much work everyone else does. Measure the 64 KB against a finished kind:445
rather than trusting it -- MLS framing and NIP-44 expansion both sit outside it.
**Re-sized since: 48 KiB and 96.** A conformant Marmot decoder bounds the whole
app payload at 64 KiB, and a page rides inside one as its JSON-escaped `content`
plus six envelope members, so a 64 KiB page was over that bound by construction
and an MDK peer dropped it without a word. 48 KiB leaves room for the escaping
whatever the mix of payloads, and 96 is the count that still binds under it.
See [marmot-v2.md](./marmot-v2.md), Phase 2.
**Test:** codec round-trip; a page over either cap is refused on receive; an
array containing a non-event is refused whole.