mantra-kmp

mantra/mantra-kmp

Fork 0

Commit Graph

Author	SHA1	Message	Date
Kgothatso Ngako	acff66a22e	feat: rebuild the group's signed record out of the rows it left behind Phase 3 of docs/member-archive.md. `ArchiveManager.assemble` walks a room's rows, rebuilds each into the event the group signed, drops anything it cannot prove, and cuts the rest into pages. Nothing sends one yet. The gate found a real bug, which is why it was the gate. Signed events are not stored as events -- `FrostSigningManager.complete` applies one and what survives is a `Mantra` row -- so an archive has to rebuild them with `toXEvent()` and stands or falls on that being byte-identical to what was signed. Every `toXEvent()` in the codebase turned out to be unused in production, written for exactly this and never called, so the "tag order matches build so the event id round-trips" comments on them were claims nothing had ever checked. One was wrong. `MantraArtifact.toArtifactEvent` put the alt tag last where `ArtifactEvent.build` puts it first, and left out the version metadata tag altogether -- because that tag is not on the artifact row at all. `fromArtifactEvent` reads the artifact's own fields and drops the version label, which `applyInnerEvent` has by then turned into the artifact's first `MantraArtifactVersion`. So the label is now a parameter, read off the initial version: the one whose `createdAt` is the artifact's, since `initialVersionOf` derives it from the same event. Neither fault would have surfaced as an error. Both produce a well-formed artifact whose id no longer matches its fields, which every receiver drops as a forgery, silently, one kind at a time. `ArchiveRoundTripTest` now signs each archivable kind with a real quorum, files it as a row, rebuilds it and asserts the signature still covers what comes out -- plus the negative case, that rebuilding with the wrong version label fails as a forgery rather than as a mistake, which is why the assembler reads the label rather than defaulting it. The allowlist narrows from nine kinds to six, and this is the finding to read.* Only six of the thirteen nip30303 kinds ever reach a signing session; the rest travel as member rumors, vouched for by the MLS frame they arrived in and by nothing that survives leaving it. An artifact version is derived rather than signed -- which is fine, because applying the archived artifact derives it again and the chapters hanging off it keep their foreign key. Nothing builds a `TranslationEvent` at all. The contributor lists have no arm in `applyInnerEvent` that writes a row. And `TranslationChunkEvent` -- the translated text itself -- is submitted by `MantraDao.saveTranslation` as its author's rumor, because a translation is one member's work rather than a group decision. So an archive restores everything a translation hangs on and not the translation: a new member gets the dialects, the artifacts, the chapters, the source chunks, which translations exist and their chapter scaffolding, and none of the prose. That is a real limit rather than a detail, so it is written into the allowlist's own doc comment, into the plan's "what this does not do", and into a test named after it -- with the three ways out sketched and none of them taken here, because the cheapest gives up the property the rest of this rests on and the best is a product decision about whether translating is an act of the group or of a member. Nothing unverifiable leaves. Every rebuilt event is checked with `isSignedByRoom` against the same room id the recipient will use. Not politeness -- the receiver checks anyway -- but so the page count says what will actually arrive: a row from a member's rumor is dropped here rather than by the recipient. Walked down the tree, not queried per kind. Only dialects and artifacts have a by-room query and the rest hang off a parent, and the walk is also what puts an artifact's version label within reach. Order is settled afterwards by `inApplyOrder` rather than by the walk, since the walk groups by artifact and the foreign keys are by kind. Paging is greedy against both caps, because they bind different archives: a room of one-line dialects hits the count first and a room of chapters hits the bytes. An event too large for a page of its own is dropped with a log rather than failing the archive -- a chapter nobody can archive is a hole, a member who gets nothing is a bigger one. Assembling only; queueing moved to Phase 5, where the thing that decides when to send lives. That keeps this testable against a real database with no outbound path in the way. Seven tests over a real in-memory database seeded through `applyInnerEvent` itself, so what is archived is what a member's device really holds rather than rows built to suit the test: every payload verifies, all six kinds appear exactly as often as they were signed, the whole archive is in dependency order end to end, a member's unsigned dialect sitting in the same room is left out, an empty room archives nothing without failing, and two archives of identical rows do not share an id -- which is what stops two members answering one request from having their pages counted towards each other's total. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-06 14:12:29 +02:00
Kgothatso Ngako	6e04f6c7af	feat: give the group's signed record an envelope it can travel in Phase 2 of docs/member-archive.md. Two kinds, three tags, a codec and two caps. Nothing sends or applies one yet -- that is phases 3 and 4 -- so this changes no behaviour at all. `ArchiveEvent` (30327) carries a page of the group's signed events, each whole, keeping its own id, author and signature so the receiver checks it rather than believing it. `ArchiveRequestEvent` (30328) is how a device with none of it asks. Why not one `SubmissionEvent` per event. The envelope fits and the meaning does not. A submission is an act -- this member is putting this event in front of this group -- and an archive asserts nothing; it re-delivers what the group already agreed. On one kind a four-hundred-event backfill is indistinguishable from four hundred new submissions and every device has to guess which it is reading. It would also be one inner event and one kind:445 per payload where a page is one, and the submission arm of `applyInnerEvent` files a chat line per payload, which an archive must not. Why 3032x and not 30313. 30313 is free beside the nip30303 document kinds and is not used, on `FrostSigningEvents`' own advice: the DKG's 30310-30316 already overlap that range and are told apart only by living in NIP-17 gift wraps instead, which it calls "an accident of routing rather than a decision, and the next family added should not rely on it." This is that next family. 30327 is also the right neighbourhood on the merits, next to `GroupKeyStateEvent` at 30326 -- an archive is a statement about the record rather than a document kind. One list is the apply order and the allowlist both, because a separate allowlist is one more thing that can disagree with the order it is applied in. The order is Room's rather than nostr's: every archivable kind has a foreign key on the one before it, and kind order is not dependency order -- a dialect (30304) has to land before an artifact (30300), and a translation chapter (30308) hangs off a translation artifact version (30306) which hangs off an artifact version (30301). So it is a list, not a `sortedBy { kind }`, and there is a test that fails if anybody makes it one. It is an allowlist first. Verification admits an event to the apply path on the strength of the group's signature, which makes every kind the group has ever signed replayable by any member at any time. A `GroupKeyStateEvent` is group-signed and passes verification perfectly, so an archive carrying an old one is a validly signed statement about what the room signs with, replayed by whoever kept a copy. Nothing but this list stops it. The contributor-list kinds (30305, 30307, 30310) are left out on the same principle from the other side: `applyInnerEvent` has no arm that writes a row for any of them, so archiving them would cost bytes and restore nothing. All-or-nothing parsing, per-payload verification. These are not in tension; they answer different questions. A page that will not parse has lost its framing, and one silently shortened by an element would report a complete archive on its page count while holding less than it says. A payload whose signature does not verify is a well-framed page with one bad event in it, and costing its honest neighbours would let a single forgery deny an entire archive. The count cap was 256 and 256 can never fire. An event carries 64 characters of id, 64 of pubkey and 128 of signature before it says anything, so the floor is about 370 bytes and a 64 KB page cannot hold much past 170 of them -- the byte cap always binds first and the count cap is a check that never runs. Found by writing the test that a page at exactly the cap still decodes, which failed. Now 128, where both bind something: the count stops a page of many small payloads, the bytes stop a page of few large ones. That test is what fails if somebody later raises one number without the other, and the doc comment says they have to move together. The `p` tag is a hint, not access control, and `ArchiveRecipientTag` says so where it is defined. The page is an ordinary group message and every member can read it, which is right, because it is their own history going back to them. What it decides is who acts: a device that is not named applies nothing, since it already holds the work and re-applying would rewrite every one of its rows to point at an archive page rather than at the event that introduced it. `ArchivePageTag` refuses an index outside its own count rather than clamping it. The pair is how a receiver decides it has everything, so a repaired one would let a truncated archive read as complete. Twenty-one tests over the codec, both caps, the allowlist, the order and the tags. Also corrects the phase-2 section of the plan, which still said 256. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-06 14:00:30 +02:00
Kgothatso Ngako	3ee1676a04	docs: plan handing a new member the group's signed history A member added after the work was done sees none of it, and nothing in the app will ever show it to them. Two independent reasons, and the second is the one that surprises people. MLS gives no history: a Welcome carries the ratchet tree at the current epoch, not the transcript, and `MarmotInboundManager` drops anything from an epoch it holds no keys for. That is forward secrecy working rather than a gap to close. But group-signed events never travel at all. `FrostSigningManager.complete` says so in as many words -- a signed event authored by the threshold key cannot go out as an inner event, because the outbound pipeline would re-author it as its sender and strip the group's signature off -- so every device derives the finished event from its own `FrostSigningItem` rows. A member who was not in the session has no items, and no later message carries the event. So the second problem does not follow from the first and is not fixed by fixing it: even a member who could decrypt the whole back-transcript would still hold nothing an artifact, chapter or chunk could be built from. Which makes an archive not a convenience but the only path, and fixes the line the design has to hold: it carries what the group signed, never the chat. Restoring the chat would undo forward secrecy on purpose, and a signed event is the only thing a new member can check for themselves. The property the whole plan rests on is already true. A room's id is the group's threshold key derived at the room's path -- `GroupKeyState.verifies` and `FrostSigningManager.signingPath` hold that invariant from their own ends -- so `isSignedByGroup`'s three checks collapse to `event.pubKey == chatRoomId`, an id check and a signature verify. No key state row, no threshold key, no path, no lookup. A member who can name the room can verify its signatures, which is exactly the position a new member is in, and it means the sender of an archive does not have to be trusted at all. Two guards the plan makes non-negotiable. Nothing on the inbound nip30303 path verifies a signature today, and that is currently correct: rumors carry an empty sig and are authenticated by the MLS frame, so nothing on the wire has ever claimed group authorship. An archive is the first thing that does, so the verify is the feature's entire security rather than hardening on top of it. And verification turns "group-signed" into an admission ticket for the apply path, which is a wider door than it looks: a `GroupKeyStateEvent` is group-signed and would pass perfectly, so an archive could replay a genuine old one and re-point what the room signs with. The archive therefore carries an allowlist of document kinds, checked outbound and independently inbound -- the same shape, and the same reasoning, as the cap on `k` in frost-batch-signing.md. Push and pull, in that order of appearance and the reverse order of importance. Pushing an archive after the Welcome is what the question asked for, and on its own it fails the way marmot-membership.md describes: it is an application message in the epoch the add created, so one that beats the Welcome there is dropped rather than deferred, silently, while the inviter sees a success. So the joiner asks instead -- a request is proof it has processed its Welcome, and it covers the reinstall and the second device, which no invite-time push can. The push stays as a latency optimisation, deliberately phased after the thing that makes it safe. Nine phases: the verifier, the events, assembling an archive, applying one and the sweep that lets pages arrive out of order, the request, the push, UI, the cross-device tests, and rollout. The sweep needs no new table -- the inbound path already stores every inner event it decrypts, so it is the shape `FrostSigningManager.replayStoredMessages` already has. Also written down, because it is the first thing this will be reported as a bug for: an archive lets a new member read everything and does not let them sign anything. `proposeSigningBatch` wants a secret share and a place in the ceremony, and a group that re-runs its ceremony derives a different room rather than re-keying this one. Closing that needs share resharing, which is a great deal more work than this and is the thing to build after it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-06 13:52:33 +02:00

Author

SHA1

Message

Date

Kgothatso Ngako

acff66a22e

feat: rebuild the group's signed record out of the rows it left behind

Phase 3 of docs/member-archive.md. `ArchiveManager.assemble` walks a room's rows,
rebuilds each into the event the group signed, drops anything it cannot prove,
and cuts the rest into pages. Nothing sends one yet.

**The gate found a real bug, which is why it was the gate.** Signed events are
not stored as events -- `FrostSigningManager.complete` applies one and what
survives is a `Mantra*` row -- so an archive has to rebuild them with `toXEvent()`
and stands or falls on that being byte-identical to what was signed. Every
`toXEvent()` in the codebase turned out to be unused in production, written for
exactly this and never called, so the "tag order matches build so the event id
round-trips" comments on them were claims nothing had ever checked.

One was wrong. `MantraArtifact.toArtifactEvent` put the alt tag last where
`ArtifactEvent.build` puts it first, and left out the version metadata tag
altogether -- because that tag is not on the artifact row at all.
`fromArtifactEvent` reads the artifact's own fields and drops the version label,
which `applyInnerEvent` has by then turned into the artifact's first
`MantraArtifactVersion`. So the label is now a parameter, read off the initial
version: the one whose `createdAt` is the artifact's, since `initialVersionOf`
derives it from the same event.

Neither fault would have surfaced as an error. Both produce a well-formed
artifact whose id no longer matches its fields, which every receiver drops as a
forgery, silently, one kind at a time. `ArchiveRoundTripTest` now signs each
archivable kind with a real quorum, files it as a row, rebuilds it and asserts
the signature still covers what comes out -- plus the negative case, that
rebuilding with the wrong version label fails as a forgery rather than as a
mistake, which is why the assembler reads the label rather than defaulting it.

**The allowlist narrows from nine kinds to six, and this is the finding to read.**
Only six of the thirteen nip30303 kinds ever reach a signing session; the rest
travel as member rumors, vouched for by the MLS frame they arrived in and by
nothing that survives leaving it. An artifact version is derived rather than
signed -- which is fine, because applying the archived artifact derives it again
and the chapters hanging off it keep their foreign key. Nothing builds a
`TranslationEvent` at all. The contributor lists have no arm in `applyInnerEvent`
that writes a row.

And `TranslationChunkEvent` -- **the translated text itself** -- is submitted by
`MantraDao.saveTranslation` as its author's rumor, because a translation is one
member's work rather than a group decision. So an archive restores everything a
translation hangs on and not the translation: a new member gets the dialects, the
artifacts, the chapters, the source chunks, which translations exist and their
chapter scaffolding, and none of the prose. That is a real limit rather than a
detail, so it is written into the allowlist's own doc comment, into the plan's
"what this does not do", and into a test named after it -- with the three ways
out sketched and none of them taken here, because the cheapest gives up the
property the rest of this rests on and the best is a product decision about
whether translating is an act of the group or of a member.

**Nothing unverifiable leaves.** Every rebuilt event is checked with
`isSignedByRoom` against the same room id the recipient will use. Not politeness
-- the receiver checks anyway -- but so the page count says what will actually
arrive: a row from a member's rumor is dropped here rather than by the recipient.

**Walked down the tree, not queried per kind.** Only dialects and artifacts have
a by-room query and the rest hang off a parent, and the walk is also what puts an
artifact's version label within reach. Order is settled afterwards by
`inApplyOrder` rather than by the walk, since the walk groups by artifact and the
foreign keys are by kind.

**Paging is greedy against both caps**, because they bind different archives: a
room of one-line dialects hits the count first and a room of chapters hits the
bytes. An event too large for a page of its own is dropped with a log rather than
failing the archive -- a chapter nobody can archive is a hole, a member who gets
nothing is a bigger one.

Assembling only; queueing moved to Phase 5, where the thing that decides when to
send lives. That keeps this testable against a real database with no outbound
path in the way.

Seven tests over a real in-memory database seeded through `applyInnerEvent`
itself, so what is archived is what a member's device really holds rather than
rows built to suit the test: every payload verifies, all six kinds appear exactly
as often as they were signed, the whole archive is in dependency order end to
end, a member's unsigned dialect sitting in the same room is left out, an empty
room archives nothing without failing, and two archives of identical rows do not
share an id -- which is what stops two members answering one request from having
their pages counted towards each other's total.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-09-06 14:12:29 +02:00

Kgothatso Ngako

6e04f6c7af

feat: give the group's signed record an envelope it can travel in

Phase 2 of docs/member-archive.md. Two kinds, three tags, a codec and two caps.
Nothing sends or applies one yet -- that is phases 3 and 4 -- so this changes no
behaviour at all.

`ArchiveEvent` (30327) carries a page of the group's signed events, each whole,
keeping its own id, author and signature so the receiver checks it rather than
believing it. `ArchiveRequestEvent` (30328) is how a device with none of it asks.

**Why not one `SubmissionEvent` per event.** The envelope fits and the meaning
does not. A submission is an *act* -- this member is putting this event in front
of this group -- and an archive asserts nothing; it re-delivers what the group
already agreed. On one kind a four-hundred-event backfill is indistinguishable
from four hundred new submissions and every device has to guess which it is
reading. It would also be one inner event and one kind:445 per payload where a
page is one, and the submission arm of `applyInnerEvent` files a chat line per
payload, which an archive must not.

**Why 3032x and not 30313.** 30313 is free beside the nip30303 document kinds
and is not used, on `FrostSigningEvents`' own advice: the DKG's 30310-30316
already overlap that range and are told apart only by living in NIP-17 gift wraps
instead, which it calls "an accident of routing rather than a decision, and the
next family added should not rely on it." This is that next family. 30327 is also
the right neighbourhood on the merits, next to `GroupKeyStateEvent` at 30326 --
an archive is a statement about the record rather than a document kind.

**One list is the apply order and the allowlist both**, because a separate
allowlist is one more thing that can disagree with the order it is applied in.

The order is Room's rather than nostr's: every archivable kind has a foreign key
on the one before it, and kind order is not dependency order -- a dialect (30304)
has to land before an artifact (30300), and a translation chapter (30308) hangs
off a translation artifact version (30306) which hangs off an artifact version
(30301). So it is a list, not a `sortedBy { kind }`, and there is a test that
fails if anybody makes it one.

It is an allowlist first. Verification admits an event to the apply path on the
strength of the group's signature, which makes every kind the group has ever
signed replayable by any member at any time. A `GroupKeyStateEvent` is
group-signed and passes verification perfectly, so an archive carrying an old one
is a validly signed statement about what the room signs with, replayed by whoever
kept a copy. Nothing but this list stops it. The contributor-list kinds (30305,
30307, 30310) are left out on the same principle from the other side:
`applyInnerEvent` has no arm that writes a row for any of them, so archiving them
would cost bytes and restore nothing.

**All-or-nothing parsing, per-payload verification.** These are not in tension;
they answer different questions. A page that will not parse has lost its framing,
and one silently shortened by an element would report a complete archive on its
page count while holding less than it says. A payload whose signature does not
verify is a well-framed page with one bad event in it, and costing its honest
neighbours would let a single forgery deny an entire archive.

**The count cap was 256 and 256 can never fire.** An event carries 64 characters
of id, 64 of pubkey and 128 of signature before it says anything, so the floor is
about 370 bytes and a 64 KB page cannot hold much past 170 of them -- the byte
cap always binds first and the count cap is a check that never runs. Found by
writing the test that a page at exactly the cap still decodes, which failed. Now
128, where both bind something: the count stops a page of many small payloads,
the bytes stop a page of few large ones. That test is what fails if somebody
later raises one number without the other, and the doc comment says they have to
move together.

**The `p` tag is a hint, not access control**, and `ArchiveRecipientTag` says so
where it is defined. The page is an ordinary group message and every member can
read it, which is right, because it is their own history going back to them. What
it decides is who *acts*: a device that is not named applies nothing, since it
already holds the work and re-applying would rewrite every one of its rows to
point at an archive page rather than at the event that introduced it.

`ArchivePageTag` refuses an index outside its own count rather than clamping it.
The pair is how a receiver decides it has everything, so a repaired one would let
a truncated archive read as complete.

Twenty-one tests over the codec, both caps, the allowlist, the order and the
tags. Also corrects the phase-2 section of the plan, which still said 256.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-09-06 14:00:30 +02:00

Kgothatso Ngako

3ee1676a04

docs: plan handing a new member the group's signed history

A member added after the work was done sees none of it, and nothing in the app
will ever show it to them. Two independent reasons, and the second is the one
that surprises people.

MLS gives no history: a Welcome carries the ratchet tree at the current epoch,
not the transcript, and `MarmotInboundManager` drops anything from an epoch it
holds no keys for. That is forward secrecy working rather than a gap to close.

But group-signed events never travel at all. `FrostSigningManager.complete` says
so in as many words -- a signed event authored by the threshold key cannot go out
as an inner event, because the outbound pipeline would re-author it as its sender
and strip the group's signature off -- so every device *derives* the finished
event from its own `FrostSigningItem` rows. A member who was not in the session
has no items, and no later message carries the event. So the second problem does
not follow from the first and is not fixed by fixing it: even a member who could
decrypt the whole back-transcript would still hold nothing an artifact, chapter
or chunk could be built from.

Which makes an archive not a convenience but the only path, and fixes the line
the design has to hold: **it carries what the group signed, never the chat.**
Restoring the chat would undo forward secrecy on purpose, and a signed event is
the only thing a new member can check for themselves.

**The property the whole plan rests on is already true.** A room's id *is* the
group's threshold key derived at the room's path -- `GroupKeyState.verifies` and
`FrostSigningManager.signingPath` hold that invariant from their own ends -- so
`isSignedByGroup`'s three checks collapse to `event.pubKey == chatRoomId`, an id
check and a signature verify. No key state row, no threshold key, no path, no
lookup. A member who can name the room can verify its signatures, which is
exactly the position a new member is in, and it means the sender of an archive
does not have to be trusted at all.

**Two guards the plan makes non-negotiable.** Nothing on the inbound nip30303
path verifies a signature today, and that is currently correct: rumors carry an
empty sig and are authenticated by the MLS frame, so nothing on the wire has ever
claimed group authorship. An archive is the first thing that does, so the verify
is the feature's entire security rather than hardening on top of it.

And verification turns "group-signed" into an admission ticket for the apply
path, which is a wider door than it looks: a `GroupKeyStateEvent` is group-signed
and would pass perfectly, so an archive could replay a genuine old one and
re-point what the room signs with. The archive therefore carries an allowlist of
document kinds, checked outbound and independently inbound -- the same shape, and
the same reasoning, as the cap on `k` in frost-batch-signing.md.

**Push and pull, in that order of appearance and the reverse order of
importance.** Pushing an archive after the Welcome is what the question asked
for, and on its own it fails the way marmot-membership.md describes: it is an
application message in the epoch the add created, so one that beats the Welcome
there is dropped rather than deferred, silently, while the inviter sees a
success. So the joiner asks instead -- a request is proof it has processed its
Welcome, and it covers the reinstall and the second device, which no invite-time
push can. The push stays as a latency optimisation, deliberately phased after the
thing that makes it safe.

Nine phases: the verifier, the events, assembling an archive, applying one and
the sweep that lets pages arrive out of order, the request, the push, UI, the
cross-device tests, and rollout. The sweep needs no new table -- the inbound path
already stores every inner event it decrypts, so it is the shape
`FrostSigningManager.replayStoredMessages` already has.

Also written down, because it is the first thing this will be reported as a bug
for: an archive lets a new member *read* everything and does not let them sign
anything. `proposeSigningBatch` wants a secret share and a place in the ceremony,
and a group that re-runs its ceremony derives a different room rather than
re-keying this one. Closing that needs share resharing, which is a great deal
more work than this and is the thing to build after it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-09-06 13:52:33 +02:00

3 Commits