docs(marmot): the v2 plan is built -- where the build chose differently, and where the rollout stands (v2 Phase 9)
docs/marmot-v2.md, Phase 9. The phase has no code in it; it is the rollout, and the rollout is a flip and a retirement that wait on the people this app's users invite. What the phase owes the document is the repo's convention for a plan that has been built: the status paragraph, the table of where the implementation chose differently, the thing found on the way that belongs to nobody's phase, and the commit the account was written against. All four are here now, and the phases themselves are kept as written, because they are the reasoning and the code reads better against the argument it came from than against a summary of itself. The table has twenty-two rows, and the first is the one to read before any of the code: the plan said the library's engine replaces the app's pipeline, and the build made a room's profile pick its pipeline instead -- legacy rooms keep the app's path, comparator and cache included, and retire with the profile; current-profile rooms go through `MarmotEngine`. The library's processor has no carve-out for the bare gift wrap a legacy direct message is carried in, so under it every one of those would have stopped decrypting, and the legacy pipeline is frozen anyway. Most of the other rows follow from that one: one store rather than four, no message-id column, the rotation manager constructed and unused, the engine hanging off the database rather than an identity's scope. The rest record the smaller turns -- the envelope's kind, the chronicle cap measured escaped, the KeyPackage not last-resort, the admin refused before a leave is attempted, captions from the participants diff rather than a snapshot column, no rename, the two non-chat kinds only, the four Phase 8 scenarios spread over four tests -- and the three follow-ups that closed lines the phases' own commits had missed: the outbound hold while a pass is open, the rejected commit discarded, and the 1210/1009 bubbles. The thing found on the way is quartz 1.15.2's: its convergence rebuilds a candidate branch by replaying its commits against the retained parent, and a member cannot replay a commit it authored, so a fork this device is half of is scored with its own branch stale and two committers swap branches rather than agree. Every device that only observed still agrees, which is the case the app depends on and the first fork test holds; the second pins the swap and says what to do the day it fails. The paragraph says the practical exposure is small and why, and that the Phase 5 follow-up on withdrawing a losing branch's rows waits on the same fix. Phase 9's own section gains "Where it stands": everything ships built and unflipped -- `DEFAULT_FOR_NEW_ROOMS` is legacy, both slots are published, no room is on the current profile unless a caller asked by name -- so the first of the three one-way doors is behind us and the other two wait on the flip. The README's row and reading-order paragraph now say the plan has been built and point at the table's first and last rows. One code change, a comment: the plan's Phase 5 asked that the two library gaps the rewrite does not close -- the retained-epoch decrypt without sender verification, and the skipped keys that do not survive `saveState` -- be said where the new code is, rather than let the rewrite look like it closed them. `MarmotEngine`'s class doc now says both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
2bbe1fc9bc
commit
de5b2bd4a6
3 files changed
+72
-9
No files matched your search
@@ -49,6 +49,14 @@ import to.curare.compose.exceptions.MarmotConvergencePendingException
|
||||
* matched to this device's bundles by the app's own table, which is profile-aware
|
||||
* and matched by reference since Phase 1, and the processor reads the rotation
|
||||
* manager only on its own Welcome path, which the app does not use.
|
||||
*
|
||||
* Two gaps the app's own pipeline had are not closed by moving to this one, and
|
||||
* should not be read as closed. A late application message decrypted under a
|
||||
* retained prior epoch is decrypted without the sender's leaf signature being
|
||||
* verified -- the library's `tryDecryptWithRetainedEpoch` has the same gap the
|
||||
* app's had. And the secret tree's skipped keys do not survive `saveState`
|
||||
* (docs/mls-skipped-keys.md); the manager holding the live group covers that
|
||||
* within a process, exactly as `MlsGroupCache` did, and no further.
|
||||
*/
|
||||
class MarmotEngine(
|
||||
private val database: CurareDatabase,
|
||||
|
||||
+6
-3
@@ -22,7 +22,7 @@ silent, or a decision that looked arbitrary and was not.
|
||||
| [jvm-target.md](./jvm-target.md) | what desktop support cost, phased — why the native chain was already done, why an empty source set in our phoenix fork was the real blocker, and why DAO tests need none of it |
|
||||
| [npub-profile-preview.md](./npub-profile-preview.md) | showing the person before a direct message is started from a pasted npub — why the preview is a screen keyed by public key, what "found" means when a row can be a placeholder, and the phase that makes the button honest about a key package that never comes |
|
||||
| [material-design-conformance.md](./material-design-conformance.md) | what the M3 foundations actually require, measured against all 43 screens — the colour pairing that renders the app's own proposals invisible, and eight phases that put the decisions back in the theme |
|
||||
| [marmot-v2.md](./marmot-v2.md) | bringing the group layer onto the Marmot current profile after the quartz 1.15.2 bump — the nine rules that reach this app, why the room's id and the routing id have to become two values, and nine phases from reading both profiles to publishing before applying |
|
||||
| [marmot-v2.md](./marmot-v2.md) | bringing the group layer onto the Marmot current profile after the quartz 1.15.2 bump — the nine rules that reach this app, why the room's id and the routing id have to become two values, nine phases from reading both profiles to publishing before applying, and the one fork the library cannot yet settle |
|
||||
|
||||
Start with the ceremony if you are new to this area; the Marmot notes all assume it.
|
||||
Read the skipped-keys note before debugging any "the other device never got it"
|
||||
@@ -59,8 +59,11 @@ smallest of the plans: it changes the entrance to the direct-message flow and
|
||||
nothing past it, so read it with `StartDirectMessageToNpubOrNip05Dialog` and
|
||||
`ChatRoomMessagingViewModel.initiateNewChat` open, its four decisions before its
|
||||
four phases, and its table of where the build chose differently first.
|
||||
The Marmot v2 note is a phased plan that has not been built, and is the largest of
|
||||
The Marmot v2 note is a phased plan that has been built, and is the largest of
|
||||
them: it assumes the membership and derivation notes, takes the quartz 1.15.2 bump
|
||||
as its Phase 0, and is ordered so that the phases that reach other clients come
|
||||
before the ones that make the app's forks settle the way theirs do; read its table
|
||||
of nine rules and its two constraints before any phase.
|
||||
of nine rules and its two constraints before any phase, and its table of where the
|
||||
build chose differently before any of the code -- its first row, that a room's
|
||||
profile picks its pipeline, is the shape of everything that follows, and its last
|
||||
is the one fork the library cannot yet settle.
|
||||
+58
-6
@@ -15,12 +15,57 @@ invite race the current pipeline has, which v2's publish rule is the real fix fo
|
||||
The second explains why a room's id *is* a key -- and that is the thing v2's routing
|
||||
rule is about to collide with.
|
||||
|
||||
**Not built.** Phase 0 is done in a worktree and is a two-file compile fix; nothing
|
||||
after it exists. The estimates add up to about five weeks, which makes this the
|
||||
largest of the plans, and it is deliberately ordered so that the phases that buy
|
||||
interoperability with other v2 clients come before the phases that make the app
|
||||
*conform* -- a member can talk to White Noise a month before its forks settle the
|
||||
way White Noise's do.
|
||||
**Built**, phases 0-8, one commit each and three follow-ups where a phase's commit
|
||||
had left a line of its own phase unbuilt; Phase 9 is the rollout and is process
|
||||
rather than code, and where it stands is noted at its end. The phases are kept as
|
||||
written because they are the reasoning, and the code reads better against the
|
||||
argument it came from than against a summary of itself. It is deliberately ordered
|
||||
so that the phases that buy interoperability with other v2 clients come before the
|
||||
phases that make the app *conform* -- a member can talk to White Noise before its
|
||||
forks settle the way White Noise's do. Where the implementation chose differently
|
||||
the table below says so, and the first of its rows is the one to read before any
|
||||
of the code:
|
||||
|
||||
| what the plan said | what it turned out to be |
|
||||
|---|---|
|
||||
| Phase 5: the library's engine *replaces* the app's pipeline -- `MarmotInboundManager` shrinks to the membership helpers, `MlsGroupCache` and the vendored `CommitOrdering` are deleted | **a room's profile picks its pipeline.** A legacy room keeps the app's pipeline exactly as it was, comparator and cache included; a current-profile room goes through `MarmotEngine`, and `NostrDao.indexMarmotGroupEvent` branches on `ChatRoom.marmotProfile()`. The library's processor has no bare-wrap carve-out, so under it every legacy direct message would have stopped decrypting, and the legacy pipeline is frozen anyway: it retires with the profile, not before |
|
||||
| Phase 1: the accessors, "in a small extension file beside `MlsGroupCache`" | `MlsGroupProfile.kt`, and six of them rather than three, because Phase 4 made the room's id a thing a group has to say about itself: `roomId()` reads the "Room id:" line `MarmotRoomIdentity` puts in the profile's description, then `0x8004`, then the MLS group id; `routingId()` is the address without the first step |
|
||||
| Phase 2: the kind guard in `mip03Rejection` "goes", and `MarmotMip03CarveOutTest` inverts into the test that it is gone | it stays, for legacy rooms: `mip03Rejection(..., legacyRoom)`. A current-profile room refuses the bare wrap as a conformant peer would; a legacy room admits it, because every build in one still sends it. The test holds both halves |
|
||||
| Phase 2: the envelope's kind is "the next free number above 30329" | 30330, `MarmotDirectMessage.ENVELOPE_KIND`; `wrap` is unchanged and `envelope` goes around it, `wrapInside` takes it off |
|
||||
| Phase 2: bring `MAX_PAGE_BYTES` to 48 KiB and re-size `MAX_PAGE_EVENTS` | 48 KiB and 96 -- measured *escaped*. A page's JSON travels as a string inside the payload's `content`, so the bound that matters is `ChronicleEvent.escapedByteSize`, and a dense page that measured 48 KiB raw escaped to 70 KiB |
|
||||
| Phase 3: `createKeyPackage(signer, lastResort = true)` | not last-resort. The notary republishes a fresh package for whichever profile is left without one when a Welcome consumes a bundle; a last-resort leaf is for a client that cannot |
|
||||
| Phase 3: run `isValid` and `isCryptographicallyValid` at ingest | the second alone, which runs the first |
|
||||
| Phase 3: schema 20 -> 21 | 20 -> 23: the two `profile` columns (Phase 3), `MarmotRouting` (Phase 4), `MarmotPublishObligation` and `MarmotOutboundGate` (Phase 6), each an `AutoMigration` |
|
||||
| Phase 4: the two creation sites call `CurrentProfileGroupFactory.createGroup` directly | `MarmotGroupFactory.create`, one wrapper for both profiles and three sites -- the plan missed `createMlsDirectMessageChatRoom`, which makes the room behind a direct message. On the current profile it mints the routing id, and the description carries the derivation sentence *and* a "Room id: <hex>" line, because a member joining from a Welcome has nothing but the group's own state to learn the room's id from |
|
||||
| Phase 4: "extend `Outcome.BlockedOn` to a package that cannot be used" | `MarmotKeyPackageProfileException` at the add step, whose message the confirmation screen shows. `BlockedOn` still means missing, because the picker -- `getMarmotKeyPackageForPublicKey(publicKey, roomProfile)` -- refuses the wrong profile before creation ever asks, so there a wrong-profile package *is* a missing one |
|
||||
| Phase 5: four stores | one, `RoomMlsGroupStateStore`, over the state blob, `MarmotRetainedEpochSecret` and `MarmotRouting`. No `KeyPackageBundleStore` and no snapshot table: the `KeyPackageRotationManager` is constructed because the processor's constructor demands one and is never consulted, and bundles stay the app's rows with the notary's per-profile republish over them. No `MarmotIngestDedupStore`: 1.15.2's processor takes no such store, so there is nothing to mark into |
|
||||
| Phase 5: `MarmotGroupEvent` gains a `marmotMessageId` column | not added. The processor dedups by message id in memory for the life of the process; the reindex sweep still dedups by event id. The follow-up if a relay ever redelivers under a second id |
|
||||
| Phase 5: one engine "held by the identity's scope" | `CurareDatabase.marmotEngine`, one per database, restored on first use. The DAOs that need it hold the database and not a view model's scope, and a switch of identity is a restart |
|
||||
| Phase 5: the timer "beside `LiveSubscriptionManager`"; an outbound row "stays on the queue" while a pass is open | `MarmotEngine.settleLater`, armed by the ingest whenever a pass opens and ticking on the engine's own scope; and `awaitSettled` on the outbound side -- a wait of up to three ticks rather than a refusal, because the outbound queue re-emits on a table change and a pass settling is not one |
|
||||
| Phase 6: "a terminal rejection resolves `FAILED`" | built in a follow-up, once "terminal" had a definition: a NIP-01 `OK` reason a relay will keep giving (`blocked:`, `invalid:`, `pow:`, `restricted:`) is a new "rejected" status that the launch-time requeue leaves alone, `rate-limited:` and `error:` are "failed" like a timeout, and `duplicate:` is "published". The obligation goes only when every relay in scope has rejected -- `BroadcastStatus`, `MarmotEngine.discardRejected` -- and the room gets a line in the relay's words |
|
||||
| Phase 6: `gate.satisfyEmptyObligation` for the group that is only its creator | `installState` straight after `stageAddMember`, keyed off the `isOneMemberInitialGroupCreation` the app already computed |
|
||||
| Phase 7: leaving is `leaveGroup` behind a `LEAVING` gate; an admin commits the proposal "on receipt, or behind a confirmation" | on receipt (`commitStagedProposals` from `ProposalStaged`), and an admin is refused before anything is raised or dropped: the library refuses an admin's SelfRemove, so the app says "hand over admin first", or, for the only admin, "make another member an admin, or disband the group" |
|
||||
| Phase 7: captions from `MarmotGroupSnapshot` against a `marmotSnapshot` column | from the diff `processGroupMembershipChanges` already made against the participants table, on every device that applies a commit, and without a `TYPE_MEMBER_REMOVED`: the tree does not say whether a member left or was removed, so `TYPE_MEMBER_LEFT` covers both. No snapshot column |
|
||||
| Phase 7: rename is `stageAppDataUpdate` and "the button is small" | not built. There is no rename for either profile today; the subject on the detail screen is display only |
|
||||
| Phase 7: a received 1210 or 1009 gets no bubble, "and neither does any other unknown kind" | those two only, in a follow-up. The bubble stays for the rest: the kind:1 arm reaches it on purpose, and it is the only trace an unrecognised payload leaves |
|
||||
| Phase 8: four scenarios in one test | spread over four: `CurrentProfileInteropJvmTest` (the join, both directions, envelope included), `GroupLifecycleJvmTest.roomWithMember` (the app's room, the stand-in joining, routing id not room id), `ConvergenceForkJvmTest` (the fork) and `PublishBeforeApplyJvmTest` (the restart). The wire-vector test is `MarmotAppPayloadTest` and `MarmotPayloadBoundsTest` in `commonTest` |
|
||||
| Phase 8: the fork -- "whichever branch won, they agree" | proven for the case the app depends on and pinned for the case it cannot yet have. Two *other* members fork; the app and a second observer are handed both commits in opposite orders and settle on the same state byte for byte. A fork *this device authored* does not: see below |
|
||||
|
||||
One thing found on the way that is the library's and not this app's. quartz 1.15.2's
|
||||
convergence rebuilds each candidate branch by replaying its commits against the
|
||||
retained parent state, and a member cannot replay a commit it authored --
|
||||
`processFramedCommit` on its own commit fails with "UpdatePath at common ancestor
|
||||
carries no ciphertext for us", since a committer holds no key for its own copath.
|
||||
So a fork this device is half of is scored with its own branch stale, and two
|
||||
committers *swap* branches rather than agree; every device that only observed
|
||||
still agrees, which the first fork test holds. The second fork test pins the swap
|
||||
exactly as observed and says in its last assertion what to do the day it fails:
|
||||
the state the missing replay would produce is already retained, and using it is
|
||||
the fix. Until then the practical exposure is small -- a device that never applies
|
||||
before publishing rarely authors one side of a fork -- and the follow-up in Phase 5
|
||||
(marking a losing branch's rows withdrawn) waits on the same fix.
|
||||
|
||||
Written against the code as it stood at `2bbe1fc9`.
|
||||
|
||||
## Where "v2" comes from, and what it asks
|
||||
|
||||
@@ -799,6 +844,13 @@ the first current-profile room a user creates (Phase 4 flipped; it can never hol
|
||||
an old build), and the envelope (Phase 2; a legacy room that ever sends one has a
|
||||
member who cannot read it, which is why it never does).
|
||||
|
||||
**Where it stands.** Everything above ships built and unflipped:
|
||||
`MarmotProfile.DEFAULT_FOR_NEW_ROOMS` is `LEGACY`, both KeyPackage slots are
|
||||
published and kept, and no room this app has made is on the current profile
|
||||
unless a caller asked for one by name -- the tests do; the creation screens do
|
||||
not. The first door is behind us the moment a build with the bump runs. The
|
||||
other two wait on the flip, which waits on the people this app's users invite.
|
||||
|
||||
---
|
||||
|
||||
## What this does not do
|
||||
|
||||
Reference in new issue
Block a user