diff --git a/docs/README.md b/docs/README.md index 76adeea4..92b74d2e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -9,7 +9,7 @@ silent, or a decision that looked arbitrary and was not. | [shared-key-ceremony.md](./shared-key-ceremony.md) | ChillDKG over NIP-17: the rounds, the approval gates, the chat transcript, participant ordering | | [shared-key-derivation.md](./shared-key-derivation.md) | deriving further keys from the group's threshold key with FROST tweaks — why not BIP32, why no chain code, and the one rule that must not be broken | | [marmot-membership.md](./marmot-membership.md) | how members join an MLS group, and the epoch race that makes a missing member look like a successful invite | -| [long-running-sync.md](./long-running-sync.md) | keeping the chat subscriptions open instead of pulling once per screen — what the request-queue pipeline assumes about short subscriptions, and how the group filter follows the room list | +| [long-running-sync.md](./long-running-sync.md) | the chat subscriptions that stay open instead of pulling once per screen — why the request queue could not simply hold one, and how the group filter follows the room list | Start with the ceremony if you are new to this area; the next two both assume it. The sync note stands alone. diff --git a/docs/long-running-sync.md b/docs/long-running-sync.md index ef8ad0c0..9ed04e0e 100644 --- a/docs/long-running-sync.md +++ b/docs/long-running-sync.md @@ -1,333 +1,219 @@ -# A long-running chat sync +# The long-running chat sync -Today every chat sync in this app is a *pull*: a screen queues a request row, a -pump drains it, the relay answers, the subscription is closed. Nothing arrives -between pulls. This document works out what it takes to keep the chat -subscriptions **open** for as long as the app is active, so relays push new gift -wraps and group events at us instead of us asking, and what has to change in the -existing pipeline before that is even possible. +Chat messages arrive because a subscription is open, not because a screen asked. +This note is what that means, what it replaced, and the handful of decisions +inside it that look arbitrary and are not. -The target is the two kinds that carry conversation: +Two subscriptions carry conversation, and they are held open for as long as the +app is in the foreground: - **kind 1059**, NIP-59 gift wraps p-tagged to us — DMs, and the Marmot Welcome events that make us a member of a new group. -- **kind 445**, Marmot group events h-tagged with a group id we belong to. +- **kind 445**, Marmot group events h-tagged with every group id we belong to. -And one hard requirement: the group subscription must widen the moment we join -or create a group, without anyone remembering to call a "subscribe" function. +The code is [`LiveSubscriptionManager`](composeApp/src/commonMain/kotlin/press/mantra/compose/managers/LiveSubscriptionManager.kt), +built and scoped by `SynchronizationViewModel`. -## What happens today +## What this replaced -Three pumps in `SynchronizationViewModel` drain three single-row queues — -broadcast, REQ, negentropy — each row observed with `... ORDER BY createdAt ASC -LIMIT 1`. Screens are producers. Nothing else opens a socket. +Every chat sync used to be a *pull*. A screen queued a request row, a pump in +`SynchronizationViewModel` drained it, the relay answered, the subscription was +closed at EOSE. Nothing arrived between pulls, so a message sent one second after +EOSE waited for the next time someone opened a screen. -| screen | queues | +| screen | queued | |---|---| -| [`ChatRoomListViewModel.scheduleSynchronization`](composeApp/src/commonMain/kotlin/press/mantra/compose/ui/view/model/ChatRoomListViewModel.kt:71) | negentropy over `1059 #p=[me] limit 50`, then negentropy over `445 #h=[every group id]`, once per DM relay | -| [`ChatMessageListViewModel.scheduleSynchronization`](composeApp/src/commonMain/kotlin/press/mantra/compose/ui/view/model/ChatMessageListViewModel.kt:113) | for an MLS room: negentropy over `445 #h=[this group]`. For a NIP-17 room: gift wraps p-tagged to the *peer* and authored by us, or the peer's kind-10050 if we lack it | +| `ChatRoomListViewModel.scheduleSynchronization` | negentropy over `1059 #p=[me]`, then over `445 #h=[every group id]`, per DM relay | +| `ChatMessageListViewModel.scheduleSynchronization` | for an MLS room, negentropy over `445 #h=[this group]`; for a NIP-17 room, gift wraps p-tagged at the peer | -A negentropy exchange that finds ids we are missing turns them into -`SynchronizeNostrEventRequest` rows, which the REQ pump then fetches in chunks of -500 ids. Both subscription types end the same way: at EOSE the flow completes, -the `finally` sends CLOSE, the semaphore permit is released. +Neither runs on open any more. The first stays as the entry point for an +explicit user-initiated refresh — the one thing the live tier does not answer, +because it is the user saying they believe something is missing. The second now +does discovery only: if we do not hold a participant's kind-10050 we cannot +address a message to them, and that is worth resolving when a chat is opened +rather than whenever a background pass reaches it. -That design is good at what it is for — reconciling a set we already have -against a set the relay has. It is structurally incapable of delivering a message -that arrives one second after EOSE. +## Why the old pipeline could not simply stop closing -## Why we cannot simply stop closing +Five properties, each of which had to be undone deliberately. None was wrong; +all of them assume a subscription is short. -Five properties of the current pipeline each have to be undone deliberately. -None of them is wrong; all of them assume a subscription is short. +**1. EOSE ends the flow.** `completeOnSubscriptionEnd` is what makes a collector +finish at all — the socket's `incomingMessages` is hot, and a filtered view of it +never completes on its own. `RelayPool.openLiveSubscription` is `query()` without +it: for a live subscription EOSE is only the boundary between stored history and +the live tail, and CLOSED is the only genuinely terminal message. -**1. EOSE ends the flow.** -[`completeOnSubscriptionEnd`](composeApp/src/commonMain/kotlin/press/mantra/compose/network/sockets/NostrIncomingMessageExt.kt:42) -is what makes a collector finish at all — the socket's `incomingMessages` is hot -and a filtered view of it never completes on its own. For a live subscription, -EOSE is not the end, it is the boundary between "stored history" and "live -tail", and CLOSED is the only genuinely terminal message. - -**2. Every subscription is force-killed at 120 seconds.** -[`SUBSCRIPTION_TIMEOUT`](composeApp/src/commonMain/kotlin/press/mantra/compose/ui/view/model/SynchronizationViewModel.kt:98) -exists so a relay that goes quiet cannot park a slot forever. A live -subscription is exactly the case it is designed to kill. +**2. Every subscription was force-killed at 120 seconds.** `SUBSCRIPTION_TIMEOUT` +exists so a relay that goes quiet cannot park a slot forever. It applies to the +queue, and nothing else. **3. There are four subscription slots, total, across all relays.** -[`subscriptionSlots`](composeApp/src/commonMain/kotlin/press/mantra/compose/ui/view/model/SynchronizationViewModel.kt:154) -is a `Semaphore(4)`. A permanently-open subscription is a permanently-held -permit. Two live subscriptions would leave the backfill queue with two slots and -a permanent halving of throughput. **Live subscriptions must not draw from this -semaphore** — they need their own budget, and the semaphore's comment ("well -under the ~20-per-connection cap relays commonly enforce") needs re-reading as a -combined budget rather than a queue-only one. +`subscriptionSlots` is a `Semaphore(4)`. A permanently-open subscription is a +permanently-held permit, so the live tier deliberately does not draw from it. The +two budgets have to be read together against what a relay tolerates per +connection (commonly ~20), not separately. -**4. The queue row *is* the subscription identity and the provenance record.** -The subId is the row's UUID, and both `saveNostrEvent` overloads -([`DatabaseNostrRepository.kt:483`](composeApp/src/commonMain/kotlin/press/mantra/compose/database/repository/DatabaseNostrRepository.kt:483), -[`:515`](composeApp/src/commonMain/kotlin/press/mantra/compose/database/repository/DatabaseNostrRepository.kt:515)) -take a request object, use its `relayURL`/`level`, and flip its status to -"processed". A live subscription has no row and never finishes. It needs a third -entry point that goes straight to -[`NostrDao.storeNostrEvent`](composeApp/src/commonMain/kotlin/press/mantra/compose/database/dao/NostrDao.kt:141) -with an explicit `relayURL` and `level = 0`. +**4. The queue row was the subscription identity and the provenance record.** Both +original `saveNostrEvent` overloads take a request object, use its `relayURL` and +`level`, and flip its status to "processed". A live subscription has no row and +never finishes, so there is a third overload carrying relay and level itself. -**5. Nothing reconnects.** -`NostrSocketClientImpl` catches a socket failure, calls `close()`, and fires -`onSocketConnectionClosed`, which only flips a boolean in `relayPoolStatus`. -There is no retry loop anywhere. Today that is invisible: every subscription is -short, and the next queued request re-opens the socket through -`ensureSocketConnectionOrThrow`. A live subscription that outlives a socket drop -is a subscription that has silently stopped, on a socket whose status flag says -so and which nobody reads. +**5. Nothing reconnected.** `NostrSocketClientImpl` caught a failure, called +`close()`, and fired a callback that only flipped a boolean nobody read. That was +survivable *only* because every subscription was short: the next queued request +opened a fresh socket on its way out. See "Reconnecting" below. -There is a sixth, quieter problem. `_incomingMessages` is a -[`MutableSharedFlow()`](composeApp/src/commonMain/kotlin/press/mantra/compose/network/sockets/NostrSocketClientImpl.kt:53) -with no replay and no buffer, and each frame is emitted from its own `launch` -([`processIncomingMessage`](composeApp/src/commonMain/kotlin/press/mantra/compose/network/sockets/NostrSocketClientImpl.kt:146)). -A message emitted while nothing is collecting is dropped, and messages can be -delivered out of order relative to the wire. For request/response that is -harmless — the collector is attached before the REQ goes out. For a firehose -whose consumer does MLS decryption and SQLite writes, it is not: give the router -a buffered channel and process in arrival order per relay. +There was a sixth, quieter one. `_incomingMessages` was a rendezvous +`MutableSharedFlow` emitted into from a coroutine launched per message, so +messages reached collectors in scheduling order rather than wire order, and an +emit with every collector busy blocked on the slowest. It now has a 256-message +buffer and is emitted into inline, on the reader. That also retired a 75ms sleep +before every EOSE, which existed to *hope* the events preceding it had already +been delivered. -## What Wisp does +## How it holds together -Wisp (`/home/sigidli/Documents/development/nostr/wisp`) keeps its subscriptions -open for the life of the foreground app. Five mechanisms are worth copying: +### Reconnecting -- **Stable, meaningful subscription ids.** `"dms"`, `"notif"`, `"grp-"`. Not - UUIDs. Re-sending a REQ with the same id replaces the filter server-side, which - makes "widen the filter" a send rather than a close-and-reopen. -- **The REQ text is retained per relay.** `activeSubscriptions: Map>`, and `resyncSubscriptions(relay)` re-sends all of them - when that relay reconnects (`relay/RelayPool.kt`). -- **A per-relay subscription budget.** `SubscriptionTracker` enforces a soft cap - of 20, with a priority prefix list (`"dms"`, `"notif"`, `"grp-"`, …) that - bypasses the cap so a transient feed subscription can never crowd out DMs. -- **Explicit reconnect policy.** `RelayLifecycleManager` observes connectivity - and app pause/resume, debounces, and drives `reconnectAll()`; on reconnect only - the long-lived prefixes are retained and everything transient is dropped. -- **One router, not one collector per request.** `viewmodel/EventRouter.kt` - dispatches by subscription id. +A socket that drops now comes back on its own, with exponential backoff (1s +doubling to 60s, plus up to 25% jitter — every relay drops at once when the +network does, and without jitter they return in lockstep). `autoReconnect` is off +by default and owned by the pool, which turns it on for exactly as long as it is +retaining a subscription for that relay: reconnecting a socket nobody is +subscribed on is battery spent on nothing. -And one detail that is a direct warning to us: Wisp deliberately does **not** -put a `since` on its kind-1059 filter, because NIP-59 randomizes `created_at` -into the past. +`RelayPool` retains the REQ text per `(relay, subscription id)` and replays it +when that relay's socket is re-established. A relay answers a repeated REQ on the +same id by *replacing* that subscription's filter, so replay is a send rather +than a close-and-reopen — and the collector, attached to the socket **client** +rather than to a session, simply starts receiving again. A live subscription +therefore needs no drop handling of its own. -## The shape to build +Negentropy is deliberately not retained. NEG-OPEN carries a fingerprint of the +local set and each round depends on the last, so replaying one mid-exchange would +reconcile against a conversation the relay is no longer having. -### A live tier beside the queue, not inside it +### The filters, and their timestamps -A `LiveSubscriptionManager` owned at the same level as `SynchronizationViewModel` -(the application scope, not a screen), holding one piece of state: +The two kinds need opposite treatment, and it is visible in our own outbound code. -```kotlin -// what we want to be subscribed to, per relay -data class LiveSubscription(val subId: String, val relayUrl: String, val filters: List) -``` +Gift wraps are stamped with `TimeUtils.randomWithTwoDays()` +([`DatabaseChatRepository.kt:344`](composeApp/src/commonMain/kotlin/press/mantra/compose/database/repository/DatabaseChatRepository.kt:344)), +so a wrap published *now* can carry a `created_at` two days old. **A `since` +anywhere near the present would silently drop a large fraction of genuinely new +messages** — and "some DMs just don't arrive" is the worst failure mode to debug. +There is none. -and doing one thing: **reconcile desired state against what has actually been -sent to each socket.** Recompute the desired set, diff it against the sent set, -send a REQ for anything added or changed, send CLOSE for anything removed. Every -trigger — a new group, a relay reconnect, an app resume, a relay-list change — -becomes a call to the same `reconcile()`. That is the property that makes this -maintainable: there is no "subscribe" path and "resubscribe" path that can drift. +Group events use `TimeUtils.now()` +([`MarmotOutboundDao.kt:578`](composeApp/src/commonMain/kotlin/press/mantra/compose/database/dao/MarmotOutboundDao.kt:578)), +so a watermark would be safe there. It still is not used: `limit` already bounds +the initial burst, and a watermark would have to be recomputed every time a +chunk's membership changed. -The sent-set belongs in `RelayPool`, next to `socketClients`, because that is -where a reconnect is observed. Mirror Wisp: keep the serialized REQ per -`(relayUrl, subId)` and re-send on connect. +`limit` does the bounding instead, because NIP-01 scopes it to the initial query +— the stored events sent before EOSE — and explicitly not to the stream that +follows. So it caps what a reconnect costs without touching the live tail at all. +100 for gift wraps; 500 for a group chunk, which covers up to 100 conversations. -### The two filters, and their timestamps +`#h` is chunked 100 group ids to a subscription. One subscription per group would +be simpler, but every Marmot group here lives on the same DM relay set, so +carrying many ids in one filter costs a fraction of the subscriptions. -```kotlin -// live-giftwrap -Filter(kinds = listOf(GiftWrapEvent.KIND), tags = mapOf("p" to listOf(myPubkey))) +### Following group membership -// live-groups- -Filter(kinds = listOf(GroupEvent.KIND), tags = mapOf("h" to groupIdChunk)) -``` +The group filter is derived from `chatRepository.observeChatRoomListByPublicKey` +— the same Flow the chat list screen renders, collected at wallet scope so it +keeps running with no chat screen open. Rooms are filtered to those with MLS +state (a NIP-17 room has none and is served by the gift wrap subscription), minus +any we have left or deleted; ids are sorted, de-duplicated with +`distinctUntilChanged`, and debounced 500ms. -The two kinds need opposite treatment on `since`, and this is not a matter of -taste — it is visible in our own outbound code. - -Gift wraps are stamped with -[`TimeUtils.randomWithTwoDays()`](composeApp/src/commonMain/kotlin/press/mantra/compose/database/repository/DatabaseChatRepository.kt:344), -so a wrap published *now* can carry a `created_at` up to two days in the past. -A live filter with `since = now` would silently drop a large fraction of -genuinely new messages, and the failure mode is "some DMs just don't arrive" — -the worst kind to debug. Either omit `since` entirely (gift wraps are low volume -and `storeNostrEvent` already no-ops on a known id) or floor it at -`now - 2 days` and accept the redelivery. - -Group events use -[`TimeUtils.now()`](composeApp/src/commonMain/kotlin/press/mantra/compose/database/dao/MarmotOutboundDao.kt:578). -A watermark is safe there: `since = (newest 445 we hold for these groups) - slack`. -Keep the slack generous (minutes, not seconds) — relay clocks drift, and the -cost of re-receiving a handful of 445s is a few no-op `storeNostrEvent` calls. - -`#h` has to be chunked. One subscription per group is the simple option and what -Wisp does for NIP-29, but Marmot groups here are all on the same DM relay set, so -one subscription carrying N group ids is far cheaper in sockets and slots. Chunk -at a conservative width (~100 ids) across `live-groups-0`, `live-groups-1`, … and -accept that a chunk boundary shift re-sends one REQ. - -### Routing - -One collector per socket, attached for the life of the socket, dispatching on -subscription id prefix — the `EventRouter` shape. That collector is also what -fixes problem six above: it is always attached, so nothing is dropped for want -of a subscriber, and it can impose ordering and back-pressure in one place. - -Events off a live subscription go to the new request-free `saveNostrEvent`, and -from there through the same `storeNostrEvent` → `MarmotInboundManager` path as -everything else. **No inbound processing changes.** A 1059 carrying a Welcome -still lands in the branch at -[`NostrDao.kt:720`](composeApp/src/commonMain/kotlin/press/mantra/compose/database/dao/NostrDao.kt:720); -a 445 still reaches -[`processGroupEvent`](composeApp/src/commonMain/kotlin/press/mantra/compose/database/dao/NostrDao.kt:415). -Only the delivery mechanism is new. - -### Keeping the group filter current - -This is the requirement that decides the design, and it is the easy part — -provided we derive the filter from the database instead of from the join sites. - -```kotlin -chatRepository.observeChatRoomListByPublicKey(publicKey) - .map { rooms -> rooms.filter { it.chatRoom.mlsGroupState != null && it.chatRoom.leftGroupAt == null } - .map { it.chatRoom.id }.sorted() } - .distinctUntilChanged() - .debounce(500.milliseconds) - .collect { groupIds -> live.setGroupIds(groupIds); live.reconcile() } -``` - -`ChatRoomListViewModel` already collects this exact Flow to render the list. The -difference is where it is collected: at application scope, so it keeps running -when no chat screen is open. - -Driving it off the DB rather than off the join sites matters because there are -several ways a group id appears, and they are not all in view models: +Deriving it from the table rather than from the join sites is the decision that +keeps this correct, because a group id can appear four ways and only one of them +is somewhere anyone would think to call a subscribe function: - we create a group (`MarmotOutboundDao`), - we are added to one — the Welcome arrives inside a gift wrap and is processed deep inside `NostrDao.storeNostrEvent`, on the *inbound* path, - membership shifts under us via a commit (`MarmotInboundManager.processGroupMembershipChanges`), -- and we leave one (`leftGroupAt`). +- we leave, or the room is deleted. -Only the first is a place a developer would naturally think to call -`subscribeToGroup()`. Observing the room list catches all four, and it closes the -loop cleanly: **a Welcome arrives on the live gift wrap subscription → a +It also closes the loop: **a Welcome lands on the gift wrap subscription → a `ChatRoom` row is written → the Flow re-emits → the group filter widens → the -first message in that new group arrives without anyone opening the chat.** +first message in that new group arrives without anyone opening a chat.** -Debounce is not optional. Joining a group writes the room, its participants, and -placeholder profiles in quick succession; without it we would re-send the group -REQ several times per join. +Reconciling is an update, not a rebuild. A chunk that already has a subscription +gets a repeated REQ under the same id, so adding a group does not interrupt +delivery on the groups already being watched. Chunks that no longer exist are +`cancelAndJoin`ed — the CLOSE goes out from the coroutine's `finally`, and +returning before it lands would let a later reconcile open a subscription on that +id which the old one then closes out from under. ### Lifecycle -Mantra has no app lifecycle observer at all today — -[`MainActivity`](composeApp/src/androidMain/kotlin/press/mantra/android/MainActivity.kt) -only calls `setContent`. Long-running subscriptions make one necessary: +`AppLifecycle` is a singleton `StateFlow` fed by a `LifecycleEventObserver` +in `MantraNavHost`. A singleton rather than something threaded through the +composition, because the consumers are application-scoped coroutines that outlive +any screen. It defaults to foreground: on a platform where the observer is not +wired up, "always on" is the behaviour that predates it, and a subscription that +never opens is far worse than one that stays open too long. -- **background:** stop the live subscriptions (CLOSE, or just drop the sockets). - Holding a socket open behind a doze window achieves nothing except battery. -- **foreground:** reconnect and reconcile. Assume the server-side subscription is - gone — Wisp's `forceReconnectAll` exists precisely because a socket that - survives an OS sleep reports `isConnected == true` while being functionally - dead — and queue a bounded catch-up negentropy pass for the gap. -- **connectivity change:** reconnect, debounced, with the suppression window - Wisp uses to avoid a resume and a network-change event double-reconnecting. +`collectLatest` over that flow is the whole mechanism. Backgrounding cancels the +block holding the subscriptions and each `finally` sends its CLOSE. -The foreground catch-up is what preserves correctness across the gap. Live -subscriptions cover the online window; negentropy covers everything else. +Returning does two things before anything else: -## What the screens stop doing, and what they keep +1. **`RelayPool.reconnectAll`** tears every socket down and immediately rebuilds + it. Trusting the connection is the mistake here — a socket that was open when + the OS suspended the process reports itself connected on the way back while + being functionally dead. Rebuilding eagerly rather than leaving it to the next + send is what makes it a *re*-connection, so retained subscriptions are replayed + and collectors still attached from before the gap resume. +2. **A catch-up reconciliation.** A live subscription answers "what is new since I + connected"; negentropy answers "what do you have that I don't". A background + gap is exactly the question only the second can answer — `limit` on the + re-opened subscriptions is a window, not a guarantee. -Once the live tier exists: +### What negentropy is still for -- `ChatRoomListViewModel.scheduleSynchronization` — drop both queues on open. - Keep the code path behind an explicit pull-to-refresh. -- `ChatMessageListViewModel.scheduleSynchronization` — drop the `mlsMessages` - branch entirely; it is a strict subset of `live-groups-*`. Keep the - `ChatMessageRelayListEvent` lookup: that is *discovery* (which relay does this - peer read from), not message sync, and it still has to happen on open. -- Everything not chat — profiles, follows, feeds, key packages — is untouched. +Live subscriptions replaced *polling*, not *reconciliation*. The queue and its +three pumps are unchanged, and negentropy remains the right tool for first +login, the foreground catch-up, "load older messages", and repairing a relay that +was unreachable while we were online. -One thing to fix while in here, since it is one of the per-chat syncs we are -proposing to stop firing: the `"sent-messages"` filter -(`kinds=[1059], authors=[me], #p=[peer]`, at -[`ChatMessageListViewModel.kt:113`](composeApp/src/commonMain/kotlin/press/mantra/compose/ui/view/model/ChatMessageListViewModel.kt:113) -and [`NostrDao.kt:643`](composeApp/src/commonMain/kotlin/press/mantra/compose/database/dao/NostrDao.kt:643)) -**cannot match anything.** Gift wraps are signed with a fresh throwaway key -([`val wrapperKeyPair = KeyPair()`](composeApp/src/commonMain/kotlin/press/mantra/compose/database/repository/DatabaseChatRepository.kt:312)), -so `authors = [myPubkey]` matches no wrap this app has ever published. It also -turns out to be unnecessary: `createNip17ChatRoom` puts the user in their own -participant list -([`NostrNip17Dao.kt:50`](composeApp/src/commonMain/kotlin/press/mantra/compose/database/dao/NostrNip17Dao.kt:50)), -so we wrap a copy to ourselves and the account-wide `#p=[me]` subscription picks -our own sent messages up on every device. Delete the filter rather than port it. +## Two things found while building this -## Negentropy's remaining job +**The `"sent-messages"` filter could never match anything.** It asked for kind +1059 with `authors=[userPublicKey]`, but gift wraps are signed with a fresh +throwaway `KeyPair()`, so the pubkey is random and never ours. It was also +unnecessary: `createNip17ChatRoom` puts the user in their own participant list, so +we wrap a copy to ourselves and the account-wide `#p=[me]` subscription picks our +own sent messages up on every device. Deleted, in the view model and at both +sites in `NostrDao`. -Live subscriptions replace *polling*, not *reconciliation*. Negentropy is still -the right tool for: +**The negentropy `saveNostrEvent` wrote outside `storeNostrEventMutex`** while +the other overload held it. `storeNostrEvent` reads an event and then writes it +and its indexes, so two of those interleaving is a lost update — survivable while +a single queue was the only writer, not survivable with a live subscription +writing alongside a backfill. -- first login / restore, where the local set is empty, -- the foreground catch-up after a background gap, -- "load older messages" in a chat, -- and repairing a relay that was unreachable while we were online. +## Not done -The distinction to hold onto: a live REQ answers "what is new since I connected"; -negentropy answers "what do you have that I don't". Neither subsumes the other, -and the queue and its three pumps stay exactly as they are. - -## Traps, in the order they will bite - -1. **Reusing the `subscriptionSlots` semaphore.** Two permanent permits out of - four halves backfill throughput forever. Give the live tier its own budget and - read the two together against the relay's real cap. -2. **`completeOnSubscriptionEnd` on a live flow.** It will complete at the first - EOSE and the subscription will look established while delivering nothing. - The live variant must terminate on CLOSED only. -3. **`since = now` on kind 1059.** Randomized wrap timestamps mean silent, - partial message loss. See above. -4. **Treating CLOSED as "retry now".** `isBackPressure` already exists in - `SynchronizationViewModel` and should be reused: a rate-limited CLOSED on a - live subscription needs exponential backoff, an auth-required CLOSED needs the - AUTH flow (which the app does not implement yet — worth knowing before - pointing this at a relay that requires it), and only an unsupported-filter - CLOSED should fall back to a queued REQ. -5. **A NOTICE has no subscription id.** It is already admitted to every - collector by `filterBySubscriptionId`; with a permanent collector attached, - every notice reaches it forever. Handle it once in the router, not per - subscription. -6. **Duplicate delivery.** `storeNostrEvent` no-ops on a known id, so duplicate - 1059s are cheap. Duplicate *445s* are not automatically cheap — - `MarmotInboundManager` maintains epoch state and a commit tracker; confirm the - re-delivery path is idempotent before widening the `since` slack. -7. **Relay list changes.** `RelaysSocketManager.updateRelayPools` calls - `relayPool.changeRelays`, which closes sockets. Live subscriptions must be - reconciled after that, or they quietly vanish on any relay edit. -8. **The DM relay set is one relay.** `Relays.DefaultDMRelayList` is - `[wss://ephemeral.mantra.press]`. Every point above about per-relay budgets is - currently a budget of one socket — which makes this cheap to build and makes a - single relay outage total. Worth widening the set at the same time. - -## Suggested order - -1. **Reconnect first.** A supervised reconnect loop in `NostrSocketClientImpl` - plus per-relay REQ retention and resync in `RelayPool`. This is a strict - improvement on its own, before any live subscription exists. -2. **The router.** One permanent collector per socket, dispatching by subId - prefix; the request-free `saveNostrEvent`. Existing pumps keep working. -3. **`live-giftwrap`.** One subscription, one relay, no `since`. Verifiable by - sending yourself a DM from another client with the chat list closed. -4. **`live-groups-*`,** driven off the room-list Flow. Verify by having a second - device add this one to a group and watching messages land with no chat screen - ever opened. -5. **Lifecycle + foreground catch-up.** -6. **Then, and only then,** delete `scheduleSynchronization` from the two chat - view models. +- **Connectivity changes.** A network switch mid-foreground is only noticed by + the socket's own reconnect loop, which handles the common case but cannot know + the network changed underneath it. Wisp's `RelayLifecycleManager` is the model: + observe connectivity, debounce, and suppress a network-change reconnect shortly + after a resume so the two do not fire twice. +- **NIP-42 AUTH.** A relay that answers CLOSED with `auth-required` is treated as + any other refusal and retried with backoff. It will never succeed. Worth + knowing before pointing this at a relay that requires auth. +- **A router.** The original design called for one permanent collector per socket + dispatching by subscription id prefix. With the inbound flow buffered and only a + handful of live subscriptions per relay, a collector per subscription has the + same properties for less machinery. If that count grows, the router is the next + move. +- **`Relays.DefaultDMRelayList` is one relay.** Every per-relay budget above is + currently a budget of one socket, which makes this cheap and makes a single + relay outage total.