96a9aa6e63 feat(chain): refactor `merge_chains` (志宇)
2f22987c9e chore(chain): fix comment (志宇)
daf588f016 feat(chain): optimize `merge_chains` (志宇)
77d35954c1 feat(chain)!: rm `local_chain::Update` (志宇)
1269b0610e test(chain): fix incorrect test case (志宇)
72fe65b65f feat(esplora)!: simplify chain update logic (志宇)
eded1a7ea0 feat(chain): introduce `CheckPoint::insert` (志宇)
519cd75d23 test(esplora): move esplora tests into src files (志宇)
a6e613e6b9 test(esplora): add `test_finalize_chain_update` (志宇)
494d253493 feat(testenv): add `genesis_hash` method (志宇)
886d72e3d5 chore(chain)!: rm `missing_heights` and `missing_heights_from` methods (志宇)
bd62aa0fe1 feat(esplora)!: remove `EsploraExt::update_local_chain` (志宇)
1e99793983 feat(testenv): add `make_checkpoint_tip` (志宇)
Pull request description:
Fixes#1354
### Description
Built on top of both #1369 and #1373, we simplify the `EsploraExt` API by removing the `update_local_chain` method and having `full_scan` and `sync` update the local chain in the same call. The `full_scan` and `sync` methods now takes in an additional input (`local_tip`) which provides us with the view of the `LocalChain` before the update. These methods now return structs `FullScanUpdate` and `SyncUpdate`.
The examples are updated to use this new API. `TxGraph::missing_heights` and `tx_graph::ChangeSet::missing_heights_from` are no longer needed, therefore they are removed.
Additionally, we used this opportunity to simplify the logic which updates `LocalChain`. We got rid of the `local_chain::Update` struct (which contained the update `CheckPoint` tip and a `bool` which signaled whether we want to introduce blocks below point of agreement). It turns out we can use something like `CheckPoint::insert` so the chain source can craft an update based on the old tip. This way, we can make better use of `merge_chains`' optimization that compares the `Arc` pointers of the local and update chain (before we were crafting the update chain NOT based on top of the previous local chain). With this, we no longer need the `Update::introduce_older_block` field since the logic will naturally break when we reach a matching `Arc` pointer.
### Notes to the reviewers
* Obtaining the `LocalChain`'s update now happens within `EsploraExt::full_scan` and `EsploraExt::sync`. Creating the `LocalChain` update is now split into two methods (`fetch_latest_blocks` and `chain_update`) that are called before and after fetching transactions and anchors.
* We need to duplicate code for `bdk_esplora`. One for blocking and one for async.
### Changelog notice
* Changed `EsploraExt` API so that sync only requires one round of fetching data. The `local_chain_update` method is removed and the `local_tip` parameter is added to the `full_scan` and `sync` methods.
* Removed `TxGraph::missing_heights` and `tx_graph::ChangeSet::missing_heights_from` methods.
* Introduced `CheckPoint::insert` which allows convenient checkpoint-insertion. This is intended for use by chain-sources when crafting an update.
* Refactored `merge_chains` to also return the resultant `CheckPoint` tip.
* Optimized the update `LocalChain` logic - use the update `CheckPoint` as the new `CheckPoint` tip when possible.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
LLFourn:
ACK 96a9aa6e63
Tree-SHA512: 3d4f2eab08a1fe94eb578c594126e99679f72e231680b2edd4bfb018ba1d998ca123b07acb2d19c644d5887fc36b8e42badba91cd09853df421ded04de45bf69
`merge_chains` now returns a tuple of the resultant checkpoint AND
changeset. This is arguably a more readable/understandable setup.
To do this, we had to create `CheckPoint::apply_changeset` which is kept
as a private method.
Thank you @ValuedMammal for the suggestion.
Co-authored-by: valuedvalued mammal <valuedmammal@protonmail.com>
e51af49ffa fix(wallet): remove generic from wallet (Rob N)
Pull request description:
### Description
The `PersistenceBackend` uses generics to describe errors returned while applying the change set to the persistence layer. This change removes generics wherever possible and introduces a new public error enum. Removing the generics from `PersistenceBackend` errors is the first step towards #1363
*Update*: I proceeded with removing the generics from `Wallet` by introducing a `Box<dyn PersistenceBackend>` .
### Notes to the reviewers
This one sort of blew up in the number of changes due to the use of generics for most of the `Wallet` error variants. The generics were only used for the persistence errors, so I removed the generics from higher level errors whenever possible. The error variants of `PersistenceBackend` may also be more expressive, but I will level that up for discussion and make any changes required.
### Changelog notice
- Changed `PersistenceBackend` errors to depend on the `anyhow` crate.
- Remove the generic `T` from `Wallet`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [x] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
evanlinjin:
ACK e51af49ffa
Tree-SHA512: 8ce4f1c495310e16145555f4a6a29a0f42cf8944eda68004595c3532580767f64f779185022147a00d75001c40d69fdf8f8de2d348eb68484b170d2a181117ff
The intention is to remove the `Update::introduce_older_blocks`
parameter and update the local chain directly with `CheckPoint`.
This simplifies the API and there is a way to do this efficiently.
Ensure that `CheckPoint::range` returns expected values by comparing
against values returned from a primitive approach.
I think it is a good idea to start writing proptests for this crate.
a2a64ffb6e fix(electrum)!: Remove `seen_at` param from `into_tx_graph` (valued mammal)
37fca35dde feat(tx_graph): Add method update_last_seen_unconfirmed (valued mammal)
Pull request description:
A method `update_last_seen_unconfirmed` is added for `TxGraph` that allows inserting a `seen_at` time for all unanchored transactions.
fixes#1336
### Changelog notice
Changed:
- Methods `into_tx_graph` and `into_confirmation_time_tx_graph`for `RelevantTxids` are changed to no longer accept a `seen_at` parameter.
Added:
- Added method `update_last_seen_unconfirmed` for `TxGraph`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK a2a64ffb6e
Tree-SHA512: 9011e63314b0e3ffcd50dbf5024f82b365bab1cc834c0455d7410b682338339ed5284caa721ffc267c65fa26d35ff6ee86cde6052e82a6a79768547fbb7a45eb
Upgrade:
- bitcoin to v0.31.0
- miniscript to v11.0.0
Note: The bitcoin upgrade includes improvements to the
`Transaction::weight()` function, it appears those guys did good, we
no longer need to add the 2 additional weight units "just in case".
That accepts a `u64` as param representing the latest timestamp
and internally calls `insert_seen_at` for all transactions in
graph that aren't yet anchored in a confirmed block.
I don't think this was ever used. The only possible usecase I can think
of is for tests, but I don't think that is a strong enough incentive for
us to keep this.
These methods allow us to query for checkpoints contained within the
linked list by height and height range. This is useful to determine
checkpoints to fetch for chain sources without having to refer back to
the `LocalChain`.
Currently this is not implemented efficiently, but in the future, we
will change `CheckPoint` to use a skip list structure.
Wrapping transactions as `Arc<Transaction>` allows us to share
transactions cheaply between the chain-source and receiving structures.
Therefore the chain-source can keep already-fetched transactions (save
bandwidth) and have a shared pointer to the transactions (save memory).
This is better than the current way we do things, which is to refer back
to the receiving structures mid-sync.
Documentation for `TxGraph` is also updated.
b290b29502 test(chain): change test case comments to docstring (志宇)
c151d8fd23 fix(chain): `KeychainTxOutIndex::lookahead_to_target` (志宇)
Pull request description:
### Description
This method was not used (so it was untested) and it was not working. This fixes it.
The old implementation used `.next_store_index` which returned the keychain's last index stored in `.inner` (which include lookahead spks). This is WRONG because `.replenish_lookahead` needs the difference from last revealed.
### Changelog notice
Fix `KeychainTxOutIndex::lookahead_to_target`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [x] I've added tests to reproduce the issue which are now passing
ACKs for top commit:
notmandatory:
ACK b290b29502
Tree-SHA512: af50c6af18b6b57494cfa37f89b0236674fa331091d791e858f67b7d0b3a1e4e11e7422029bd6a2dc1c795914cdf6d592a14b42a62ca7c7c475ba6ed37182539
5489f905a4 feat(chain): add `map_anchors` for `TxGraph` and `ChangeSet` (Antonio Yang)
022d5a21cf test(chain) use `Anchor` generic on `init_graph` (Antonio Yang)
Pull request description:
### Description
Fix#1295
### Checklists
#### All Submissions:
* [X] I've signed all my commits
* [X] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [X] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
ACKs for top commit:
evanlinjin:
ACK 5489f905a4
LLFourn:
ACK 5489f905a4
Tree-SHA512: c8327f2e7035a46208eb32c6da1f9f0bc3e8625168450c5b0b39f695268e42b0b9053b6eb97805b116328195d77af7ca9edb1f12206c50513fbe295dded542e7
13ab5a835d chore(chain): Improve TxGraph::ChangeSet docs (LLFourn)
dbbd514242 fix(chain)!: rm duplicate `is_empty` method in tx graph changeset (志宇)
ae00e1ee7b fix(chain): tx_graph::ChangeSet::is_empty (LLFourn)
Pull request description:
🙈
### Changelog notice
- Fix bug in `tx_graph::ChangeSet::is_empty` where is returns true even when it wasn't empty
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
ACKs for top commit:
LLFourn:
Self-ACK: 13ab5a835d
evanlinjin:
ACK 13ab5a835d
Tree-SHA512: b9f1f17fd2ed0f8e2337a8033e1cbd3e9f15b1ad4b32da3f0eb73a30913d6798e7a08d6b297d93bd08c2e1c388226e97648650ac636846b2c7aa95c3bcefbcfd
e687c27096 fix(chain): convert to `ConfirmationTime` fix (志宇)
Pull request description:
### Description
We forgot to include the `last_seen` when converting from `ChainPosition` to `ConfirmationTime`.
### Notes to the reviewers
No brainer.
### Changelog notice
* Fix `ConfirmationTime` conversion from `ChainPosition` to include the `last_seen`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
LLFourn:
ACK e687c27096
Tree-SHA512: 182f804309c7ea7f2a42911bc00d836ef48a8617c156b2275d6908b07f619b2466039f54728dd3ca2552f4cd11528a8733618f4ce6a4c88d7e88a05a3d82ffaf
8305e64849 feat(chain): expose `SpkIterator::new_with_range` (志宇)
Pull request description:
### Description
This method is useful, I'm not sure why it is not public.
I've also updated the docs and tests.
### Changelog notice
Added
* Expose `SpkIterator::new_with_range`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
danielabrozzoni:
utACK 8305e64849
LLFourn:
ACK 8305e64849
Tree-SHA512: 26389bf65bea74742ebb4d170f3d21aeb02e5be1b3fd20ef16eb318393145809d2b21b96767ebb42acfd8dd789a82bb2d7b29ded24249dabf06bd1e6d23cf0cf
b6a58d4f9b doc(chain,esplora): minor documentation improvements (vmammal)
Pull request description:
* Corrects an awkward use of the word 'an'
* Adds missing backticks, brackets in a few places
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
danielabrozzoni:
ACK b6a58d4f9b
Tree-SHA512: 42f12f218a9849b6fc1c8da28c4d41ca86fcc9ea52311d28f67fadbec030b6cb6986774d89a30e7a2b5ee569b08c7cfa2af71bbb9b06efacfa685dbe95820f73
e6433fb2c1 feat(persist): Add stage_and_commit to Persist (LLFourn)
0bee46e75b fix(store): Remove lifetime (LLFourn)
Pull request description:
Remove gratuitous use of lifetimes in the main persistence struct `Store`. Having lifetimes on this means that you have to keep the magic bytes alive longer than the database which is particularly offensive if you have to send the database to another thread. On top of that the bytes aren't even read.
ACKs for top commit:
evanlinjin:
ACK e6433fb2c1
Tree-SHA512: 7f6d9d60951a8ceaee30719d0771e15632c6fad0702294af15409c5df492669a07299874ef5ee34e3d75bdecbbd41df29bced3ff16b2360d5d5c7687ef677ffc
a4f28c079e chore: improve LocalChain::apply_header_connected_to doc (LLFourn)
8ec65f0b8e feat(example): add RPC wallet example (Vladimir Fomene)
a7d01dc39a feat(chain)!: make `IndexedTxGraph::apply_block_relevant` more efficient (志宇)
e0512acf94 feat(bitcoind_rpc)!: emissions include checkpoint and connected_to data (志宇)
8f2d4d9d40 test(chain): `LocalChain` test for update that is shorter than original (志宇)
9467cad55d feat(wallet): introduce block-by-block api (Vladimir Fomene)
d3e5095df1 feat(chain): add `apply_header..` methods to `LocalChain` (志宇)
2b61a122ff feat(chain): add `CheckPoint::from_block_ids` convenience method (志宇)
Pull request description:
### Description
Introduce block-by-block API for `bdk::Wallet`. A `wallet_rpc` example is added to demonstrate syncing `bdk::Wallet` with the `bdk_bitcoind_rpc` chain-source crate.
The API of `bdk_bitcoind_rpc::Emitter` is changed so the receiver knows how to connect to the block emitted.
### Notes to the reviewers
### Changelog notice
Added
* `Wallet` methods to apply full blocks (`apply_block` and `apply_block_connected_to`) and a method to apply a batch of unconfirmed transactions (`apply_unconfirmed_txs`).
* `CheckPoint::from_block_ids` convenience method.
* `LocalChain` methods to apply a block header (`apply_header` and `apply_header_connected_to`).
* Test to show that `LocalChain` can apply updates that are shorter than original. This will happen during reorgs if we sync wallet with `bdk_bitcoind_rpc::Emitter`.
Fixed
* `InsertTxError` now implements `std::error::Error`.
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
LLFourn:
self-ACK: a4f28c079e
evanlinjin:
ACK a4f28c079e
Tree-SHA512: e39fb65b4e69c0a6748d64eab12913dc9cfe5eb8355ab8fb68f60a37c3bb2e1489ddd8f2f138c6470135344f40e3dc671928f65d303fd41fb63f577b30895b60
In the example_cli we were not always committing (seemingly by mistake).
This then caused all the examples to have to compensate by manually
committing.
71fff1613d feat(chain): add txout methods to `KeychainTxOutIndex` (志宇)
83e7b7ec40 docs(chain): improve `KeychainTxOutIndex` docs (志宇)
9294e30943 docs(wallet): improve docs for unbounded spk iterator methods (志宇)
b74c2e2622 fix(wallet): use efficient peek address logic (志宇)
81aeaba48a feat(chain): add `SpkIterator::descriptor` method (志宇)
c7b47af72f refactor(chain)!: revamp `KeychainTxOutIndex` API (志宇)
705690ee8f feat(chain): make output of `SpkTxOutIndex::unused_spks` cloneable (志宇)
Pull request description:
Closes#1268
### Description
Previously `SpkTxOutIndex` methods can be called from `KeychainTxOutIndex` due to the `DeRef` implementation. However, the internal `SpkTxOut` will also contain lookahead spks resulting in an error-prone API.
`SpkTxOutIndex` methods are now not directly callable from `KeychainTxOutIndex`. Methods of `KeychainTxOutIndex` are renamed for clarity. I.e. methods that return an unbounded spk iter are prefixed with `unbounded`.
In addition to this, I also optimized the peek-address logic of `bdk::Wallet` using the optimized `<SpkIterator as Iterator>::nth` implementation.
### Notes to the reviewers
This is mostly refactoring, but can also be considered a bug-fix (as the API before was very problematic).
### Changelog notice
Changed
* Wallet's peek-address logic is optimized by making use of `<SpkIterator as Iterator>::nth`.
* `KeychainTxOutIndex` API is refactored to better differentiate between methods that return unbounded vs stored spks.
* `KeychainTxOutIndex` no longer directly exposes `SpkTxOutIndex` methods via `DeRef`. This was problematic because `SpkTxOutIndex` also contains lookahead spks which we want to hide.
Added
* `SpkIterator::descriptor` method which gets a reference to the internal descriptor.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
~* [ ] I've added tests for the new feature~
* [x] I've added docs for the new feature
#### Bugfixes:
* [x] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
danielabrozzoni:
ACK 71fff1613d
Tree-SHA512: f29c7d2311d0e81c4fe29b8f57c219c24db958194fad5de82bb6d42d562d37fd5d152be7ee03a3f00843be5760569ad29b848250267a548d7d15320fd5292a8f
