43aed386bc esplora: also test the gap limit bounds in the async client (Antoine Poinsot)
cb713e5b8c esplora: also fix the gap limit check in the async client (Antoine Poinsot)
2c4e90a76f esplora: scan gap limit bounds testing (Antoine Poinsot)
18bd329617 esplora: fix incorrect gap limit check in blocking client (Antoine Poinsot)
Pull request description:
The gap limit was checked such as if the last_index was not None but the last_active_index was, we'd consider having reached it. But the last_index is never None for this check. This effectively made it so the gap limit was always 1: if the first address isn't used last_active_index will be None and we'd return immediately.
Fix this by avoiding error-prone Option comparisons and correctly checking we've reached the gap limit before breaking out of the loop.
ACKs for top commit:
danielabrozzoni:
ACK 43aed386bc
evanlinjin:
ACK 43aed386bc
Tree-SHA512: c6a172e0627380add56aec79e7fe36c0358e092b59f0bea8885d3524e65c10336291943efe6aeb5cc83f90c4f2146ed63215e56e08583d703b6ab57284487f03
`PersistBackend::is_empty` is removed. Instead, `load_from_persistence`
returns an option of the changeset. `None` means persistence is empty.
This is a better API than a separate method. We can now differentiate
between a persisted single changeset and nothing persisted.
`Store::aggregate_changeset` is refactored to return a `Result` instead
of a tuple. A new error type (`AggregateChangesetsError`) is introduced
to include the partially-aggregated changeset in the error. This is a
more idiomatic API.
These methods try to load wallet from persistence and initializes the
wallet instead if non-existant.
An internal helper method `create_signers` is added to reuse code.
Documentation is also improved.
`Wallet::new` now creates a new wallet. `Wallet::load` loads an existing
wallet. The network type is now recoverable from persistence. Error
types have been simplified.
b1461f05d0 fix(wallet_esplora): missing_heights uses the... ...graph update (Daniela Brozzoni)
Pull request description:
Fixes#1151.
When wallet_esplora_* was used to sync a wallet containing a transaction confirmed some time ago (more than 10-15 blocks ago), the transaction would be stuck in an "unconfirmed" state forever.
At the first scan time, `update_local_chain` would just fetch the last 10 to 15 blocks (depending on the server used), and `tx_graph.missing_heights` wouldn't return the tx's confirmation block as it was called on the original, non-updated tx_graph.
So, after the first scan, we would have a transaction in memory with an anchor that doesn't exist in our local_chain, and try_get_chain_position would return unconfirmed.
When scanning again, missing_heights would find the missing anchor, but `update_local_chain` wouldn't include it as it's older than ASSUME_FINAL_DEPTH.
The missing block would be downloaded every time, but never included in the local_chain, and the transaction would remain unconfirmed forever.
Here we call missing_heights on the updated graph, so that it can correctly return the anchor height, and `update_local_chain` can fetch it and include it in the chain.
### Notes to the reviewers
I'm not sure if this is the right approach, so I'm opening this PR to gather feedback. I still need to add tests, I'll do so once we decide if this is the right way to go or not.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [x] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
evanlinjin:
ACK b1461f05d0
Tree-SHA512: ba0cf85929644ee737dbc77e6afec662845532de0f120917aa6000ca8f5db79d0cb3971bd92285b5c5b5d26042b60b6c8536f50c9bd49615e31f5da28e80a509
* avoid holding mutex lock over io
* document `CHANNEL_BOUND` const
* use the `relevant` variant of `batch_insert_unconfirmed`
* print elapsed time in stdout for various updates
* `bdk_chain` dependency is added. In the future, we will introduce a
separate `bdk_core` crate to contain shared types.
* replace `Emitter::new` with `from_height` and `from_checkpoint`
* `from_height` emits from the given start height
* `from_checkpoint` uses the provided cp to find agreement point
* introduce logic that ensures emitted blocks can connect with
receiver's `LocalChain`
* in our rpc example, we can now `expect()` chain updates to always
since we are using checkpoints and receiving blocks in order
Instead of comparing the blockhash against the emitted_blocks map
to see whether the block is part of the emitter's best chain, we
reduce the `last_mempool_tip` height to the last agreement height
during the polling logic.
The benefits of this is we have tighter bounds for avoiding re-
emission. Also, it will be easier to replace `emitted_blocks` to
a `CheckPoint` (since we no longer rely on map lookup).
Instead of inserting anchors and seen_at timestamp in the same method,
we have three separate methods. This makes the API easier to understand
and makes `IndexedTxGraph` more consistent with the `TxGraph` API.
For `IndexedTxGraph`:
- Remove `InsertTxItem` type (this is too complex).
- `batch_insert_relevant` now uses a simple tuple `(&tx, anchors)`.
- `batch_insert` is now also removed, as the same functionality can be
done elsewhere.
- Add internal helper method `index_tx_graph_changeset` so we don't need
to create a seprate `TxGraph` update in each method.
- `batch_insert_<relevant>_unconfirmed` no longer takes in an option of
last_seen.
- `batch_insert_unconfirmed` no longer takes a reference of a
transaction (since we apply all transactions anyway, so there is no
need to clone).
For `TxGraph`:
- Add `batch_insert_unconfirmed` method.
* `CheckPoint::from_header` allows us to construct a checkpoint from
block header.
* `CheckPoint::into_update` transforms the cp into a
`local_chain::Update`.
e89cf5a16a refactor: use set_lookahead in set_lookahead_for_all (Vladimir Fomene)
Pull request description:
### Description
Use set_lookahead in set_lookahead_for_all.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
realeinherjar:
ACK: e89cf5a16a
evanlinjin:
ACK e89cf5a16a
Tree-SHA512: 02d226be7adcfd5e23ecb9d17539b0e089cb55ddf9c6de980155960f8181f2d3ea3ac9a93b2c1b7e0a8d4e4c821d92f65405852c696c9a367193a42d60c1aac6
6d601a7e88 test(chain): Add test for conflicting transactions (Daniela Brozzoni)
48ca95b541 test(chain): Add test for walk_ancestors (Daniela Brozzoni)
59a2403e28 test(chain): Introduce TxTemplate (Daniela Brozzoni)
6e511473a5 test(chain): add block_id! utility macro (Daniela Brozzoni)
62de55f12d fix(chain): Consider conflicting ancestors in... ...try_get_chain_pos (Daniela Brozzoni)
a3e8480ad9 doc(chain): Clarify direct_conflicts_of_tx's docs (Daniela Brozzoni)
4742d88ea3 feat(chain): Introduce TxAncestors, walk_ancestors (Daniela Brozzoni)
2f26eca607 fix(chain): TxDescendants performs a BFS (Daniela Brozzoni)
486e0e1437 doc(chain): Fix typos (Daniela Brozzoni)
Pull request description:
<!-- You can erase any parts of this template not applicable to your Pull Request. -->
### Description
Fixes#1063.
This PR introduces a new `TxTemplate` struct to test different transaction conflict scenarios in `TxGraph`.
The following transaction conflict scenarios are tested:
- 2 unconfirmed txs with different last_seens conflict. The most recent tx should be the only tx that appears in the list methods.
- 3 unconfirmed txs with different last_seens conflict. The most recent tx should be the only tx that appears in the list methods.
- An unconfirmed tx U conflicts with a tx anchored in orphaned block O. O has higher last_seen. O should be the only tx that appears in the list methods.
- An unconfirmed tx U conflicts with a tx anchored in orphaned block O. U has higher last_seen. U should be the only tx that appears in the list methods.
- Multiple unconfirmed txs conflict with a confirmed tx. None of the unconfirmed txs should appear in the list methods.
- B and B' conflict. C spends B. B' is anchored in best chain. B and C should not appear in the list methods.
- B and B' conflict. C spends B. B is anchored in best chain. B' should not appear in the list methods.
- B and B' conflict. C spends both B and B'. C is impossible.
- B and B' conflict. C spends both B and B'. C is impossible. B' is confirmed.
- B and B' conflict. C spends both B and B'. C is impossible. D spends C.
These tests revealed that `TxGraph::walk_conflicts` was not checking ancestors of the root tx for conflicts. `TxGraph::walk_conflicts` has been refactored to check for conflicting ancestor transactions by using a new `TxAncestors` iterator in `TxGraph`.
### Changelog notice
- Introduced `tx_template` module
- Introduced `TxGraph::TxAncestors` iterator
- Refactored `TxGraph::walk_conflicts` to use `TxGraph::TxAncestors`
- Added `walk_ancestors` to `TxGraph`
### Checklists
All Submissions:
- [x] I've signed all my commits
- [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
- [x] I ran cargo fmt and cargo clippy before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
evanlinjin:
ACK 6d601a7e88
Tree-SHA512: ea151392874c4312233e4e10299579f4eee4a7100ae344b4d7f19994284b49c1e43f37338bed931d16e77326021166ea0b94d6de3ccf50a8fabb25139a8e69b4
e48b911c8d refactor: Make test errors more readable (Daniela Brozzoni)
a7a1d9b2fb fix: non-wildcard descriptors should return an.. ..spk only if index is equal to 0 (Daniela Brozzoni)
cc1a43c495 fix: SpkIterator::new_with_range takes wildcards.. ..into account (Daniela Brozzoni)
Pull request description:
### Description
When you pass a non-wildcard descriptor in `new_with_range`, we make
sure that the range length is at most 1; if that's not the case, we
shorten it.
We would previously use `new_with_range` without this check and with a
non-wildcard descriptor in `spks_of_all_keychains`, this meant creating
a spkiterator that would go on producing the same spks over and over
again, causing some issues with syncing on electrum/esplora.
To reproduce the bug, run in `example-crates/example_electrum`:
```
cargo run "sh(wsh(or_d(c:pk_k(cPGudvRLDSgeV4hH9NUofLvYxYBSRjju3cpiXmBg9K8G9k1ikCMp),c:pk_k(cSBSBHRrzqSXFmrBhLkZMzQB9q4P9MnAq92v8d9a5UveBc9sLX32))))#zp9pcfs9" scan
```
### Changelog notice
- Fixed a bug where `KeychainTxOutIndex::spks_of_all_keychains`/`KeychainTxOutIndex::spks_of_keychain` would return an iterator yielding infinite spks even for non-wildcard descriptors.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK e48b911c8d
Tree-SHA512: 87627505049eadcec979a05888ec0d8a25c4743c03696a7db68348d457c2bf006d9b3b69c99e208f7812fc5b0234dd5a98b4a923c2486615c7678c3ab523f8cf
5fb5061645 ci: fix msrv dependency versions for rustls (Steve Myers)
dd5b8d7599 test(wallet): add check_fee!(wallet,psbt) macro and use it in place of psbt.fee_amount() (Steve Myers)
465d53cc88 docs(wallet): update docs for calculate_fee/fee_rate and add_foreign_utxo (Steve Myers)
036299803f feat(wallet): add Wallet::insert_txout function and updated docs for fee functions (Steve Myers)
d443fe7f66 feat(tx_graph)!: change TxGraph::calculate_fee to return Result<u64,CalculateFeeError> (Steve Myers)
b4c31cd5ba feat(wallet)!: remove TransactionDetails from bdk::Wallet API (Steve Myers)
Pull request description:
### Description
Removed `TransactionDetails` and changed `Wallet::get_tx` to return a `CanonicalTx`, and `TxBuilder::finish` to return only a `PartiallySignedTransaction`. This should fix#922 and fix#1015.
I also added `Wallet` functions to get a `Transaction` send and receive amounts, fee, and `FeeRate`.
see: https://github.com/bitcoindevkit/bdk/issues/922#issuecomment-1652814975
### Notes to the reviewers
Alot of wallet tests had to change since `TxBuilder::finish` only returns a PSBT now.
I added a new `CalculateFeeError` which follows changes coming in #1028.
### Changelog notice
Added
- Wallet::sent_and_received function
- Wallet::calculate_fee and Wallet::calculate_fee_rate functions
- Wallet::error::CalculateFeeError
- Wallet::insert_txout function to allow inserting foreign TxOuts
BREAKING CHANGES:
Removed
- TransactionDetails struct
Changed
- Wallet::get_tx now returns CanonicalTx instead of TransactionDetails
- TxBuilder::finish now returns only a PartiallySignedTransaction
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
evanlinjin:
ACK 5fb5061645
Tree-SHA512: 1a0be1c229b8871e5ee23ba6874ff40370170477a0a8bb104c0197e7fd97765d84854285f863dd1b38a34c3b71815e75e4db5b25288c81caea99a14ddaa78254
18e8da3937 docs: Add doctest for Anchor implementation on BlockId (Vladimir Fomene)
480c2730de feat: Implement Anchor for BlockId (Vladimir Fomene)
Pull request description:
### Description
This PR fixes#1062.
It implements the `Anchor` trait on `BlockId`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
evanlinjin:
ACK 18e8da3937
Tree-SHA512: af9da8b4e9c7f2a110340efa4e4c86d9b96ea1275f0a66f08e26a47fcacd89e318a32c845e2a4302f3332eeb4c6a976c3771b1d9881adf4feda90d6ba1a2dd9d
2263a58448 refactor: Rename `wallet_esplora` to... ...`wallet_esplora_blocking` (Daniela Brozzoni)
11ac26f6b2 docs: Update README.md (Daniela Brozzoni)
Pull request description:
### Description
- Fixes the release timeline
- Adds an explanation for the crates under `example-crates`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
notmandatory:
ACK 2263a58448
evanlinjin:
ACK 2263a58448
Tree-SHA512: fd0eecb4a1051b8f88ae858ad6231b254beaad50f925d656a307c66bd6a20770cf698032d1d7e0179d93ac169cfa5d8c7c3b8def4c91334afc27f7a6f9d7558a
62f253103c ci: Update tokio version (Daniela Brozzoni)
80f5ecf3be feat(chain): Add `initial_changeset` to graphs (Daniela Brozzoni)
ea50b6a932 refactor!(chain): Unify ChangeSet nomenclature (Daniela Brozzoni)
Pull request description:
Solves #1022
### Changelog notice
Rename `indexed_tx_graph::IndexedAdditions` to `indexed_tx_graph::ChangeSet`
Rename `indexed_tx_graph::IndexedAdditions::graph_additions` to `indexed_tx_graph::ChangeSet::graph`
Rename `indexed_tx_graph::IndexedAdditions::index_additions` to `indexed_tx_graph::ChangeSet::indexer`
Rename `tx_graph::Additions` to `tx_graph::ChangeSet`
Rename `keychain::DerivationAdditions` to `keychain::ChangeSet`
Rename `CanonicalTx::node` to `CanonicalTx::tx_node`
Rename `CanonicalTx::observed_as` to `CanonicalTx::chain_position`
Rename `LocalChangeSet` to `WalletChangeSet`
Rename `LocalChangeSet::chain_changeset` to `WalletChangeSet::chain`
Rename `LocalChangeSet::indexed_additions` to `WalletChangeSet::indexed_tx_graph`
Rename `LocalUpdate` to `WalletUpdate`
Make `TxGraph::determine_changeset` `pub(crate)`
Add `TxGraph::initial_changeset`
Add `IndexedTxGraph::initial_changeset`
Remove `TxGraph::insert_txout_preview`
Remove `TxGraph::insert_tx_preview`
Remove `insert_anchor_preview`
Remove `insert_seen_at_preview`
### Notes to reviewers
I'm not sure whether we want to keep `TxGraph::determine_changeset` public or not
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK 62f253103c
Tree-SHA512: 401bdf49c5f2082971cfcd54725ce98abd08263de7571205a324a2adbb5359be4f48454def415bedadde0a9c81db437b91c13cd6e892ba54b215059f75006dd5
1da3b304bb ci: Pin rustls to keep the MSRV (Daniela Brozzoni)
792b39fa92 Explicitly deny multipath keys (Daniela Brozzoni)
b73385dbd2 Update wallet_electrum to rust-bitcoin 0.30.0 (Daniela Brozzoni)
3dac3f9bba Update example_electrum to rust-bitcoin 0.30.0 (Daniela Brozzoni)
2949bdc7b8 Update example_cli to rust-bitcoin 0.30.0 (Daniela Brozzoni)
468d2a0a3b Update tmp_plan to rust-bitcoin 0.30.0 (Daniela Brozzoni)
b8ac16d03c Update coin_select to rust-bitcoin 0.30.0 (Daniela Brozzoni)
6c29e53ee8 Update wallet_esplora and wallet_esplora_async to... ...rust-bitcoin 0.30.0 (Daniela Brozzoni)
6eb079576f Update crates/esplora to rust-bitcoin 0.30.0 (Daniela Brozzoni)
91b0f0ba29 Update crates/electrum to bitcoin 0.30.0 (Daniela Brozzoni)
f4e3ba3265 Update bdk to bitcoin 0.30.0 (Daniela Brozzoni)
853d361751 Update bdk_chain to bitcoin 0.30.0 (Daniela Brozzoni)
Pull request description:
### Description
Updates to rust-bitcoin 0.30.0 and miniscript 0.10.0
Not covered in this PR:
- https://github.com/bitcoindevkit/bdk/issues/1036. Although the latter is deprecated, I think it's better if I update it in a separate PR, as this one is pretty big already.
- https://github.com/bitcoindevkit/bdk/issues/1037
- https://github.com/bitcoindevkit/bdk/issues/1038
Heads up, I'm explicitly denying multipath descriptors until we have better tests for them. See the commit message of 23fba7aee9b55bf59db73c296b9fb025a256502e
### Changelog notice
- Update to `rust-bitcoin` 0.30.0 and `miniscript` 10.0.0
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK 1da3b304bb
Tree-SHA512: ff1457ed711f9f8cdb446ea10aaf124632f539c02406da94317d8dc38013b321217d3bdcb2df4bd72b2ed92116b52e9c6b98ee91d4d508a579c67449a7caa549
b206a985cf fix: Even more refactoring to code and documentation (志宇)
bea8e5aff4 fix: `TxGraph::missing_blocks` logic (志宇)
db15e03bdc fix: improve more docs and more refactoring (志宇)
95312d4d05 fix: docs and some minor refactoring (志宇)
8bf7a997f7 Refactor `debug_assertions` checks for `LocalChain` (志宇)
315e7e0b4b fix: rm duplicate `bdk_tmp_plan` module (志宇)
af705da1a8 Add exclusion of example cli `*.db` files in `.gitignore` (志宇)
eabeb6ccb1 Implement linked-list `LocalChain` and update chain-src crates/examples (志宇)
Pull request description:
Fixes#997
Replaces #1002
### Description
This PR changes the `LocalChain` implementation to have blocks stored as a linked-list. This allows the data-src thread to hold a shared ref to a single checkpoint and have access to the whole history of checkpoints without cloning or keeping a lock on `LocalChain`.
The APIs of `bdk::Wallet`, `esplora` and `electrum` are also updated to reflect these changes. Note that the `esplora` crate is rewritten to anchor txs in the confirmation block (using the esplora API's tx status block_hash). This guarantees 100% consistency between anchor blocks and their transactions (instead of anchoring txs to the latest tip). `ExploraExt` now has separate methods for updating the `TxGraph` and `LocalChain`.
A new method `TxGraph::missing_blocks` is introduced for finding "floating anchors" of a `TxGraph` update (given a chain).
Additional changes:
* `test_local_chain.rs` is refactored to make test cases easier to write. Additional tests are also added.
* Examples are updated.
* Exclude example-cli `*.db` files in `.gitignore`.
* Rm duplicate `bdk_tmp_plan` module.
### Notes to the reviewers
This is the smallest possible division of #1002 without resulting in PRs that do not compile. Since we have changed the API of `LocalChain`, we also need to change `esplora`, `electrum` crates and examples alongside `bdk::Wallet`.
### Changelog notice
* Implement linked-list `LocalChain`. This allows the data-src thread to hold a shared ref to a single checkpoint and have access to the whole history of checkpoints without cloning or keeping a lock on `LocalChain`.
* Rewrote `esplora` chain-src crate to anchor txs to their confirmation blocks (using esplora API's tx-status `block_hash`).
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
LLFourn:
ACK b206a985cf
Tree-SHA512: a513eecb4f1aae6a5c06a69854e4492961424312a75a42d74377d363b364e3d52415bc81b4aa3fbc3f369ded19bddd07ab895130ebba288e8a43e9d6186e9fcc
Thank you @vladimirfomene for these suggestions.
* Rename `LocalUpdate::keychain` to `LocalUpdate::last_active_indices`.
* Change docs for `CheckPoint` to be more descriptive.
* Fix incorrect logic in `update_local_chain` for `EsploraExt` and
`EsploraAsyncExt`.
Shout out to @LLFourn for these suggestions.
* Improve/fix `LocalChain` documentation
* Refactor `TxGraph::missing_blocks` to make it more explicit that
`last_block` has state.
* `update_local_chain` method of `EsploraExt` and `EsploraAsyncExt` now
returns a `local_chain::Update` instead of just a `CheckPoint`.
This commit changes the `LocalChain` implementation to have blocks
stored as a linked-list. This allows the data-src thread to hold a
shared ref to a single checkpoint and have access to the whole history
of checkpoints without cloning or keeping a lock on `LocalChain`.
The APIs of `bdk::Wallet`, `esplora` and `electrum` are also updated to
reflect these changes. Note that the `esplora` crate is rewritten to
anchor txs in the confirmation block (using the esplora API's tx status
block_hash). This guarantees 100% consistency between anchor blocks and
their transactions (instead of anchoring txs to the latest tip).
`ExploraExt` now has separate methods for updating the `TxGraph` and
`LocalChain`.
A new method `TxGraph::missing_blocks` is introduced for finding
"floating anchors" of a `TxGraph` update (given a chain).
Additional changes:
* `test_local_chain.rs` is refactored to make test cases easier to
write. Additional tests are also added.
* Examples are updated.
* Fix `tempfile` dev dependency of `bdk_file_store` to work with MSRV
Co-authored-by: LLFourn <lloyd.fourn@gmail.com>
Other changes:
* The `async-https` feature of `bdk_esplora` is no longer default.
* Rename `ObservedAs` to `ChainPosition`.
* Set temporary MSRV to 1.60.0 to compile all workspace members will all
features.
* `ElectrumUpdate::missing_full_txs` now returns a `Vec<Txid>` so we
don't keep a reference to the passed-in `graph`.
* `ElectrumUpdate::finalize*` methods now takes in `missing` txids
instead of `full_txs`. `Client::batch_transaction_get` is called
within the methods.
Other changes:
* `wallet::ChangeSet` is now made public externally. This is required as
a wallet db should implement `PersistBackend<wallet::ChangeSet>`.
This corresponds to `keychain::KeychainChangeSet` but for the redesigned
structures with `LocalChain`.
This structure is now used in `Wallet` as well as the examples.
There are a number of improvements that can be done, but it is in a
decent state to be usable.
Possible improvements:
* Remove requirement to retry obtaining ALL data after reorg is
detected. Transactions can be anchored to a lower block (not block
tip), and an `assume_final_depth` value can be used.
* The logic to finalize an update with confirmation time can be improved
during reorgs to not require returning an error.
ed89de752c Improve txout filter/listing method docs for `TxGraph` (志宇)
fb75aa94a9 Clarify `TxGraph::try_filter_chain_unspents` logic (志宇)
96b1075132 Fix and improve `Persist::commit` method (志宇)
e01d17d59b Improve `txout` listing and balance APIs for redesigned structures (志宇)
Pull request description:
### Description
As noted in https://github.com/bitcoindevkit/bdk/issues/971#issuecomment-1542408941.
Instead of relying on a `OwnedIndexer` trait to filter for relevant txouts, we move the listing and balance methods from `IndexedTxGraph` to `TxGraph` and add an additional input (list of relevant outpoints) to these methods.
This provides a simpler implementation and a more flexible API.
#### Other Fixes
The `Persist::commit` method is fixed in 96b1075132.
Previously, regardless of whether writing to persistence backend is successful or not, the logic always cleared `self.staged`. This is changed to only clear `self.staged` after successful write.
Additionally, the written changeset (if any) is returned, and `PersistBackend::write_changes` will not be called if `self.staged` is empty.
### Notes to the reviewers
Yes, slightly more boilerplate to do the same things, but less code to maintain and a much more flexible API. Very worth it IMO.
### Changelog notice
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
LLFourn:
ACK ed89de752c
Tree-SHA512: efae18c13ee74eff801febca61cb16bf4d8f3203ced96172b9ef555d8d2a749a382f87d024e8c92aacbab7eb4e50a8337de798e10524291ad80c6b35c4dc0161
Previously, regardless of whether writing to persistence backend is
successful or not, the logic always cleared `self.staged`. This is
changed to only clear `self.staged` after successful write.
Additionally, the written changeset (if any) is returned, and
`PersistBackend::write_changes` will not be called if `self.staged` is
empty.
Instead of relying on a `OwnedIndexer` trait to filter for relevant
txouts, we move the listing and balance methods from `IndexedTxGraph` to
`TxGraph` and add an additional input (list of relevant outpoints) to
these methods.
This provides a simpler implementation and a more flexible API.
4963240599 Add more `impl`s for `Append` and docs for file store `magic` (志宇)
2aa08a5898 [persist_redesign] Introduce redesigned `persist` types (志宇)
Pull request description:
### Description
This is part of #895 and #971
* Introduce a more generic version of the `keychain::persist::*` structures that only needs a single generic for the changeset type.
Additional changes:
* The `Append` trait has a new method `is_empty`.
* Introduce `Store` structure for `bdk_file_store` (which implements `PersistBackend`).
### Changelog notice
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
Top commit has no ACKs.
Tree-SHA512: 0211fbe7d7e27805d3ed3a80b42f184cdff1cebb32fd559aa9838e4a7f7c7e47b6c366b6ef68e299f876bafed549b8d1d8b8cc0366bf5b61db079504a565b9b4
This is a more generic version of `keychain::persist::*` structures.
Additional changes:
* The `Append` trait has a new method `is_empty`.
* Introduce `Store` structure for `bdk_file_store`.
* Introduce `LocalChain::inner` method to get the inner map of block
height to hash.
* Replace `LocalChain::get_block` (which outputted `BlockId`, thus able
to return invalid representation) with `get_blockhash` that just
returns a `BlockHash`.
* Remove `TODO` comments that should be github tickets.
If `BlockId` implements `Anchor`, the meaning is ambiguous. We cannot
tell whether it means the tx is anchors at the block, or whether it also
means the tx is confirmed at that block.
Instead, `ConfirmationHeightAnchor` and `ConfirmationTimeAnchor` structs
are introduced as non-ambiguous `Anchor` implementations.
Additionally, `TxGraph::relevant_heights` is removed because it is also
ambiguous. What heights are deemed relevant? A simpler and more flexible
method `TxGraph::all_anchors` is introduced instead.
`TxGraph::try_get_chain_position` used to always exclude unconfirmed
transactions with last_seen value of 0. However, what is the point of
including a transaction in the graph if it cannot be part of the chain
history? Additionally, maybe sometimes we don't wish to use the
last_seen field at all.
The new behavior will consider unconfirmed transactions with last_seen
of 0.
* `Additions` now implements `Append` and uses `Append` to implement
`append()`.
* `append()` logic enforces that `last_seen` values should only
increase.
* Test written for `append()` with `last_seen` behaviour.
Instead of forcing all transactions inserted to use the same anchors, we
change the API to have unique anchors per transaction.
This allows for more flexibility in general. For example, use `Anchor`
implementations that contain the position in a block of a transaction.
The `insert_relevant_txs` test has also been changed to used
`KeychainTxOutIndex` so that index additions can be checked
(`SpkTxOutIndex` has no additions).
Additionally, generic bounds of some `IndexedTxGraph` list methods have
been fixed.
The `ConfirmationHeight` trait has been removed in favour of a second
method on the `Anchor` trait: `confirmation_height_upper_bound()`.
Methods `try_balance_at()` and `balance_at()` of `IndexedTxGraph` have
been removed as they do not provide additional functionality.
`IndexedTxGraph::insert_relevant_txs` now uses two loops, the first loop
indexes all transactions first. This is done as some indexes require
ancestor transactions to be indexed first and we cannot guarantee that
the input transactions are in topological order.
* Remove `chain_oracle::CacheBackend` for now as it is not used.
* `SparseChain` does not need to implement `ChainOracle`.
* Remove filter predicate for `list..` methods of `TxGraph` and
`IndexedTxGraph` as this is premature optimisation.
* `Append` can be implemented for all `BTreeMap`s and `BTreeSet`s,
instead of only `local_chain::ChangeSet`.
The `HashSet` was used for iterating without duplicate items. However,
since `anchors` is a `BTreeSet`, heights are in order. So a single
variable tracking last height will be sufficient.
志宇
