d3a14d411d fix: enable blocking-https-rustls feature on esplora client (thunderbiscuit)
Pull request description:
The [`blocking` feature on the rust-esplora-client library](https://github.com/bitcoindevkit/rust-esplora-client/blame/master/Cargo.toml#L35) changed from ureq to minreq, which does not come with https enabled by default, breaking previously working code that simply enabled the `blocking` feature on the `bdk_esplora` crate.
This change will enable what is currently the "default https" [flag for the minreq library](https://docs.rs/minreq/latest/minreq/#https-or-https-rustls) when using the `blocking` feature on bdk_esplora, reverting that breaking change.
### Notes to the reviewers
Another way we could do this (let me know if this is preferable) is to add a new feature called `blocking-https-rustls`:
```rust
blocking = ["esplora-client/blocking"]
blocking-https-rustls = ["esplora-client/blocking-https-rustls"]
```
### Changelog notice
<!-- Notice the release manager should include in the release tag message changelog -->
<!-- See https://keepachangelog.com/en/1.0.0/ for examples -->
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
ACKs for top commit:
storopoli:
ACK d3a14d411d
evanlinjin:
ACK d3a14d411d
Tree-SHA512: d25495186ceba2fcd04bc9ff0aebfb32ac5db6885ef8e4df1e304c5ee5264f6161821e06d29367d2837afcc64a53f1553e7c0bb065e6a2e46dc08b8e04c2ad8e
Introduce a new API for getting addresses from the Wallet that
reflects a similiar interface as the underlying indexer
`KeychainTxOutIndex` in preparation for removing `AddressIndex` enum.
Before this change, the only way to get an address was via the methods
`try_get{_internal}_address` which required a `&mut` reference to the
wallet, matching on the desired AddressIndex variant. This is too
restrictive since for example peeking or listing unused addresses
shouldn't change the state of the wallet. Hence we provide separate
methods for each use case which makes for a more efficient API.
e51af49ffa fix(wallet): remove generic from wallet (Rob N)
Pull request description:
### Description
The `PersistenceBackend` uses generics to describe errors returned while applying the change set to the persistence layer. This change removes generics wherever possible and introduces a new public error enum. Removing the generics from `PersistenceBackend` errors is the first step towards #1363
*Update*: I proceeded with removing the generics from `Wallet` by introducing a `Box<dyn PersistenceBackend>` .
### Notes to the reviewers
This one sort of blew up in the number of changes due to the use of generics for most of the `Wallet` error variants. The generics were only used for the persistence errors, so I removed the generics from higher level errors whenever possible. The error variants of `PersistenceBackend` may also be more expressive, but I will level that up for discussion and make any changes required.
### Changelog notice
- Changed `PersistenceBackend` errors to depend on the `anyhow` crate.
- Remove the generic `T` from `Wallet`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [x] I've added docs for the new feature
#### Bugfixes:
* [x] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
evanlinjin:
ACK e51af49ffa
Tree-SHA512: 8ce4f1c495310e16145555f4a6a29a0f42cf8944eda68004595c3532580767f64f779185022147a00d75001c40d69fdf8f8de2d348eb68484b170d2a181117ff
The intention is to remove the `Update::introduce_older_blocks`
parameter and update the local chain directly with `CheckPoint`.
This simplifies the API and there is a way to do this efficiently.
Previously, we would update the `TxGraph` and `KeychainTxOutIndex`
first, then create a second update for `LocalChain`. This required
locking the receiving structures 3 times (instead of twice, which
is optimal).
This PR eliminates this requirement by making use of the new `query`
method of `CheckPoint`.
Examples are also updated to use the new API.
Ensure that `CheckPoint::range` returns expected values by comparing
against values returned from a primitive approach.
I think it is a good idea to start writing proptests for this crate.
a2a64ffb6e fix(electrum)!: Remove `seen_at` param from `into_tx_graph` (valued mammal)
37fca35dde feat(tx_graph): Add method update_last_seen_unconfirmed (valued mammal)
Pull request description:
A method `update_last_seen_unconfirmed` is added for `TxGraph` that allows inserting a `seen_at` time for all unanchored transactions.
fixes#1336
### Changelog notice
Changed:
- Methods `into_tx_graph` and `into_confirmation_time_tx_graph`for `RelevantTxids` are changed to no longer accept a `seen_at` parameter.
Added:
- Added method `update_last_seen_unconfirmed` for `TxGraph`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK a2a64ffb6e
Tree-SHA512: 9011e63314b0e3ffcd50dbf5024f82b365bab1cc834c0455d7410b682338339ed5284caa721ffc267c65fa26d35ff6ee86cde6052e82a6a79768547fbb7a45eb
Upgrade:
- bitcoin to v0.31.0
- miniscript to v11.0.0
Note: The bitcoin upgrade includes improvements to the
`Transaction::weight()` function, it appears those guys did good, we
no longer need to add the 2 additional weight units "just in case".
and `into_confirmation_time_tx_graph`, since now it makes
sense to use `TxGraph::update_last_seen_unconfirmed`.
Also, use `update_last_seen_unconfirmed` in examples for
electrum/esplora. We show how to update the last seen
time for transactions by calling `update_last_seen_unconfirmed`
on the graph update returned from a blockchain source, passing
in the current time, before applying it to another `TxGraph`.
That accepts a `u64` as param representing the latest timestamp
and internally calls `insert_seen_at` for all transactions in
graph that aren't yet anchored in a confirmed block.
I don't think this was ever used. The only possible usecase I can think
of is for tests, but I don't think that is a strong enough incentive for
us to keep this.
These methods allow us to query for checkpoints contained within the
linked list by height and height range. This is useful to determine
checkpoints to fetch for chain sources without having to refer back to
the `LocalChain`.
Currently this is not implemented efficiently, but in the future, we
will change `CheckPoint` to use a skip list structure.
798ed8ced2 fix: remove deprecated `max_satisfaction_weight (Jose Storopoli)
Pull request description:
### Description
Continuation of #1115.
Closes#1036.
* Change deprecated `max_satisfaction_weight` to `max_weight_to_satisfy`
* Remove `#[allow(deprecated)]` flags
### Notes to the reviewers
I've changed all `max_satisfaction_weight()` to `max_weight_to_satisfy()` in `Wallet.get_available_utxo()` and `Wallet.build_fee_bump()`. Checking the docs on the `miniscript` crate for `max_weight_to_satisfy` has the following note:
We are testing if the underlying descriptor `is.segwit()` or `.is_taproot`,
then adding 4WU if true or leaving as it is otherwise.
Another thing, we are not testing in BDK tests for legacy (pre-segwit) descriptors.
Should I also add them to this PR?
### Changelog notice
### Fixed
Replace the deprecated `max_satisfaction_weight` from `rust-miniscript` to `max_weight_to_satisfy`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
evanlinjin:
ACK 798ed8ced2
Tree-SHA512: 60babecee13c24915348ddb64894127a76a59d9421d52ea37acc714913685d57cc2be1904f9d0508078dd1db1f7d7dad83a734af5ee981801ca87de2e9984429
8ab58af093 feat(chain)!: wrap `TxGraph` txs with `Arc` (志宇)
Pull request description:
### Description
This PR makes `TxGraph` store transactions as `Arc<Transaction>`.
`Arc<Transaction>` can be shared between the chain-source and receiving structures. This allows the chain-source to keep the already-fetched transactions (save bandwith) and have a shared pointer to the transaction (save memory).
Our current logic to avoid re-fetching transactions is to refer back to the receiving structures. However, this means an additional round of locking our receiving structures.
### Notes to the reviewers
This will make more sense once I update the esplora/electrum chain sources to make use of both #1369 and this PR. The result would be chain sources which would only require locking the receiving structures twice (once for initiating the update, and once for applying the update).
### Changelog notice
* Changed `TxGraph` to store transactions as `Arc<Transaction>`. This allows chain-sources to cheaply keep a copy of already-fetched transactions.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
LLFourn:
ACK 8ab58af093
notmandatory:
ACK 8ab58af093
Tree-SHA512: 81d7ad35fed7f253a3b902f09a41aaef2877f6201d4f6e78318741bf00e4b898a8000d878ffcbfe75701094ce3e9021bd718b190a655331a9e11f0ad66bdb02f
- changes the code implementation to "the maximum number of consecutive unused addresses"
in esplora async and blocking extjensions.
- treat `stop_gap = 0` as `stop_gap = 1` for all purposes.
- renames `past_gap_limit` to `gap_limit_reached` to indicate we want to break once the gap
limit is reached and not go further in `full_scan`, as suggested in
https://github.com/bitcoindevkit/bdk/issues/1227#issuecomment-1859040463
- change the tests according to the new implementation.
- add notes on what `stop_gap` means and links to convergent definition in other
Bitcoin-related software.
Closes#1227
Wrapping transactions as `Arc<Transaction>` allows us to share
transactions cheaply between the chain-source and receiving structures.
Therefore the chain-source can keep already-fetched transactions (save
bandwidth) and have a shared pointer to the transactions (save memory).
This is better than the current way we do things, which is to refer back
to the receiving structures mid-sync.
Documentation for `TxGraph` is also updated.
7c9ba3cfc8 chore(electrum,testenv): move `test_reorg_is_detected_in_electrsd` (志宇)
2462e90415 chore(esplora): rm custom WASM target spec test dependency (志宇)
04d0ab5a97 test(electrum): added scan and reorg tests Added scan and reorg tests to check electrum functionality using `TestEnv`. (Wei Chen)
4edf533b67 chore: extract `TestEnv` into separate crate `TestEnv` is extracted into its own crate to serve as a framework for testing other block explorer APIs. (Wei Chen)
6e648fd5af chore: update MSRV dependency for nightly docs (Wei Chen)
Pull request description:
### Description
`TestEnv` is extracted into its own crate with `electrsd` support added so that `TestEnv` can also serve `esplora` and `electrum`.
The tests in the `esplora` crate have also been updated to use `TestEnv`.
The `tx_can_become_unconfirmed_after_reorg()` test in `test_electrum` suggests that the electrum issue where a reorged tx would be stuck at confirmed does not exist anymore.
### Notes to the reviewers
The code for `tx_can_become_unconfirmed_after_reorg()` was adapted from the same test in `bitcoind_rpc/test_emitter`. This electrum version of the test requires extra review, as I am uncertain if I used the API efficiently.
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK 7c9ba3cfc8
Tree-SHA512: 36c6501e477abb7ae68073b6f4776f1f69f8964010ab758aa3677aae96df10f2cb632421872cacd73b37123d6db8a9dbefb5b6416e0dd524b712bf3fc56b7139
06d7dc5c3a doc(bdk): Update bdk README (vmammal)
Pull request description:
fixes#1044
### Notes
The code snippet is a compile fail because variables `txout` and `outpoint` are not initialized. Any other suggestions are welcome.
There is still some old commented code in the file that's probably not needed unless we want to provide another example snippet, say for getting new address info.
* [x] Fix broken links
* [x] Use a new unused tprv
* [ ] Try to make persistence example make sense, or maybe just do away with it
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK 06d7dc5c3a
Tree-SHA512: 09a671bc6bea574d7a4b42b64718380ee71a0c5d36c6b7ca1dc19a2c567de510b27ccc91fe05e7178bf31c562db66bc64f660415de5bb2f32f369b13c44ad3d2
5840ce473e fix(bdk): Remove extra taproot fields when finalizing Psbt (vmammal)
8c78a42163 test(psbt): Fixup test_psbt_multiple_internalkey_signers (vmammal)
Pull request description:
We currently allow removing `partial_sigs` from a finalized PSBT, which is relevant to non-taproot inputs, however taproot related PSBT fields were left in place despite the recommendation of BIP371 to remove them once the `final_script_witness` is constructed. This can cause confusion for parsers that encounter extra taproot metadata in an already satisfied input.
Fix this by introducing a new member to SignOptions `remove_taproot_extras`, which when true will remove extra taproot related data from a PSBT upon successful finalization. This change makes removal of all taproot extras the default but configurable.
fixes#1243
### Notes to the reviewers
If there's a better or more descriptive name for `remove_taproot_extras`, I'm open to changing it.
### Changelog notice
Fixed an [issue](https://github.com/bitcoindevkit/bdk/issues/1243) finalizing taproot inputs following BIP371
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
<!-- * [ ] This pull request breaks the existing API -->
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
evanlinjin:
ACK 5840ce473e
Tree-SHA512: 69f022c6f736500590e36880dd2c845321464f89af6a2c67987d1c520f70a298328363cade0f55f270c4e7169e740bd4ada752ec2c75afa02b6a5a851f9030c3
