5840ce473e430de4c4e3698734e9667cc476fee4 fix(bdk): Remove extra taproot fields when finalizing Psbt (vmammal)
8c78a42163dee06b640f46d74255df37dbc53873 test(psbt): Fixup test_psbt_multiple_internalkey_signers (vmammal)
Pull request description:
We currently allow removing `partial_sigs` from a finalized PSBT, which is relevant to non-taproot inputs, however taproot related PSBT fields were left in place despite the recommendation of BIP371 to remove them once the `final_script_witness` is constructed. This can cause confusion for parsers that encounter extra taproot metadata in an already satisfied input.
Fix this by introducing a new member to SignOptions `remove_taproot_extras`, which when true will remove extra taproot related data from a PSBT upon successful finalization. This change makes removal of all taproot extras the default but configurable.
fixes#1243
### Notes to the reviewers
If there's a better or more descriptive name for `remove_taproot_extras`, I'm open to changing it.
### Changelog notice
Fixed an [issue](https://github.com/bitcoindevkit/bdk/issues/1243) finalizing taproot inputs following BIP371
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
<!-- * [ ] This pull request breaks the existing API -->
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
evanlinjin:
ACK 5840ce473e430de4c4e3698734e9667cc476fee4
Tree-SHA512: 69f022c6f736500590e36880dd2c845321464f89af6a2c67987d1c520f70a298328363cade0f55f270c4e7169e740bd4ada752ec2c75afa02b6a5a851f9030c3
b290b2950221578d73c3194a1accfd080f83631a test(chain): change test case comments to docstring (志宇)
c151d8fd230fb87f27aadb965157f94f01f021a2 fix(chain): `KeychainTxOutIndex::lookahead_to_target` (志宇)
Pull request description:
### Description
This method was not used (so it was untested) and it was not working. This fixes it.
The old implementation used `.next_store_index` which returned the keychain's last index stored in `.inner` (which include lookahead spks). This is WRONG because `.replenish_lookahead` needs the difference from last revealed.
### Changelog notice
Fix `KeychainTxOutIndex::lookahead_to_target`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [x] I've added tests to reproduce the issue which are now passing
ACKs for top commit:
notmandatory:
ACK b290b2950221578d73c3194a1accfd080f83631a
Tree-SHA512: af50c6af18b6b57494cfa37f89b0236674fa331091d791e858f67b7d0b3a1e4e11e7422029bd6a2dc1c795914cdf6d592a14b42a62ca7c7c475ba6ed37182539
We currently allow removing `partial_sigs` from a finalized Psbt,
which is relevant to non-taproot inputs, however taproot related Psbt
fields were left in place despite the recommendation of BIP371 to remove
them once the `final_script_witness` is constructed. This can cause
confusion for parsers that encounter extra taproot metadata in an
already satisfied input.
Fix this by introducing a new member to SignOptions
`remove_taproot_extras`, which when true will remove extra taproot
related data from a Psbt upon successful finalization. This change
makes removal of all taproot extras the default but configurable.
test(wallet): Add test
`test_taproot_remove_tapfields_after_finalize_sign_option`
that checks various fields have been cleared for taproot
Psbt `Input`s and `Output`s according to BIP371.
to verify the signature of the input and ensure the right internal
key is used to sign. This fixes a shortcoming of a previous
version of the test that relied on the result of `finalize_psbt`
but would have erroneously allowed signing with the wrong key.
5489f905a434ecc06867603c7c421e3e50d993ca feat(chain): add `map_anchors` for `TxGraph` and `ChangeSet` (Antonio Yang)
022d5a21cff6c46fb869f8fd538b4026e531ee57 test(chain) use `Anchor` generic on `init_graph` (Antonio Yang)
Pull request description:
### Description
Fix#1295
### Checklists
#### All Submissions:
* [X] I've signed all my commits
* [X] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [X] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [ ] I've added tests for the new feature
* [ ] I've added docs for the new feature
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [ ] I've added tests to reproduce the issue which are now passing
* [ ] I'm linking the issue being fixed by this PR
ACKs for top commit:
evanlinjin:
ACK 5489f905a434ecc06867603c7c421e3e50d993ca
LLFourn:
ACK 5489f905a434ecc06867603c7c421e3e50d993ca
Tree-SHA512: c8327f2e7035a46208eb32c6da1f9f0bc3e8625168450c5b0b39f695268e42b0b9053b6eb97805b116328195d77af7ca9edb1f12206c50513fbe295dded542e7
13ab5a835d59341e387fbbefec12fe4f48e5f3c8 chore(chain): Improve TxGraph::ChangeSet docs (LLFourn)
dbbd51424284a960a95c3801b02dc1973322725f fix(chain)!: rm duplicate `is_empty` method in tx graph changeset (志宇)
ae00e1ee7b95c622fc3992c0804dd4b4517efc7b fix(chain): tx_graph::ChangeSet::is_empty (LLFourn)
Pull request description:
🙈
### Changelog notice
- Fix bug in `tx_graph::ChangeSet::is_empty` where is returns true even when it wasn't empty
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [ ] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
ACKs for top commit:
LLFourn:
Self-ACK: 13ab5a835d59341e387fbbefec12fe4f48e5f3c8
evanlinjin:
ACK 13ab5a835d59341e387fbbefec12fe4f48e5f3c8
Tree-SHA512: b9f1f17fd2ed0f8e2337a8033e1cbd3e9f15b1ad4b32da3f0eb73a30913d6798e7a08d6b297d93bd08c2e1c388226e97648650ac636846b2c7aa95c3bcefbcfd
552f11cb5f66a9366dbc05c801fcae043ebe5b82 feat(esplora): include previous `TxOut`s for fee calculation The previous `TxOut` for transactions received from an external wallet are added as floating `TxOut`s to `TxGraph` to allow for fee calculation. (Wei Chen)
Pull request description:
### Description
Partially implements #1265.
The previous `TxOut` for transactions received from an external wallet are added as floating `TxOut`s to `TxGraph` to allow for fee calculation.
### Notes to the reviewers
Currently only the `esplora` portion of #1265 has been implemented.
The `electrum` portion will potentially be done in a new PR, as discussed on the 1/30/24 Lib call.
### Checklists
#### To Do:
* [ ] Implement `electrum` portion of #1265.
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
evanlinjin:
re-ACK 552f11cb5f66a9366dbc05c801fcae043ebe5b82
danielabrozzoni:
ACK 552f11cb5f66a9366dbc05c801fcae043ebe5b82
Tree-SHA512: 752a24ebd0b9ad7952c1b093ecb251473e346c77b860c1a80c73418130189227405a0f0d7652967cf8c7b89994e8c37df96cd52b52b6daff9cc8c88b5194069a
9bb39a3a3fcdebeec53a8a771091954c845db7a5 Avoid a wildcard match in tx construction (Steven Roose)
9e098a5b6d0802eb39f3e200605b8576c77030eb tx_builder: Support setting explicit nSequence for foreign inputs (Steven Roose)
Pull request description:
Fixes https://github.com/bitcoindevkit/bdk/issues/1315.
ACKs for top commit:
evanlinjin:
ACK 9bb39a3a3fcdebeec53a8a771091954c845db7a5
danielabrozzoni:
utACK 9bb39a3a3fcdebeec53a8a771091954c845db7a5
Tree-SHA512: 42c96a58a762fa8737402ebd0132ce20ce0359c996cee9feeecb0b84e6fb73305be1aec9429fb97ba932486f0b35ac5caed7b43656456c5fb053e55330a12d47
1c15cb2f9169bb08a4128c183eeeca80fff97ab7 ref(example_cli): Add new struct Init (vmammal)
89a7ddca7f6ed8c65ed0124774f3a868d08faf68 ref(esplora): `Box` a large `esplora_client::Error` (vmammal)
097d818d4c7fff12d72ce0ef10ceb9b575c154e1 ref(wallet): `Wallet::preselect_utxos` now accepts a `&TxParams` (vmammal)
f11d663b7efb98dd72fed903ade8c5e7af0b5a3a ref(psbt): refactor body of `get_utxo_for` to address `clippy::manual_map` (vmammal)
4679ca1df7209d2b356fbb8cbd675f07756d1301 ref(example_cli): add typedefs to reduce type complexity (vmammal)
64a90192d9d49c3df22d2b5f489ba20222fd941a refactor: remove old clippy allow attributes (vmammal)
Pull request description:
closes#1127
There are several instances in the code where we allow clippy lints that would otherwise be flagged during regular checks. It would be preferable to minimize the number of "clippy allow" attributes either by fixing the affected areas or setting a specific configuration in `clippy.toml`. In cases where we have to allow a particular lint, it should be documented why the lint doesn't apply.
For context see https://github.com/bitcoindevkit/bdk/issues/1127#issuecomment-1784256647 as well as the commit message details.
One area I'm unsure of is whether `Box`ing a large error in 4fc2216 is the right approach. Logically it makes sense to avoid allocating a needlessly heavy `Result`, but I haven't studied the implications or tradeoffs of such a change.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK 1c15cb2f9169bb08a4128c183eeeca80fff97ab7
Tree-SHA512: 5fa3796a33678651414e7aad7ef8309b4cbe2a9ab00dce094964b40784edb2f46a44067785d95ea26f4cd88d593420485be94c9b09ac589f632453fbd8c94d85
These lints either resolved themselves, or the code has changed such that
they no longer apply, hence they can be removed with no further changes.
`clippy::derivable_impls`
`clippy::needless_collect`
`clippy::almost_swapped`
c5afbaa95d7bc699981d6fe7f9964a41b7587d5c ci: update zstd-sys version to work with MSRV 1.63 (Steve Myers)
929b5ddb0c2e0dc7c346c08888201ac54efab84d refactor(esplora): better variable naming and docs (志宇)
216648bcfd2c622890bf7f50fb2eaa4396c42ed1 chore(esplora): Clarify consistency guarantees (LLFourn)
63fa710319f231f7f28d92cb49b0399f3250c9da fix(esplora): reuse returned height instead of zipping (志宇)
6f824cf325c6c882b00742d814be1d72602f05a6 test(esplora): introduce test cases for `update_local_chain` (志宇)
f05e8502e69a051af9869be6ba5ad9a376b25fc2 feat(esplora): greatly simplify `update_local_chain` (志宇)
Pull request description:
Fixes#1199
### Description
After a second look at the `update_local_chain` implementations, it was clear that they were over complicated. This PR simplifies the `EsploraExt::update_local_chain` method(s) of the `bdk_esplora` crate and adds a whole bunch of tests.
### Notes to the reviewers
The description of #1199 is very brief, however @danielabrozzoni mentioned about potentially-problematic logic with `ASSUME_FINAL_DEPTH`. The logic was indeed convoluted and it did not need to be that way. This PR removes the need for `ASSUME_FINAL_DEPTH`.
### Changelog notice
Fixed
- Simplified `EsploraExt::update_local_chain` logic.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
~* [ ] This pull request breaks the existing API~
~* [ ] I've added tests to reproduce the issue which are now passing~ (there are now lots of tests though)
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
evanlinjin:
ACK c5afbaa95d7bc699981d6fe7f9964a41b7587d5c
notmandatory:
utACK c5afbaa95d7bc699981d6fe7f9964a41b7587d5c
Tree-SHA512: f588493a4643f0f68d3181f27adf91793d4e336be6e853a26289e0916ed83169e1067260b75e627f190842f691ec095e668c0b799ca80e7da2849dd28de32754
5299db34cb9117ad1b66a6afcb51f6ca7e1f0d95 fix(wallet): filter duplicates before coin selection (志宇)
d9501187ef3c4436b7baedb23a4d4f6b1f46a03d test(wallet): fix tests helpers to generate unique utxos (志宇)
Pull request description:
Fixes#1240
### Description
We now filter out duplicate coins before calling `CoinSelectionAlgorithm::coin_select`. If a UTXO exists in both `required_utxos` and `optional_utxos`, only the copy in `required_utxos` will be kept.
Test helper methods are also updated to not create duplicate UTXOs.
### Changelog notice
Fixed
* Filter out duplicate UTXOs before calling `CoinSelectionAlgorithm::coin_select`. If a UTXO exists in both `required_utxos` and `optional_utxos`, only the copy in `required_utxos` will be kept.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
~* [ ] This pull request breaks the existing API~
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
danielabrozzoni:
utACK 5299db34cb9117ad1b66a6afcb51f6ca7e1f0d95
Tree-SHA512: 9cb5517b7f74f89c06172efc344766b16b3216a25b1ebdd6eb84767a9e103124cead9eb0a7f3b5feb562fbb925517a9bf0404399de74b4e898982a5b3795aa04
3829fc18c7f2d16d2c72852bde4b4b70bff8e370 doc(store): update doc for `Store::aggregate_changesets` (vmammal)
Pull request description:
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
evanlinjin:
ACK 3829fc18c7f2d16d2c72852bde4b4b70bff8e370
Tree-SHA512: 55e3dea6db4e023edcd6f12e3c2e8f63c79654b178e01e2fb8b66f3a66399aabedb1d159cd015b78bc2166be6e640e0ed684e3571f2c4f2f6a53fa68d2ef9849
5611c9e42adb4106f48039e19ab0797b266c7fff fix(file_store): rm lifetime from `FileError` (志宇)
Pull request description:
### Description
The lifetime on the error needed to be the same as the input magic bytes which was annoying.
### Changelog notice
Fixed
* Remove `bdk_file_store::FileError` lifetime.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
LLFourn:
ACK 5611c9e42adb4106f48039e19ab0797b266c7fff
Tree-SHA512: 0eca5128c71c9adb10a712d169a2170b8cbb91678f93986957e86b26dbfed3ab19217a136b57328249a13a51cc06e6c424d58683d74a0fa3d29c22af04a95591
e687c27096e1c8f36c3a129dc6d92af9c5d05a30 fix(chain): convert to `ConfirmationTime` fix (志宇)
Pull request description:
### Description
We forgot to include the `last_seen` when converting from `ChainPosition` to `ConfirmationTime`.
### Notes to the reviewers
No brainer.
### Changelog notice
* Fix `ConfirmationTime` conversion from `ChainPosition` to include the `last_seen`.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
ACKs for top commit:
LLFourn:
ACK e687c27096e1c8f36c3a129dc6d92af9c5d05a30
Tree-SHA512: 182f804309c7ea7f2a42911bc00d836ef48a8617c156b2275d6908b07f619b2466039f54728dd3ca2552f4cd11528a8733618f4ce6a4c88d7e88a05a3d82ffaf
51bd01b3dd3d68c1628d74be9dc6d0f1cf1f15fe fix(file_store): recover file offset after read (志宇)
66dc34e75ab1ec2ae533bd540327721f6226eca1 refactor(file_store): Use BufReader but simplify (LLFourn)
c8717646700bdac0f1e13ec499481bd1fee30ffd test(file_store): `last_write_is_short` (志宇)
a3aa8b6682a3a13958fd5fbadc4074a1907a78db feat(file_store)!: optimize `EntryIter` by reducing syscalls (志宇)
Pull request description:
### Description
`EntryIter` performance is improved by reducing syscalls. The underlying file reader is wrapped with `BufReader` (to reduce calls to `read` and `seek`).
Two new tests are introduced. One ensures correct behavior when the last changeset write is too short. The other ensures the next write position is correct after a short read.
### Notes to the reviewers
This is extracted from #1172 as suggested by https://github.com/bitcoindevkit/bdk/pull/1172#pullrequestreview-1817465627.
### Changelog notice
Changed
* `EntryIter` performance is improved by reducing syscalls.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
LLFourn:
ACK 51bd01b3dd3d68c1628d74be9dc6d0f1cf1f15fe
Tree-SHA512: 9c25f9f2032cb2d551f3fe4ac62b856ceeb69a388f037b34674af366c55629a2eaa2b90b1ae4fbd425415ea8d02f44493a6c643b4b1a57f4507e87aa7ade3736
8305e648494cbdd1f6438904ec367f8442fba606 feat(chain): expose `SpkIterator::new_with_range` (志宇)
Pull request description:
### Description
This method is useful, I'm not sure why it is not public.
I've also updated the docs and tests.
### Changelog notice
Added
* Expose `SpkIterator::new_with_range`
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
danielabrozzoni:
utACK 8305e648494cbdd1f6438904ec367f8442fba606
LLFourn:
ACK 8305e648494cbdd1f6438904ec367f8442fba606
Tree-SHA512: 26389bf65bea74742ebb4d170f3d21aeb02e5be1b3fd20ef16eb318393145809d2b21b96767ebb42acfd8dd789a82bb2d7b29ded24249dabf06bd1e6d23cf0cf
