5c940c33cb1f1a166c6e28e8bbfd9cdaef6c1ab6 Fix wallet sync not finding coins of addresses which are not cached (志宇)
Pull request description:
Fixes #521
Fixes #451
^ However, only for electrum-based `Blockchain` implementations. For RPC and Compact Block Filters, syncing works differently, and so are the bugs - I have created a separate ticket for this (#677).
### Description
Previously, electrum-based blockchain implementations only synced for `scriptPubKey`s that are already cached in `Database`.
This PR introduces a feedback mechanism, that uses `stop_gap` and the difference between "current index" and "last active index" to determine whether we need to cache more `scriptPubKeys`.
The `WalletSync::wallet_setup` trait now may return an `Error::MissingCachedScripts` error which contains the number of extra `scriptPubKey`s to cache, in order to satisfy `stop_gap` for the next call.
`Wallet::sync` now calls `WalletSync` in a loop, caching in-between subsequent calls (if needed).
#### Notes to reviewers
1. The caveat to this solution is that it is not the most efficient. Every call to `WalletSync::wallet_setup` starts polling the Electrum-based server for `scriptPubKey`s starting from index 0.
However, I feel like this solution is the least "destructive" to the API of `Blockchain`. Also, once the `bdk_core` sync logic is integration, we can select specific ranges of `scriptPubKey`s to sync.
2. Also note that this PR only fixes the issue for electrum-based `Blockchain` implementations (currently `blockchain::electrum` and `blockchain::esplora` only).
3. Another thing to note is that, although `Database` assumes 1-2 keychains, the current `WalletSync` "feedback" only returns one number (which is interpreted as the larger "missing count" of the two keychains). This is done for simplicity, and because we are planning to only have one keychain per database in the future.
f0c876e7bf/src/blockchain/mod.rs (L157-L161)
4. Please have a read of https://github.com/bitcoindevkit/bdk/pull/672#issuecomment-1186929465 for additional context.
### Checklists
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### Bugfixes:
* [x] This pull request breaks the existing API
* [x] I've added tests to reproduce the issue which are now passing
* [x] I'm linking the issue being fixed by this PR
ACKs for top commit:
afilini:
ACK 5c940c33cb1f1a166c6e28e8bbfd9cdaef6c1ab6
Tree-SHA512: aee917ed4821438fc0675241432a7994603a09a77d5a72e96bad863e7cdd55a9bc6fbd931ce096fef1153905cf1b786e1d8d932dc19032d549480bcda7c75d1b
BDK

A modern, lightweight, descriptor-based wallet library written in Rust!
Project Homepage | Documentation
About
The bdk
library aims to be the core building block for Bitcoin wallets of any kind.
- It uses Miniscript to support descriptors with generalized conditions. This exact same library can be used to build single-sig wallets, multisigs, timelocked contracts and more.
- It supports multiple blockchain backends and databases, allowing developers to choose exactly what's right for their projects.
- It's built to be cross-platform: the core logic works on desktop, mobile, and even WebAssembly.
- It's very easy to extend: developers can implement customized logic for blockchain backends, databases, signers, coin selection, and more, without having to fork and modify this library.
Examples
Sync the balance of a descriptor
use bdk::Wallet;
use bdk::database::MemoryDatabase;
use bdk::blockchain::ElectrumBlockchain;
use bdk::SyncOptions;
use bdk::electrum_client::Client;
use bdk::bitcoin::Network;
fn main() -> Result<(), bdk::Error> {
let blockchain = ElectrumBlockchain::from(Client::new("ssl://electrum.blockstream.info:60002")?);
let wallet = Wallet::new(
"wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/0/*)",
Some("wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/1/*)"),
Network::Testnet,
MemoryDatabase::default(),
)?;
wallet.sync(&blockchain, SyncOptions::default())?;
println!("Descriptor balance: {} SAT", wallet.get_balance()?);
Ok(())
}
Generate a few addresses
use bdk::{Wallet, database::MemoryDatabase};
use bdk::wallet::AddressIndex::New;
fn main() -> Result<(), bdk::Error> {
let wallet = Wallet::new(
"wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/0/*)",
Some("wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/1/*)"),
bitcoin::Network::Testnet,
MemoryDatabase::default(),
)?;
println!("Address #0: {}", wallet.get_address(New)?);
println!("Address #1: {}", wallet.get_address(New)?);
println!("Address #2: {}", wallet.get_address(New)?);
Ok(())
}
Create a transaction
use bdk::{FeeRate, Wallet, SyncOptions};
use bdk::database::MemoryDatabase;
use bdk::blockchain::ElectrumBlockchain;
use bdk::electrum_client::Client;
use bdk::wallet::AddressIndex::New;
use bitcoin::base64;
use bitcoin::consensus::serialize;
fn main() -> Result<(), bdk::Error> {
let blockchain = ElectrumBlockchain::from(Client::new("ssl://electrum.blockstream.info:60002")?);
let wallet = Wallet::new(
"wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/0/*)",
Some("wpkh([c258d2e4/84h/1h/0h]tpubDDYkZojQFQjht8Tm4jsS3iuEmKjTiEGjG6KnuFNKKJb5A6ZUCUZKdvLdSDWofKi4ToRCwb9poe1XdqfUnP4jaJjCB2Zwv11ZLgSbnZSNecE/1/*)"),
bitcoin::Network::Testnet,
MemoryDatabase::default(),
)?;
wallet.sync(&blockchain, SyncOptions::default())?;
let send_to = wallet.get_address(New)?;
let (psbt, details) = {
let mut builder = wallet.build_tx();
builder
.add_recipient(send_to.script_pubkey(), 50_000)
.enable_rbf()
.do_not_spend_change()
.fee_rate(FeeRate::from_sat_per_vb(5.0));
builder.finish()?
};
println!("Transaction details: {:#?}", details);
println!("Unsigned PSBT: {}", base64::encode(&serialize(&psbt)));
Ok(())
}
Sign a transaction
use bdk::{Wallet, SignOptions, database::MemoryDatabase};
use bitcoin::base64;
use bitcoin::consensus::deserialize;
fn main() -> Result<(), bdk::Error> {
let wallet = Wallet::new(
"wpkh([c258d2e4/84h/1h/0h]tprv8griRPhA7342zfRyB6CqeKF8CJDXYu5pgnj1cjL1u2ngKcJha5jjTRimG82ABzJQ4MQe71CV54xfn25BbhCNfEGGJZnxvCDQCd6JkbvxW6h/0/*)",
Some("wpkh([c258d2e4/84h/1h/0h]tprv8griRPhA7342zfRyB6CqeKF8CJDXYu5pgnj1cjL1u2ngKcJha5jjTRimG82ABzJQ4MQe71CV54xfn25BbhCNfEGGJZnxvCDQCd6JkbvxW6h/1/*)"),
bitcoin::Network::Testnet,
MemoryDatabase::default(),
)?;
let psbt = "...";
let mut psbt = deserialize(&base64::decode(psbt).unwrap())?;
let finalized = wallet.sign(&mut psbt, SignOptions::default())?;
Ok(())
}
Testing
Unit testing
cargo test
Integration testing
Integration testing require testing features, for example:
cargo test --features test-electrum
The other options are test-esplora
, test-rpc
or test-rpc-legacy
which runs against an older version of Bitcoin Core.
Note that electrs
and bitcoind
binaries are automatically downloaded (on mac and linux), to specify you already have installed binaries you must use --no-default-features
and provide BITCOIND_EXE
and ELECTRS_EXE
as environment variables.
License
Licensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.