frost/bdk
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge bitcoindevkit/bdk#1172: Introduce block-by-block API to bdk::Wallet and add RPC wallet example

Browse Source 
a4f28c079e chore: improve LocalChain::apply_header_connected_to doc (LLFourn)
8ec65f0b8e feat(example): add RPC wallet example (Vladimir Fomene)
a7d01dc39a feat(chain)!: make `IndexedTxGraph::apply_block_relevant` more efficient (志宇)
e0512acf94 feat(bitcoind_rpc)!: emissions include checkpoint and connected_to data (志宇)
8f2d4d9d40 test(chain): `LocalChain` test for update that is shorter than original (志宇)
9467cad55d feat(wallet): introduce block-by-block api (Vladimir Fomene)
d3e5095df1 feat(chain): add `apply_header..` methods to `LocalChain` (志宇)
2b61a122ff feat(chain): add `CheckPoint::from_block_ids` convenience method (志宇)

Pull request description:

  ### Description

  Introduce block-by-block API for `bdk::Wallet`. A `wallet_rpc` example is added to demonstrate syncing `bdk::Wallet` with the `bdk_bitcoind_rpc` chain-source crate.

  The API of `bdk_bitcoind_rpc::Emitter` is changed so the receiver knows how to connect to the block emitted.

  ### Notes to the reviewers

  ### Changelog notice

  Added
  * `Wallet` methods to apply full blocks (`apply_block` and `apply_block_connected_to`) and a method to apply a batch of unconfirmed transactions (`apply_unconfirmed_txs`).
  * `CheckPoint::from_block_ids` convenience method.
  * `LocalChain` methods to apply a block header (`apply_header` and `apply_header_connected_to`).
  * Test to show that `LocalChain` can apply updates that are shorter than original. This will happen during reorgs if we sync wallet with `bdk_bitcoind_rpc::Emitter`.

  Fixed
  * `InsertTxError` now implements `std::error::Error`.

  #### All Submissions:

  * [x] I've signed all my commits
  * [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
  * [x] I ran `cargo fmt` and `cargo clippy` before committing

  #### New Features:

  * [x] I've added tests for the new feature
  * [x] I've added docs for the new feature

ACKs for top commit:
  LLFourn:
    self-ACK: a4f28c079e
  evanlinjin:
    ACK a4f28c079e

Tree-SHA512: e39fb65b4e69c0a6748d64eab12913dc9cfe5eb8355ab8fb68f60a37c3bb2e1489ddd8f2f138c6470135344f40e3dc671928f65d303fd41fb63f577b30895b60
This commit is contained in:
志宇
2024-01-19 23:07:20 +08:00
parent 0a2a57060b a4f28c079e
commit 25653d71b8
12 changed files with 983 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
example-crates/example_bitcoind_rpc_polling/README.md Normal file
View File

@@ -0,0 +1,68 @@
# Example RPC CLI
### Simple Regtest Test
1. Start local regtest bitcoind.
```
mkdir -p /tmp/regtest/bitcoind
bitcoind -regtest -server -fallbackfee=0.0002 -rpcuser=<your-rpc-username> -rpcpassword=<your-rpc-password> -datadir=/tmp/regtest/bitcoind -daemon
```
2. Create a test bitcoind wallet and set bitcoind env.
```
bitcoin-cli -datadir=/tmp/regtest/bitcoind -regtest -rpcuser=<your-rpc-username> -rpcpassword=<your-rpc-password> -named createwallet wallet_name="test"
export RPC_URL=127.0.0.1:18443
export RPC_USER=<your-rpc-username>
export RPC_PASS=<your-rpc-password>
```
3. Get test bitcoind wallet info.
```
bitcoin-cli -rpcwallet="test" -rpcuser=<your-rpc-username> -rpcpassword=<your-rpc-password> -datadir=/tmp/regtest/bitcoind -regtest getwalletinfo
```
4. Get new test bitcoind wallet address.
```
BITCOIND_ADDRESS=$(bitcoin-cli -rpcwallet="test" -datadir=/tmp/regtest/bitcoind -regtest -rpcuser=<your-rpc-username> -rpcpassword=<your-rpc-password> getnewaddress)
echo $BITCOIND_ADDRESS
```
5. Generate 101 blocks with reward to test bitcoind wallet address.
```
bitcoin-cli -datadir=/tmp/regtest/bitcoind -regtest -rpcuser=<your-rpc-username> -rpcpassword=<your-rpc-password> generatetoaddress 101 $BITCOIND_ADDRESS
```
6. Verify test bitcoind wallet balance.
```
bitcoin-cli -rpcwallet="test" -datadir=/tmp/regtest/bitcoind -regtest -rpcuser=<your-rpc-username> -rpcpassword=<your-rpc-password> getbalances
```
7. Set descriptor env and get address from RPC CLI wallet.
```
export DESCRIPTOR="wpkh(tprv8ZgxMBicQKsPfK9BTf82oQkHhawtZv19CorqQKPFeaHDMA4dXYX6eWsJGNJ7VTQXWmoHdrfjCYuDijcRmNFwSKcVhswzqs4fugE8turndGc/1/*)"
cargo run -- --network regtest address next
```
8. Send 5 test bitcoin to RPC CLI wallet.
```
bitcoin-cli -rpcwallet="test" -datadir=/tmp/regtest/bitcoind -regtest -rpcuser=<your-rpc-username> -rpcpassword=<your-rpc-password> sendtoaddress <address> 5
```
9. Sync blockchain with RPC CLI wallet.
```
cargo run -- --network regtest sync
<CNTRL-C to stop syncing>
```
10. Get RPC CLI wallet unconfirmed balances.
```
cargo run -- --network regtest balance
```
11. Generate 1 block with reward to test bitcoind wallet address.
```
bitcoin-cli -datadir=/tmp/regtest/bitcoind -rpcuser=<your-rpc-username> -rpcpassword=<your-rpc-password> -regtest generatetoaddress 10 $BITCOIND_ADDRESS
```
12. Sync the blockchain with RPC CLI wallet.
```
cargo run -- --network regtest sync
<CNTRL-C to stop syncing>
```
13. Get RPC CLI wallet confirmed balances.
```
cargo run -- --network regtest balance
```
14. Get RPC CLI wallet transactions.
```
cargo run -- --network regtest txout list
```

34
example-crates/example_bitcoind_rpc_polling/src/main.rs
View File

@@ -14,7 +14,7 @@ use bdk_bitcoind_rpc::{
use bdk_chain::{
bitcoin::{constants::genesis_block, Block, Transaction},
indexed_tx_graph, keychain,
local_chain::{self, CheckPoint, LocalChain},
local_chain::{self, LocalChain},
ConfirmationTimeHeightAnchor, IndexedTxGraph,
};
use example_cli::{
@@ -42,7 +42,7 @@ type ChangeSet = (
#[derive(Debug)]
enum Emission {
Block { height: u32, block: Block },
Block(bdk_bitcoind_rpc::BlockEvent<Block>),
Mempool(Vec<(Transaction, u64)>),
Tip(u32),
}
@@ -178,17 +178,20 @@ fn main() -> anyhow::Result<()> {
let mut last_db_commit = Instant::now();
let mut last_print = Instant::now();
while let Some((height, block)) = emitter.next_block()? {
while let Some(emission) = emitter.next_block()? {
let height = emission.block_height();
let mut chain = chain.lock().unwrap();
let mut graph = graph.lock().unwrap();
let mut db = db.lock().unwrap();
let chain_update =
CheckPoint::from_header(&block.header, height).into_update(false);
let chain_changeset = chain
.apply_update(chain_update)
.apply_update(local_chain::Update {
tip: emission.checkpoint,
introduce_older_blocks: false,
})
.expect("must always apply as we receive blocks in order from emitter");
let graph_changeset = graph.apply_block_relevant(block, height);
let graph_changeset = graph.apply_block_relevant(&emission.block, height);
db.stage((chain_changeset, graph_changeset));
// commit staged db changes in intervals
@@ -256,7 +259,8 @@ fn main() -> anyhow::Result<()> {
loop {
match emitter.next_block()? {
Some((height, block)) => {
Some(block_emission) => {
let height = block_emission.block_height();
if sigterm_flag.load(Ordering::Acquire) {
break;
}
@@ -264,7 +268,7 @@ fn main() -> anyhow::Result<()> {
block_count = rpc_client.get_block_count()? as u32;
tx.send(Emission::Tip(block_count))?;
}
tx.send(Emission::Block { height, block })?;
tx.send(Emission::Block(block_emission))?;
}
None => {
if await_flag(&sigterm_flag, MEMPOOL_EMIT_DELAY) {
@@ -293,13 +297,17 @@ fn main() -> anyhow::Result<()> {
let mut chain = chain.lock().unwrap();
let changeset = match emission {
Emission::Block { height, block } => {
let chain_update =
CheckPoint::from_header(&block.header, height).into_update(false);
Emission::Block(block_emission) => {
let height = block_emission.block_height();
let chain_update = local_chain::Update {
tip: block_emission.checkpoint,
introduce_older_blocks: false,
};
let chain_changeset = chain
.apply_update(chain_update)
.expect("must always apply as we receive blocks in order from emitter");
let graph_changeset = graph.apply_block_relevant(block, height);
let graph_changeset =
graph.apply_block_relevant(&block_emission.block, height);
(chain_changeset, graph_changeset)
}
Emission::Mempool(mempool_txs) => {

15
example-crates/wallet_rpc/Cargo.toml Normal file
View File

@@ -0,0 +1,15 @@
[package]
name = "wallet_rpc"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
bdk = { path = "../../crates/bdk" }
bdk_file_store = { path = "../../crates/file_store" }
bdk_bitcoind_rpc = { path = "../../crates/bitcoind_rpc" }
anyhow = "1"
clap = { version = "3.2.25", features = ["derive", "env"] }
ctrlc = "2.0.1"

45
example-crates/wallet_rpc/README.md Normal file
View File

@@ -0,0 +1,45 @@
# Wallet RPC Example
```
$ cargo run --bin wallet_rpc -- --help
wallet_rpc 0.1.0
Bitcoind RPC example usign `bdk::Wallet`
USAGE:
wallet_rpc [OPTIONS] <DESCRIPTOR> [CHANGE_DESCRIPTOR]
ARGS:
<DESCRIPTOR> Wallet descriptor [env: DESCRIPTOR=]
<CHANGE_DESCRIPTOR> Wallet change descriptor [env: CHANGE_DESCRIPTOR=]
OPTIONS:
--db-path <DB_PATH>
Where to store wallet data [env: BDK_DB_PATH=] [default: .bdk_wallet_rpc_example.db]
-h, --help
Print help information
--network <NETWORK>
Bitcoin network to connect to [env: BITCOIN_NETWORK=] [default: testnet]
--rpc-cookie <RPC_COOKIE>
RPC auth cookie file [env: RPC_COOKIE=]
--rpc-pass <RPC_PASS>
RPC auth password [env: RPC_PASS=]
--rpc-user <RPC_USER>
RPC auth username [env: RPC_USER=]
--start-height <START_HEIGHT>
Earliest block height to start sync from [env: START_HEIGHT=] [default: 481824]
--url <URL>
RPC URL [env: RPC_URL=] [default: 127.0.0.1:8332]
-V, --version
Print version information
```

182
example-crates/wallet_rpc/src/main.rs Normal file
View File

@@ -0,0 +1,182 @@
use bdk::{
bitcoin::{Block, Network, Transaction},
wallet::Wallet,
};
use bdk_bitcoind_rpc::{
bitcoincore_rpc::{Auth, Client, RpcApi},
Emitter,
};
use bdk_file_store::Store;
use clap::{self, Parser};
use std::{path::PathBuf, sync::mpsc::sync_channel, thread::spawn, time::Instant};
const DB_MAGIC: &str = "bdk-rpc-wallet-example";
/// Bitcoind RPC example usign `bdk::Wallet`.
///
/// This syncs the chain block-by-block and prints the current balance, transaction count and UTXO
/// count.
#[derive(Parser, Debug)]
#[clap(author, version, about, long_about = None)]
#[clap(propagate_version = true)]
pub struct Args {
/// Wallet descriptor
#[clap(env = "DESCRIPTOR")]
pub descriptor: String,
/// Wallet change descriptor
#[clap(env = "CHANGE_DESCRIPTOR")]
pub change_descriptor: Option<String>,
/// Earliest block height to start sync from
#[clap(env = "START_HEIGHT", long, default_value = "481824")]
pub start_height: u32,
/// Bitcoin network to connect to
#[clap(env = "BITCOIN_NETWORK", long, default_value = "testnet")]
pub network: Network,
/// Where to store wallet data
#[clap(
env = "BDK_DB_PATH",
long,
default_value = ".bdk_wallet_rpc_example.db"
)]
pub db_path: PathBuf,
/// RPC URL
#[clap(env = "RPC_URL", long, default_value = "127.0.0.1:8332")]
pub url: String,
/// RPC auth cookie file
#[clap(env = "RPC_COOKIE", long)]
pub rpc_cookie: Option<PathBuf>,
/// RPC auth username
#[clap(env = "RPC_USER", long)]
pub rpc_user: Option<String>,
/// RPC auth password
#[clap(env = "RPC_PASS", long)]
pub rpc_pass: Option<String>,
}
impl Args {
fn client(&self) -> anyhow::Result<Client> {
Ok(Client::new(
&self.url,
match (&self.rpc_cookie, &self.rpc_user, &self.rpc_pass) {
(None, None, None) => Auth::None,
(Some(path), _, _) => Auth::CookieFile(path.clone()),
(_, Some(user), Some(pass)) => Auth::UserPass(user.clone(), pass.clone()),
(_, Some(_), None) => panic!("rpc auth: missing rpc_pass"),
(_, None, Some(_)) => panic!("rpc auth: missing rpc_user"),
},
)?)
}
}
#[derive(Debug)]
enum Emission {
SigTerm,
Block(bdk_bitcoind_rpc::BlockEvent<Block>),
Mempool(Vec<(Transaction, u64)>),
}
fn main() -> anyhow::Result<()> {
let args = Args::parse();
let rpc_client = args.client()?;
println!(
"Connected to Bitcoin Core RPC at {:?}",
rpc_client.get_blockchain_info().unwrap()
);
let start_load_wallet = Instant::now();
let mut wallet = Wallet::new_or_load(
&args.descriptor,
args.change_descriptor.as_ref(),
Store::<bdk::wallet::ChangeSet>::open_or_create_new(DB_MAGIC.as_bytes(), args.db_path)?,
args.network,
)?;
println!(
"Loaded wallet in {}s",
start_load_wallet.elapsed().as_secs_f32()
);
let balance = wallet.get_balance();
println!("Wallet balance before syncing: {} sats", balance.total());
let wallet_tip = wallet.latest_checkpoint();
println!(
"Wallet tip: {} at height {}",
wallet_tip.hash(),
wallet_tip.height()
);
let (sender, receiver) = sync_channel::<Emission>(21);
let signal_sender = sender.clone();
ctrlc::set_handler(move || {
signal_sender
.send(Emission::SigTerm)
.expect("failed to send sigterm")
});
let emitter_tip = wallet_tip.clone();
spawn(move || -> Result<(), anyhow::Error> {
let mut emitter = Emitter::new(&rpc_client, emitter_tip, args.start_height);
while let Some(emission) = emitter.next_block()? {
sender.send(Emission::Block(emission))?;
}
sender.send(Emission::Mempool(emitter.mempool()?))?;
Ok(())
});
let mut blocks_received = 0_usize;
for emission in receiver {
match emission {
Emission::SigTerm => {
println!("Sigterm received, exiting...");
break;
}
Emission::Block(block_emission) => {
blocks_received += 1;
let height = block_emission.block_height();
let hash = block_emission.block_hash();
let connected_to = block_emission.connected_to();
let start_apply_block = Instant::now();
wallet.apply_block_connected_to(&block_emission.block, height, connected_to)?;
wallet.commit()?;
let elapsed = start_apply_block.elapsed().as_secs_f32();
println!(
"Applied block {} at height {} in {}s",
hash, height, elapsed
);
}
Emission::Mempool(mempool_emission) => {
let start_apply_mempool = Instant::now();
wallet.apply_unconfirmed_txs(mempool_emission.iter().map(|(tx, time)| (tx, *time)));
wallet.commit()?;
println!(
"Applied unconfirmed transactions in {}s",
start_apply_mempool.elapsed().as_secs_f32()
);
break;
}
}
}
let wallet_tip_end = wallet.latest_checkpoint();
let balance = wallet.get_balance();
println!(
"Synced {} blocks in {}s",
blocks_received,
start_load_wallet.elapsed().as_secs_f32(),
);
println!(
"Wallet tip is '{}:{}'",
wallet_tip_end.height(),
wallet_tip_end.hash()
);
println!("Wallet balance is {} sats", balance.total());
println!(
"Wallet has {} transactions and {} utxos",
wallet.transactions().count(),
wallet.list_unspent().count()
);
Ok(())
}