e6433fb2c1 feat(persist): Add stage_and_commit to Persist (LLFourn)
0bee46e75b fix(store): Remove lifetime (LLFourn)
Pull request description:
Remove gratuitous use of lifetimes in the main persistence struct `Store`. Having lifetimes on this means that you have to keep the magic bytes alive longer than the database which is particularly offensive if you have to send the database to another thread. On top of that the bytes aren't even read.
ACKs for top commit:
evanlinjin:
ACK e6433fb2c1
Tree-SHA512: 7f6d9d60951a8ceaee30719d0771e15632c6fad0702294af15409c5df492669a07299874ef5ee34e3d75bdecbbd41df29bced3ff16b2360d5d5c7687ef677ffc
a4f28c079e chore: improve LocalChain::apply_header_connected_to doc (LLFourn)
8ec65f0b8e feat(example): add RPC wallet example (Vladimir Fomene)
a7d01dc39a feat(chain)!: make `IndexedTxGraph::apply_block_relevant` more efficient (志宇)
e0512acf94 feat(bitcoind_rpc)!: emissions include checkpoint and connected_to data (志宇)
8f2d4d9d40 test(chain): `LocalChain` test for update that is shorter than original (志宇)
9467cad55d feat(wallet): introduce block-by-block api (Vladimir Fomene)
d3e5095df1 feat(chain): add `apply_header..` methods to `LocalChain` (志宇)
2b61a122ff feat(chain): add `CheckPoint::from_block_ids` convenience method (志宇)
Pull request description:
### Description
Introduce block-by-block API for `bdk::Wallet`. A `wallet_rpc` example is added to demonstrate syncing `bdk::Wallet` with the `bdk_bitcoind_rpc` chain-source crate.
The API of `bdk_bitcoind_rpc::Emitter` is changed so the receiver knows how to connect to the block emitted.
### Notes to the reviewers
### Changelog notice
Added
* `Wallet` methods to apply full blocks (`apply_block` and `apply_block_connected_to`) and a method to apply a batch of unconfirmed transactions (`apply_unconfirmed_txs`).
* `CheckPoint::from_block_ids` convenience method.
* `LocalChain` methods to apply a block header (`apply_header` and `apply_header_connected_to`).
* Test to show that `LocalChain` can apply updates that are shorter than original. This will happen during reorgs if we sync wallet with `bdk_bitcoind_rpc::Emitter`.
Fixed
* `InsertTxError` now implements `std::error::Error`.
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
LLFourn:
self-ACK: a4f28c079e
evanlinjin:
ACK a4f28c079e
Tree-SHA512: e39fb65b4e69c0a6748d64eab12913dc9cfe5eb8355ab8fb68f60a37c3bb2e1489ddd8f2f138c6470135344f40e3dc671928f65d303fd41fb63f577b30895b60
In the example_cli we were not always committing (seemingly by mistake).
This then caused all the examples to have to compensate by manually
committing.
Previously `SpkTxOutIndex` methods can be called from
`KeychainTxOutIndex` due to the `DeRef` implementation. However, the
internal `SpkTxOut` will also contain lookahead spks resulting in an
error-prone API.
`SpkTxOutIndex` methods are now not directly callable from
`KeychainTxOutIndex`. Methods of `KeychainTxOutIndex` are renamed for
clarity. I.e. methods that return an unbounded spk iter are prefixed
with `unbounded`.
Previously, emissions are purely blocks + the block height. This means
emitted blocks can only connect to previous-adjacent blocks. Hence, sync
must start from genesis and include every block.
The wallet is currently created without setting any lookahead value for
the keychain. This implicitly makes it a lookahead of 0. As this is a
high-level interface we should avoid footguns and aim for a reasonable
default.
Instead of simply patching it for wallet, we alter `KeychainTxOutIndex`
to have a default lookahead value. Additionally, instead of a
per-keychain lookahead, the constructor asks for a `lookahead` value.
This avoids the footguns of having methods which allows the caller the
decrease the `lookahead` (and therefore panicing). This also simplifies
the API.
Co-authored-by: Antoine Poisot <darosior@protonmail.com>
Co-authored-by: 志宇 <hello@evanlinjin.me>
`PersistBackend::is_empty` is removed. Instead, `load_from_persistence`
returns an option of the changeset. `None` means persistence is empty.
This is a better API than a separate method. We can now differentiate
between a persisted single changeset and nothing persisted.
`Store::aggregate_changeset` is refactored to return a `Result` instead
of a tuple. A new error type (`AggregateChangesetsError`) is introduced
to include the partially-aggregated changeset in the error. This is a
more idiomatic API.
These methods try to load wallet from persistence and initializes the
wallet instead if non-existant.
An internal helper method `create_signers` is added to reuse code.
Documentation is also improved.
`Wallet::new` now creates a new wallet. `Wallet::load` loads an existing
wallet. The network type is now recoverable from persistence. Error
types have been simplified.
85c62532a5 docs(bitcoind_rpc): better `Emitter::mempool` explanation (志宇)
b69c13ddf6 example_bitcoind_rpc: tweaks (志宇)
5f34df8489 bitcoind_rpc!: bring back `CheckPoint`s to `Emitter` (志宇)
57590e0a1f bitcoind_rpc: rm `BlockHash` from `Emitter::last_mempool_tip` (志宇)
6d4b33ef91 chain: split `IndexedTxGraph::insert_tx` into 3 methods (志宇)
4f5695d43a chain: improvements to `IndexedTxGraph` and `TxGraph` APIs (志宇)
150d6f8ab6 feat(example_bitcoind_rpc_polling): add example for RPC polling (志宇)
4f10463d9e test(bitcoind_rpc): add no_agreement_point test (志宇)
a73dac2d91 test(bitcoind_rpc): initial tests for `Emitter` (志宇)
bb7424d11d feat(bitcoind_rpc): introduce `bitcoind_rpc` crate (志宇)
240657b167 chain: add batch-insert methods for `IndexedTxGraph` (志宇)
43bc813c64 chain: add helper methods on `CheckPoint` (志宇)
b3db5ca9df feat(chain): add `AnchorFromBlockPosition` trait (志宇)
f795a43cc7 feat(example_cli): allow chain specific args in examples (志宇)
Pull request description:
### Description
This PR builds on top of #1034 and adds the `bitcoind_rpc` chain-src module and example.
### Notes to the reviewers
Don't merge this until #1034 is in!
### Changelog notice
* Add `bitcoind_rpc` chain-source module.
* Add `example_bitcoind_rpc` example module.
* Add `AnchorFromBlockPosition` trait which are for anchors that can be constructed from a given block, height and position in block.
* Add helper methods to `IndexedTxGraph` and `TxGraph` for batch operations and applying blocks directly.
* Add helper methods to `CheckPoint` for easier construction from a block `Header`.
### Checklists
* [x] Add test: we should detect when an initially-confirmed transaction is "unconfirmed" during a reorg.
* [x] Improve `example_bitcoind_rpc`: add `live` command.
* [x] Improve docs.
* [x] Reintroduce `CheckPoint`.
#### All Submissions:
* [x] I've signed all my commits
* [x] I followed the [contribution guidelines](https://github.com/bitcoindevkit/bdk/blob/master/CONTRIBUTING.md)
* [x] I ran `cargo fmt` and `cargo clippy` before committing
#### New Features:
* [x] I've added tests for the new feature
* [x] I've added docs for the new feature
ACKs for top commit:
notmandatory:
Re ACK 85c62532a5
Tree-SHA512: 88dbafbebaf227b18c69f2ea884e3e586bf9c11e5e450eb4872ade1d1ccd5cf1e33ce9930a6f5aa918baa3e92add7503858b039b8c9d553a281ad6d833f08a49
* avoid holding mutex lock over io
* document `CHANNEL_BOUND` const
* use the `relevant` variant of `batch_insert_unconfirmed`
* print elapsed time in stdout for various updates
* `bdk_chain` dependency is added. In the future, we will introduce a
separate `bdk_core` crate to contain shared types.
* replace `Emitter::new` with `from_height` and `from_checkpoint`
* `from_height` emits from the given start height
* `from_checkpoint` uses the provided cp to find agreement point
* introduce logic that ensures emitted blocks can connect with
receiver's `LocalChain`
* in our rpc example, we can now `expect()` chain updates to always
since we are using checkpoints and receiving blocks in order
Instead of inserting anchors and seen_at timestamp in the same method,
we have three separate methods. This makes the API easier to understand
and makes `IndexedTxGraph` more consistent with the `TxGraph` API.
For `IndexedTxGraph`:
- Remove `InsertTxItem` type (this is too complex).
- `batch_insert_relevant` now uses a simple tuple `(&tx, anchors)`.
- `batch_insert` is now also removed, as the same functionality can be
done elsewhere.
- Add internal helper method `index_tx_graph_changeset` so we don't need
to create a seprate `TxGraph` update in each method.
- `batch_insert_<relevant>_unconfirmed` no longer takes in an option of
last_seen.
- `batch_insert_unconfirmed` no longer takes a reference of a
transaction (since we apply all transactions anyway, so there is no
need to clone).
For `TxGraph`:
- Add `batch_insert_unconfirmed` method.
...graph update
Fixes#1151.
When wallet_esplora_* was used to sync a wallet containing a transaction
confirmed some time ago (more than 10-15 blocks ago), the transaction would
be stuck in an "unconfirmed" state forever.
At the first scan time, `update_local_chain` would just fetch the last 10 to
15 blocks (depending on the server used), and `tx_graph.missing_heights`
wouldn't return the tx's confirmation block as it was called on the
original, non-updated tx_graph.
So, after the first scan, we would have a transaction in memory with an
anchor that doesn't exist in our local_chain, and try_get_chain_position
would return unconfirmed.
When scanning again, missing_heights would find the missing anchor, but
`update_local_chain` wouldn't include it as it's older than
ASSUME_FINAL_DEPTH.
The missing block would be downloaded every time, but never included in
the local_chain, and the transaction would remain unconfirmed forever.
Here we call missing_heights on the updated graph, so that it can
correctly return the anchor height, and `update_local_chain` can
fetch it and include it in the chain.
We remove `ElectrumUpdate` and return tuples instead for `ElectrumExt`
methods. We introduce the `IncompleteTxGraph` structure to specifically
hodl the incomplete `TxGraph`.
This change is motivated by @LLFourn's comment: 794bf37e63 (r1305432603)
Added
- Wallet::sent_and_received function
- Wallet::calculate_fee and Wallet::calculate_fee_rate functions
- Wallet::error::CalculateFeeError
BREAKING CHANGES:
Removed
- TransactionDetails struct
Changed
- Wallet::get_tx now returns CanonicalTx instead of TransactionDetails
- TxBuilder::finish now returns only a PartiallySignedTransaction
