50 Commits

Author SHA1 Message Date
softsimon
4407b42aab
Merge branch 'master' into nymkappa/bugfix/tx-fetcher-crash 2023-01-26 17:02:35 +04:00
softsimon
dcc8b81ca6
Merge branch 'master' into nymkappa/bugfix/cleanup-logs 2023-01-05 00:46:21 +04:00
nymkappa
a975936d3c
Don't try to import LN historical stats if no topology folder is set 2022-12-28 12:13:41 +01:00
nymkappa
3a7dffbe09
Fix crash when channel short id is not valid 2022-12-06 10:51:01 +01:00
nymkappa
efb48271f9
Cleanup logs 2022-12-01 15:52:06 +01:00
softsimon
5e72ecfdc9
Support Maxmind Lite
fixes #2553
2022-11-22 16:13:27 +09:00
nymkappa
faec398cf0
Log correct maxmind mysql updates - fix stats import processed files counter 2022-09-09 14:59:49 +02:00
nymkappa
cd2ef20d5c
Harcode lunanode, FDCservers and cogent asn 2022-08-30 17:06:48 +02:00
wiz
af805f15c7
Merge pull request #2441 from mempool/nymkappa/feature/ip-check
Fix wrong ASN for Lunanode ip ranges
2022-08-30 15:53:19 +02:00
nymkappa
1d571c284c
Fix wrong ASN for Lunanode ip ranges 2022-08-29 23:36:18 +02:00
wiz
1a5ee32565
Merge branch 'master' into nymkappa/bugfix/historical-node-decoding 2022-08-29 23:05:30 +02:00
nymkappa
0847e15d07
Update node sockets during historical import so we don't have to truncate 2022-08-29 09:01:07 +02:00
nymkappa
f25ec08f5e
Format historical node sockets with update topology outputs 2022-08-29 08:43:26 +02:00
wiz
1cd0796428
Merge branch 'master' into nymkappa/bugfix/disable-ln-import-testsignet 2022-08-28 14:24:05 +02:00
nymkappa
35de1d4d9a
Disable LN historical import if not mainnet 2022-08-28 08:24:29 +02:00
nymkappa
6b049f2c33
Manually delete some lightning_stats rows once the topology import is completed 2022-08-28 07:59:23 +02:00
nymkappa
7ba8a3da84
Hardcode some condition to invalidate imported topology files 2022-08-28 07:59:23 +02:00
softsimon
85c428be25
Store capacity and channels in nodes table 2022-08-26 02:31:14 +04:00
nymkappa
a151a90d2f
Import historical nodes 2022-08-24 12:12:16 +02:00
nymkappa
5b9b717a93
Fix LN stats importer with new data "cleanupTopology" structure 2022-08-19 18:07:26 +02:00
nymkappa
e7d99e9653
Insert channels from historical data 2022-08-19 12:34:38 +02:00
nymkappa
1ef4485a26
Use timestamp instead of date in stats table 2022-08-19 12:09:58 +02:00
nymkappa
0243769a02
Improve error logging in ln import 2022-08-18 11:14:34 +02:00
nymkappa
57e0980134
Import json topology 2022-08-18 11:06:20 +02:00
nymkappa
c37b4cadb1
Wrap LN importer into try/catch 2022-08-18 07:48:58 +02:00
nymkappa
8dc41257ce
Remove xml parser - Read only .topology file and assume json format 2022-08-16 21:47:52 +02:00
nymkappa
a71262f538
Assume topology file are in .json - trim log 2022-08-16 19:29:00 +02:00
nymkappa
264ce1222a
Remove "invalid data skipping fix" from stats importer 2022-08-16 19:00:08 +02:00
nymkappa
06453edc7c
Fix LN stats node per network count 2022-08-12 12:12:34 +02:00
nymkappa
61e512b8f7
Refactor the LN backend and add more logs 2022-08-09 11:12:05 +02:00
nymkappa
6a52725b63
Make sure we work with integer in the stats importer 2022-08-09 10:28:40 +02:00
nymkappa
abb078f7ee
Convert to short_id before fetching the funding tx 2022-08-09 09:21:31 +02:00
nymkappa
47363b477e
Refactor the LN backend and add more logs 2022-08-09 09:20:25 +02:00
nymkappa
54669281de
Run node stats every 10 minutes, only keep the latest entry per day 2022-08-04 18:27:36 +02:00
nymkappa
6be2985b40
Fix daily LN stats crash 2022-08-03 12:13:55 +02:00
nymkappa
a25af16f7c
Fetch funding tx for clightning channels 2022-08-02 18:34:20 +02:00
nymkappa
b6ba3c5781
Ignore channels fee rate > 5000ppm or base fee > 5000 in stats 2022-08-02 18:15:34 +02:00
nymkappa
5b521cfc7c
Don't insert gapped gossip data upon restart 2022-08-02 17:56:46 +02:00
nymkappa
d7f2f4136c
Small cleanup 2022-08-02 16:00:40 +02:00
nymkappa
5d7e42195f
Reduce massive gaps in the imported historical LN data 2022-08-02 16:00:40 +02:00
nymkappa
7fdf95ad34
Remove buggy tx vout value fetching and improve performances 2022-08-02 16:00:40 +02:00
nymkappa
5287490894
Make sure to not count channels twice 2022-08-02 16:00:40 +02:00
nymkappa
2daf94f65a
Re-use LN stats importer code to log daily LN stats 2022-08-02 16:00:39 +02:00
nymkappa
91ada9ce75
Integrate LN stats importer into the main process 2022-08-02 16:00:39 +02:00
nymkappa
4ea1e98547
Import LN historical statistics (network wide + per node) 2022-08-02 16:00:38 +02:00
wiz
ad9e989598
Correct maxmind geoip-db GEOLITE2_ISP to GEOIP2_ISP 2022-07-23 23:53:28 +02:00
nymkappa
ffe22399d5
Integrate GeoIP2 ISP database 2022-07-23 23:33:13 +02:00
nymkappa
93e93d44f4
Use country iso code in ln nodes per country page url 2022-07-17 09:53:02 +02:00
nymkappa
3d2ff7ef62
Store AS organization in geo_names 2022-07-12 12:12:10 +02:00
softsimon
519494668b
Use maxmind to store node locations 2022-07-11 17:52:38 +02:00