52 Commits

Author SHA1 Message Date
nymkappa
8dbba64994 Wrap lightning stats importer into try/catch 2023-04-01 14:56:18 +09:00
nymkappa
184698aeee Move some notice into info 2023-03-05 08:23:05 +09:00
softsimon
dbdf1815bb Merge branch 'master' into nymkappa/bugfix/tx-fetcher-crash 2023-01-26 17:02:35 +04:00
softsimon
92a356f10e Merge branch 'master' into nymkappa/bugfix/cleanup-logs 2023-01-05 00:46:21 +04:00
nymkappa
3ca72e3af0 Don't try to import LN historical stats if no topology folder is set 2022-12-28 12:13:41 +01:00
nymkappa
cf08e7cf38 Fix crash when channel short id is not valid 2022-12-06 10:51:01 +01:00
nymkappa
4631ba67b7 Cleanup logs 2022-12-01 15:52:06 +01:00
softsimon
3b9e3c33f6 Support Maxmind Lite
fixes #2553
2022-11-22 16:13:27 +09:00
nymkappa
f0ded96c22 Log correct maxmind mysql updates - fix stats import processed files counter 2022-09-09 14:59:49 +02:00
nymkappa
24251843dc Harcode lunanode, FDCservers and cogent asn 2022-08-30 17:06:48 +02:00
wiz
dfbab0a6a6 Merge pull request #2441 from mempool/nymkappa/feature/ip-check
Fix wrong ASN for Lunanode ip ranges
2022-08-30 15:53:19 +02:00
nymkappa
6fba330263 Fix wrong ASN for Lunanode ip ranges 2022-08-29 23:36:18 +02:00
wiz
cacc9d72f1 Merge branch 'master' into nymkappa/bugfix/historical-node-decoding 2022-08-29 23:05:30 +02:00
nymkappa
094ee66bc0 Update node sockets during historical import so we don't have to truncate 2022-08-29 09:01:07 +02:00
nymkappa
ed6e79d86e Format historical node sockets with update topology outputs 2022-08-29 08:43:26 +02:00
wiz
ac125464c7 Merge branch 'master' into nymkappa/bugfix/disable-ln-import-testsignet 2022-08-28 14:24:05 +02:00
nymkappa
5d35f7519f Disable LN historical import if not mainnet 2022-08-28 08:24:29 +02:00
nymkappa
c5159105d5 Manually delete some lightning_stats rows once the topology import is completed 2022-08-28 07:59:23 +02:00
nymkappa
b66a87e50c Hardcode some condition to invalidate imported topology files 2022-08-28 07:59:23 +02:00
softsimon
cf662ca070 Store capacity and channels in nodes table 2022-08-26 02:31:14 +04:00
nymkappa
0f258f25d7 Import historical nodes 2022-08-24 12:12:16 +02:00
nymkappa
eb87828fc0 Fix LN stats importer with new data "cleanupTopology" structure 2022-08-19 18:07:26 +02:00
nymkappa
b9664e1c3e Insert channels from historical data 2022-08-19 12:34:38 +02:00
nymkappa
f6e1ee8017 Use timestamp instead of date in stats table 2022-08-19 12:09:58 +02:00
nymkappa
dcff15c219 Improve error logging in ln import 2022-08-18 11:14:34 +02:00
nymkappa
678bf009ea Import json topology 2022-08-18 11:06:20 +02:00
nymkappa
440683f994 Wrap LN importer into try/catch 2022-08-18 07:48:58 +02:00
nymkappa
51c3d8a5a3 Remove xml parser - Read only .topology file and assume json format 2022-08-16 21:47:52 +02:00
nymkappa
b71d90d749 Assume topology file are in .json - trim log 2022-08-16 19:29:00 +02:00
nymkappa
d61af09b74 Remove "invalid data skipping fix" from stats importer 2022-08-16 19:00:08 +02:00
nymkappa
3e43a6750b Fix LN stats node per network count 2022-08-12 12:12:34 +02:00
nymkappa
e720f70235 Refactor the LN backend and add more logs 2022-08-09 11:12:05 +02:00
nymkappa
8a09a94453 Make sure we work with integer in the stats importer 2022-08-09 10:28:40 +02:00
nymkappa
34f581e8eb Convert to short_id before fetching the funding tx 2022-08-09 09:21:31 +02:00
nymkappa
295f0dd8ff Refactor the LN backend and add more logs 2022-08-09 09:20:25 +02:00
nymkappa
6317d0867a Run node stats every 10 minutes, only keep the latest entry per day 2022-08-04 18:27:36 +02:00
nymkappa
f61a04ec60 Fix daily LN stats crash 2022-08-03 12:13:55 +02:00
nymkappa
d9077412e2 Fetch funding tx for clightning channels 2022-08-02 18:34:20 +02:00
nymkappa
822a677624 Ignore channels fee rate > 5000ppm or base fee > 5000 in stats 2022-08-02 18:15:34 +02:00
nymkappa
b630816b2e Don't insert gapped gossip data upon restart 2022-08-02 17:56:46 +02:00
nymkappa
71abbae05d Small cleanup 2022-08-02 16:00:40 +02:00
nymkappa
2bddd97e85 Reduce massive gaps in the imported historical LN data 2022-08-02 16:00:40 +02:00
nymkappa
5205d92486 Remove buggy tx vout value fetching and improve performances 2022-08-02 16:00:40 +02:00
nymkappa
2c07b7f4fb Make sure to not count channels twice 2022-08-02 16:00:40 +02:00
nymkappa
5eed515bdb Re-use LN stats importer code to log daily LN stats 2022-08-02 16:00:39 +02:00
nymkappa
6ea171e45a Integrate LN stats importer into the main process 2022-08-02 16:00:39 +02:00
nymkappa
068f7392dd Import LN historical statistics (network wide + per node) 2022-08-02 16:00:38 +02:00
wiz
2878561e52 Correct maxmind geoip-db GEOLITE2_ISP to GEOIP2_ISP 2022-07-23 23:53:28 +02:00
nymkappa
6795f69343 Integrate GeoIP2 ISP database 2022-07-23 23:33:13 +02:00
nymkappa
cfad17d049 Use country iso code in ln nodes per country page url 2022-07-17 09:53:02 +02:00