Skip to content
This repository was archived by the owner on Jul 19, 2024. It is now read-only.

Fix slow response#26

Open
longhoangwkm wants to merge 1 commit intoCounterpartyXCP:masterfrom
longhoangwkm:hotfix/slow_response
Open

Fix slow response#26
longhoangwkm wants to merge 1 commit intoCounterpartyXCP:masterfrom
longhoangwkm:hotfix/slow_response

Conversation

@longhoangwkm
Copy link

Problem:

curl -XGET localhost:8432/a/1AuGvfDfNhbeVNUDzXHjLmEnFnWv3pkjjG/utxos

Response time: ~60 seconds or time out.

Debug:
To get utxos, we searchRange all blocks by addressToScriptID:
If latest block is ~1M6. Split all blocks to n time to hit db by chunkSize.
Example, latest block is ~1M6. / chunkSize(10) = 160k times.
i found that chunkSize too small, hit to leveldb too much.
Reduce n to help reduce response time

Solution:
I tried to increase chunkSize value to 100, 1000, 10_000 and 100_000.
Benchmark show that with chunkSize = 10_000 return good response Time
< 1 seconds on mainnet

$ time curl localhost:8432/a/1AuGvfDfNhbeVNUDzXHjLmEnFnWv3pkjjG/utxos
real	0m0.638s
user	0m0.005s
sys	0m0.015s

$ time curl localhost:8432/a/1Afb76aMKPTpSppV9nZrf3Nfmg4KXvikeA/utxos
real	0m0.169s
user	0m0.006s
sys	0m0.005s

$ time curl localhost:8432/a/19s5TAWJNdsRxzoHN51TPsWnCieD2h1M1y/utxos
real	0m0.090s
user	0m0.006s
sys	0m0.004s

chunkSize too small(10).
Scan all block take n*M times.
- M is average time to hit to leveldb
- n = highestBlock / chunkSize
Reduce n to help reduce response time
@jdogresorg
Copy link

jdogresorg commented May 30, 2020

I just noticed this pull requests and applied the suggested fix to the indexd instances on my servers and it seems to be running MUCH faster. Thank you very much @longhoangwkm for submitting this fix.

@chiguireitor ^^

@longhoangwkm
Copy link
Author

You're welcome. Btw, we sometimes have problems with exceeding the max limit for the number of node object errors. I think we should keep developing this CIP: https://counterpartytalk.org/t/cip22-change-indexd-to-addrindexrs/5870. We need a big change, I'm eager to contribute to a new project.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants