whoisdomain.query() - memory leaks #30
Description
Describe the bug
Running thousands of whoisdomain.guery() calls under multiprocesses or multithreads. After few hours I noticed OS mem consumption increase from 4-5GB to 30-35GB. After digging into my code and setting up more strict garbage collection came to conclusion that whoisdomain might be area of leak. Running under Win with default sysinternals whois.exe
To Reproduce
Python code:
import whoisdomain
import gc
from memory_profiler import profile
#Instead of result from db, get any array of domain names
sql_select = f'SELECT domain FROM three_letter_com LIMIT 50'
result = db.execute_single(sql_select, '')
@profile
def check():
for item in result:
print(f'Checking domain: {item[0]}')
try:
whoisdomain.query(item[0])
except whoisdomain.WhoisPrivateRegistry as exc:
print(exc)
except whoisdomain.WhoisCommandFailed as exc:
print(exc)
except whoisdomain.WhoisQuotaExceeded as exc:
print(exc)
except whoisdomain.FailedParsingWhoisOutput as exc:
print(exc)
except whoisdomain.UnknownTld as exc:
print(exc)
except whoisdomain.UnknownDateFormat as exc:
print(exc)
except whoisdomain.WhoisCommandTimeout as exc:
print(exc)
gc.collect()
check()
Outputs
