Custom python functions to help you further analyse machine learning models and diagnostic test.
Will help you make plots and compute evaluation metrics as seen in Nature Article, Leong et al. 2021
Metrics to compute and plot:
- AUC = Area Under the Curve
- NRI = Net Reclassification Index
- IDI = Integrated Discrimination Improvement
- Functions to compute bootstrap p-values for AUC and NRI differences
Run "example.ipynb" Jupyter notebook to see and use functions
pip install -r requirements.txt
pip install .pytest -qAUC [AUC = \int_0^1 TPR(FPR), dFPR ]
NRI [NRI = (P_{\text{up}|\text{event}} - P_{\text{down}|\text{event}}) + (P_{\text{down}|\text{non-event}} - P_{\text{up}|\text{non-event}})]
IDI [IDI = (\bar{p}{\text{new},1} - \bar{p}{\text{ref},1}) - (\bar{p}{\text{new},0} - \bar{p}{\text{ref},0})]
Import the package and call any metric helpers. See the example notebook for a detailed walkthrough.
Code and concepts further explained in the following post: "Area Under the Curve and Beyond" or "On Medium/Towards Data Science"
contact Lambert Leong