isotonic calibration #795
Open
isotonic calibration #795
+586
−1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Fleur Petit <fleur.petit@alliander.com>
Signed-off-by: Fleur Petit <fleur.petit@alliander.com>
fleur-petit
force-pushed
the
feature/673-calibration-isotonic-quantile
branch
from
4bd91ed to
469f92c
Compare
…73-calibration-isotonic-quantile
Lars800
reviewed
Dec 19, 2025
...penstef-models/src/openstef_models/transforms/postprocessing/isotonic_quantile_calibrator.py
Show resolved
Hide resolved
...penstef-models/src/openstef_models/transforms/postprocessing/isotonic_quantile_calibrator.py
Show resolved
Hide resolved
Comment on lines
+132
to
+135
| # Remove NaN values (required for IsotonicRegression) | ||
| mask = ~(np.isnan(predictions) | np.isnan(actuals)) | ||
| predictions_clean = predictions[mask] | ||
| actuals_clean = actuals[mask] |
Collaborator
There was a problem hiding this comment.
Calibriation can be quite tricky in case we have a very small data. To limit this risk we could introduce a warning or condition on the number of datapoints. If the fraction or absolute number of datapoints is below this threshold, we can warn the user or even skip the calibration entirely.
Collaborator
Author
There was a problem hiding this comment.
The .05 is a bit arbitrary, what do you think?
| Returns: | ||
| Array of locally estimated quantile values matching the input order. | ||
| """ | ||
| # Sort by predicted values |
Collaborator
There was a problem hiding this comment.
Should we set a lower limit on the length of the window? If we use the np.quantile we need at least 100 observations. Otherwise we risk Q50 == Q51
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements a postprocessing calibration method for quantile estimation using sk-learn's IsotonicRegression.
Changes
IsotonicCalibratorto postprocessing transforms.IsotonicCalibratorfunctionality on different data with different settings.IsotonicCalibratoron observed vs expected quantilesTesting