Better visualisations for entropies

[XachaB]

This adds zones of interpredictibility to the heatmap computation (they are also heatmaps), as well as heatmaps of the resulting distillations.

eg:

qumin action=ent_heatmap data=source/vlexique/vlexique.package.json entropy.importFile=results/metrics/entropies.csv heatmap.order="[Mode,Tense,Number,Person,Gender]" heatmap.cols="[Mode,Tense,Gender]"

Produces:

This introduces a heatmap.cols keyword to know which features to show in columns (the others go to rows). If not given, all cells are given in rows.

[XachaB]

I think this is an improvement because:

• I am fed-up with re-implementing this for each dataset, I want a generic solution.
• it is really nice to get an overview of the results (and whether something looks odd)
• The distillation is much easier to read than the full heatmap

[julesbouton]

I didn't review the full code yet, but:

• This fails (maybe because everything is interpredictable in non-overabundant settings):

qumin action=H data=./tests/data/TestPackage/test.package.json

• Do you properly exclude n-predictor computation results ?

I will review the rest of the code this afternoon.

[XachaB]

Thanks, I'll fix those

[julesbouton]

Seems fine to me!

[XachaB]

I added colorbars like so for frequencies (when available):

They represent only cell frequencies, but may help provide some help in interpretation.

[julesbouton]

Wow, nice! Is there a way to disable them easily if a user wants a simple output? And do they scale well for npreds heatmaps?

[julesbouton]

Actually, I did also rewrite part of the heatmap code in the probability of success branch (#67), and I added a few options, as an easy way to disable the n_pairs graph, etc. Maybe I could already bring back those changes in v3, since they can be useful.

[XachaB]

I didn't do n-preds, but it would be doable (with n columns for frequency for example ?)

I added a few options, as an easy way to disable the n_pairs graph, etc. Maybe I could already bring back those changes in v3, since they can be useful.

That sounds good, can you add them to this PR ?

[XachaB]

I haven't added an option to disable printing of the frequencies: do we want that ? I would like to avoid proliferation of fine grained options. I can see the case for: we do a good enough figure with all the info we have, then it's up to the user to write their own code if they want finer control. Of course, you can argue the opposite, and perhaps we'll fall in agreement ;)

[XachaB]

Note: if the weighs we calculate for pairs of cells were given like entropies (in the measure/value columns), then we'd get a heatmap of probabilistic weights for free. Wouldn't that be helpful too ?

[julesbouton]

True, and the are already given when setting token_freq.cells=True ! Le 6 mars 2025 18:01:20 GMT+01:00, Sacha Beniamine ***@***.***> a écrit :

…

XachaB left a comment (XachaB/Qumin#74) Note: if the weighs we calculate for pairs of cells were given like entropies (in the measure/value columns), then we'd get a heatmap of probabilistic weights for free. Wouldn't that be helpful too ? -- Reply to this email directly or view it on GitHub: #74 (comment) You are receiving this because your review was requested. Message ID: ***@***.***>

[julesbouton]

Yes of course! As soon as possible, but the end of the week is intense! Le 6 mars 2025 17:54:43 GMT+01:00, Sacha Beniamine ***@***.***> a écrit :

…

XachaB left a comment (XachaB/Qumin#74) I didn't do n-preds, but it would be doable (with n columns for frequency for example ?) > I added a few options, as an easy way to disable the n_pairs graph, etc. Maybe I could already bring back those changes in v3, since they can be useful. That sounds good, can you add them to this PR ? -- Reply to this email directly or view it on GitHub: #74 (comment) You are receiving this because your review was requested. Message ID: ***@***.***>

[julesbouton]

OK, I agree. I think we should just be able to easily produce a figure containing only the heatmap for 1 pred, which is typically what you want to share (that's the reason behind the options to disable the debug and the n_preds). This is even more striking when you add a probability of success and the debug of the probability of success: too many plots on the same figure. Le 6 mars 2025 17:55:56 GMT+01:00, Sacha Beniamine ***@***.***> a écrit :

…

XachaB left a comment (XachaB/Qumin#74) I haven't added an option to disable printing of the frequencies: do we want that ? I would like to avoid proliferation of fine grained options. I can see the case for: we do a good enough figure with all the info we have, then it's up to the user to write their own code if they want finer control. Of course, you can argue the opposite, and perhaps we'll fall in agreement ;) -- Reply to this email directly or view it on GitHub: #74 (comment) You are receiving this because your review was requested. Message ID: ***@***.***>

[XachaB]

Ok, ok, I'll add the options.

Yes of course! As soon as possible, but the end of the week is intense!

No rush of course !

[julesbouton]

I reorganised the code for the heatmaps to make it more readable. It it also a bit more versatile, although we do not use its full potential yet. For instance, it should be possible to easily add new metrics / debug computations.

Next step: add the frequencies for n_preds. Should be fairly easy.

[julesbouton]

Currently merging this: something is broken in dev. I'm trying to fix it.

[julesbouton]

In my latest commit, I :

• I added an export of both the predictors probability and the predictor-target probability in all entropy runs.
• I use this export for the frequencies, so that we have a probability for n-predictors combination.
• We could also add the heatmap of the pair probabilities that you mentionned. But is it really useful if we already have the frequencies for the predictors and the targets on the side of the heatmap?

The new entropies file looks like this:

predictor,predicted,measure,value,n_pairs,n_preds,dataset,pair_probability,pred_probability,probability_source
second,third,cond_entropy,0.0,2,1,test,0.11219602550871902,0.46187363834422657,tokens
third,second,cond_entropy,0.0,2,1,test,0.06945468245777843,0.13071895424836602,tokens

[XachaB]

I think we can merge now, before it becomes its own monster. This meets the initial specs plus some :)