Predictive accuracy = zero

[WalterDurka]

Dear Gideon,
I try to use conStruct on a ddRAD dataset (allele frequencies at 5080 loci of 283 worldwide populations) of a selfing plant species.

Q1: cross validation reaches values of zero for the spatial model at K>2. I wonder whether this is reasonable, as in your examples (2018 paper) , values close to zero are found, but never exactly 0. Are these zeros reliable?
Q2: In cross validation, even for n.iter = 3000, I get a series of warnings (see below), although the trace plots look fairly homogeneous. Can I ignore these warnings? When?

Thanks for any hints
Walter

del.ni.iter3000.spK2_trace.plots.chain_1.pdf

"Warning messages:
1: There were 245 divergent transitions after warmup. See
https://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
to find out why this is a problem and how to eliminate them.
2: Examine the pairs() plot to diagnose sampling problems

3: The largest R-hat is 1.19, indicating chains have not mixed.
Running the chains for more iterations may help. See
https://mc-stan.org/misc/warnings.html#r-hat
4: Bulk Effective Samples Size (ESS) is too low, indicating posterior means and medians may be unreliable.
Running the chains for more iterations may help. See
https://mc-stan.org/misc/warnings.html#bulk-ess
5: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
Running the chains for more iterations may help. See
https://mc-stan.org/misc/warnings.html#tail-ess "

[gbradburd]

Hi Walter! Sorry for the slow reply - I'm on sabbatical right now and have been much slower on computer stuff as a result. Anyway:

1. The cross-validation procedure we use (outlined in our 2018 paper, in the Appendix), sets the standardized log likelihood of the testing partition under the best model to be 0. So, I'd interpret your results to mean that there is strong support at all values of K for the spatial model over the nonspatial one, and that (w/ the caveat that it might be useful to zoom in on the y-axis for values of K>2) the gains in predictive accuracy for K>3 are either nonexistent or negligible.
2. Those warnings indicate that STAN thinks the mixing is not ideal. Because all downstream inferences are predicated on the good mixing behavior of STAN's HMC, that could potentially be a problem. That said, if I see the same results consistently across several independent runs, and the MCMC diagnosis looks ok, I will sometimes ignore them (don't tell anybody 😬 ).