PsyR - an R package for better inference in multivariate statistical analysis #17
Description
Title
PsyR - an R package for better inference in multivariate statistical analysis
Short description and the goals for the OHBM BrainHack
We consistently use massive datasets across neuroscience and psychology. The routine gathering of big data requires that we are well equipped with tools that allow us to conduct appropriate multivariate statistics. This project aims to produce an R package that allows the researcher to overcome little discussed limitations of traditional multivariate statistical analyses.
Multivariate statistical analysis (e.g. MANOVA and repeated-measures ANOVA) typically follows a two stage procedure; an omnibus test of the global null hypothesis followed by post-hoc tests of specific effects. It is not well known that under certain circumstances, such as when the omnibus test is overpowered, that the type 1 error rate for this procedure is drastically inflated, sometimes to a type 1 error rate of 1! It is even less well known that this procedure can lead to an even lessor known type IV error, which is the incorrect interpretation of a correctly rejected hypothesis. This is caused when the follow-up contrasts are inadequate to test the question of interest, as can occur when testing simple effects.
It is possible to avoid these dragons by using an alternative procedure where all inferences are derived from simultaneous confidence intervals (SCIs) on contrasts of interests. The 'simultaneous' bit means that the same statistic contributes to both the omnibus and the contrast tests for significance, which controls the type 1 error rate. Even better, computing confidence intervals on contrasts of interests allows reseachers to move away from binary decision-making (is something significant or not?) to interpretations involving magnitude (how big is this effect likely to be at the population level?).
One piece of software (PSY) can produce SCIs appropriate for both planned analyses (where contrasts are defined independently of the data) and for more flexible analyses where contrasts are defined on a post-hoc basis. However, this software is only available for use on windows and cannot be scripted into reproducible workflows. We have, over several Brainhacks, built an R package that implements the functions of PSY, and to make this method of statistical inference available to the masses!
Our goals for the OHBM Brainhack 2025 are:
- Unlock the final fancy function of Psy; converting from pascal code to an R function the approximation required to compute the cdf of a particular distribution needed to calculate test statistics for 2 x 2 balanced ANOVA designs.
- Write vignettes and help documentation for the package's functions so far.
- Explore packages and ways to make contrast definitions easier for novel users.
- Add Scheffe contrast functionality to the R package.
- Add/ensure contrast scaling functions for appropriate interpretations of output.
Link to the Project
https://github.com/garner-code/PsyR
Image/Logo for the OHBM brainhack website
https://raw.githubusercontent.com/garner-code/PSY2R/refs/heads/main/presentations/Psy2R-logo.jpeg
Project lead
Kelly Garner
Github: garner-code
Discord: @kel-accords
Main Hub
Brisbane
Link to the Project pitch
No response
Other hubs covered by the leaders
- Brisbane
- Hybrid (Asia / Pacific)
- Hybrid (Europe / Middle East / Africa)
- Hybrid (Americas)
Skills
Coding skills in R
Package development in R
Experience with writing numerical approximation functions
Understanding of, experience in, or curiosity with multivariate statistical analysis
A desire for tools for better statistical inferences
A love of documentation and/or sensible naming of variables
Esoteric coding skills - i.e. pascal
Recommended tutorials for new contributors
https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project
https://github.com/kel-github/PSY2R/blob/main/resources/PSYHELP.pdf
https://marissabarlaz.github.io/portfolio/contrastcoding/
https://rvlenth.github.io/emmeans/reference/emmeans-package.html
https://www.youtube.com/watch?v=gfPP2pQ8Rms
Good first issues
- documentation/writing vignettes
- implementation of Scheffe contrast procedures
- analyse your own data and find our bugs!
Twitter summary
PsyR - an R package for better inference in multivariate statistical analysis. We're making a package to reduce type 1 and type 4 errors in multivariate analysis - for the people!
Short name for the Discord chat channel (~15 chars)
PsyR
Please read and follow the OHBM Code of Conduct
- I agree to follow the OHBM Code of Conduct during the hackathon