Rxivist logo

Analyzing Nested Experimental Designs - A User-Friendly Resampling Method to Determine Experimental Significance

By Rishikesh U Kulkarni, Catherine L Wang, Carolyn R Bertozzi

Posted 30 Jun 2021
bioRxiv DOI: 10.1101/2021.06.29.450439

While hierarchical experimental designs are near-ubiquitous in neuroscience and biomedical research, researchers often do not take the structure of their datasets into account while performing statistical hypothesis tests. Resampling-based methods are a flexible strategy for performing these analyses but are difficult due to the lack of open-source software to automate test construction and execution. To address this, we report Hierarch, a Python package to perform hypothesis tests and compute confidence intervals on hierarchical experimental designs. Using a combination of permutation resampling and bootstrap aggregation, Hierarch can be used to perform hypothesis tests that maintain nominal Type I error rates and generate confidence intervals that maintain the nominal coverage probability without making distributional assumptions about the dataset of interest. Hierarch makes use of the Numba JIT compiler to reduce p-value computation times to under one second for typical datasets in biomedical research. Hierarch also enables researchers to construct user-defined resampling plans that take advantage of Hierarch's Numba-accelerated functions. Hierarch is freely available as a Python package at https://github.com/rishi-kulkarni/hierarch.

Download data

  • Downloaded 683 times
  • Download rankings, all-time:
    • Site-wide: 49,401
    • In neuroscience: 6,921
  • Year to date:
    • Site-wide: 9,816
  • Since beginning of last month:
    • Site-wide: 18,199

Altmetric data

Downloads over time

Distribution of downloads per paper, site-wide