Conference on High-Dimensional Statistics

High-Dimensional Statistics has grown out of modern research activities in diverse fields such as science, technology, and business, aided by powerful computing. It encompasses several emerging fields in statistics such as high-dimensional inference, dimension reduction, data mining, machine learning, and bioinformatics.

John Storey

Title of Talk: Cross-Dimensional Inference of Dependent High-Dimensional Data

Abstract: A growing number of modern scientific problems in areas such as genomics, neurobiology, and spatial epidemiology involve the measurement and analysis of thousands of related features that may be stochastically dependent at arbitrarily strong levels. In this work, we consider the scenario where the features follow a multivariate Normal distribution. We demonstrate that dependence is manifested as random variation shared among features, and that standard methods may yield highly unstable inference due to dependence, even when the dependence is fully parameterized and utilized in the procedure. We propose a “cross-dimensional inference” framework that alleviates the problems due to dependence by modeling and removing the variation shared among features, while also properly regularizing estimation across features. We demonstrate the framework on both simultaneous point estimation and multiple hypothesis testing in scenarios derived from the scientific applications of interest