# AdaPT: An interactive procedure for multiple testing with side information

@article{Lei2016AdaPTAI, title={AdaPT: An interactive procedure for multiple testing with side information}, author={Lihua Lei and William Fithian}, journal={arXiv: Methodology}, year={2016} }

We consider the problem of multiple hypothesis testing with generic side information: for each hypothesis $H_i$ we observe both a p-value $p_i$ and some predictor $x_i$ encoding contextual information about the hypothesis. For large-scale problems, adaptively focusing power on the more promising hypotheses (those more likely to yield discoveries) can lead to much more powerful multiple testing procedures. We propose a general iterative framework for this problem, called the Adaptive p-value… Expand

#### 82 Citations

Contextual Online False Discovery Rate Control

- Mathematics, Computer Science
- AISTATS
- 2020

A new class of powerful online testing procedures are proposed, where the rejections thresholds are learnt sequentially by incorporating contextual information and previous results, and it is proved that any rule in this class controls online FDR under some standard assumptions. Expand

A general interactive framework for false discovery rate control under structural constraints

- Mathematics
- 2020

We propose a general framework based on selectively traversed accumulation rules (STAR) for interactive multiple testing with generic structural constraints on the rejection set. It combines… Expand

ZAP: $Z$-value Adaptive Procedures for False Discovery Rate Control with Side Information

- Mathematics
- 2021

Adaptive multiple testing with covariates is an important research direction that has gained major attention in recent years. It has been widely recognized that leveraging side information provided… Expand

AdaPT-GMM: Powerful and robust covariate-assisted multiple testing

- Mathematics
- 2021

We propose a new empirical Bayes method for covariate-assisted multiple testing with false discovery rate (FDR) control, where we model the local false discovery rate for each hypothesis as a… Expand

BONuS: Multiple multivariate testing with a data-adaptivetest statistic

- Mathematics
- 2021

We propose a new adaptive empirical Bayes framework, the Bag-Of-Null-Statistics (BONuS) procedure, for multiple testing where each hypothesis testing problem is itself multivariate or nonparametric.… Expand

Optimal False Discovery Rate Control for Large Scale Multiple Testing with Auxiliary Information

- Mathematics
- 2020

Large-scale multiple testing is a fundamental problem in high dimensional statistical inference. It is increasingly common that various types of auxiliary information, reflecting the structural… Expand

Multiple testing with the structure adaptive Benjamini-Hochberg algorithm

- Mathematics
- 2016

In multiple testing problems, where a large number of hypotheses are tested simultaneously, false discovery rate (FDR) control can be achieved with the well-known Benjamini-Hochberg procedure, which… Expand

Towards "simultaneous selective inference": post-hoc bounds on the false discovery proportion

- Mathematics
- 2018

The false discovery rate (FDR) has become a popular Type-I error criterion for multiple testing, but it is not without its flaws. Indeed, (a) controlling the mean of the false discovery proportion… Expand

Fast and covariate-adaptive method amplifies detection power in large-scale multiple hypothesis testing

- Medicine, Computer Science
- Nature Communications
- 2019

The authors develop AdaFDR, a new statistical method for multiple hypothesis testing that adaptively learns the decision threshold and amplifies the discovery power, which is computationally efficient and allows multi-dimensional covariates with both numeric and categorical values. Expand

Conditional calibration for false discovery rate control under dependence

- Mathematics
- 2020

We introduce a new class of methods for finite-sample false discovery rate (FDR) control in multiple testing problems with dependent test statistics where the dependence is fully or partially known.… Expand

#### References

SHOWING 1-10 OF 63 REFERENCES

Covariate powered cross-weighted multiple testing with false discovery rate control

- Mathematics
- 2017

Consider a large-scale multiple testing setup where we observe pairs $((P_i, X_i))_{1\leq i \leq m}$ of p-values $P_i$ and covariates $X_i$, such that $P_i \perp X_i$ under the null hypothesis. Our… Expand

Single-index modulated multiple testing

- Mathematics
- 2014

In the context of large-scale multiple testing, hypotheses are often accompanied with certain prior information. In this paper, we present a single-index modulated (SIM) multiple testing procedure,… Expand

The optimal discovery procedure: a new approach to simultaneous significance testing

- Mathematics
- 2007

The Neyman-Pearson lemma provides a simple procedure for optimally testing a single hypothesis when the null and alternative distributions are known. This result has played a major role in the… Expand

Multiple testing with the structure adaptive Benjamini-Hochberg algorithm

- Mathematics
- 2016

In multiple testing problems, where a large number of hypotheses are tested simultaneously, false discovery rate (FDR) control can be achieved with the well-known Benjamini-Hochberg procedure, which… Expand

Accumulation Tests for FDR Control in Ordered Hypothesis Testing

- Mathematics, Computer Science
- 2015

This article develops a family of “accumulation tests” to choose a cutoff k that adapts to the amount of signal at the top of the ranked list, and introduces a new method in this family, the HingeExp method, which offers higher power to detect true signals compared to existing techniques. Expand

A direct approach to false discovery rates

- Mathematics
- 2002

Multiple-hypothesis testing involves guarding against much more complicated errors than single-hypothesis testing. Whereas we typically control the type I error rate for a single-hypothesis test, a… Expand

Power of Ordered Hypothesis Testing

- Mathematics, Computer Science
- ICML
- 2016

An improved version of Selective SeqStep is derived which is analogous to Storey's improvement on the Benjamini-Hochberg procedure and has favorable performance for both good and bad prior orderings. Expand

Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach

- Mathematics
- 2004

Summary. The false discovery rate (FDR) is a multiple hypothesis testing quantity that describes the expected proportion of false positive results among all rejected null hypotheses. Benjamini and… Expand

Controlling the false discovery rate: a practical and powerful approach to multiple testing

- Mathematics
- 1995

SUMMARY The common approach to the multiplicity problem calls for controlling the familywise error rate (FWER). This approach, though, has faults, and we point out a few. A different approach to… Expand

A general convex framework for multiple testing with prior information

- Mathematics
- 2016

Using prior information may improve power in frequentist multiple testing. P-value weighting is a promising methodology where each test is conducted at a different level, using critical values based… Expand