Working off-campus? Learn about our remote access options

British Journal of Mathematical and Statistical Psychology
Original Article

A note on Type S/M errors in hypothesis testing

Jiannan Lu

Corresponding Author

E-mail address: jiannl@microsoft.com

Microsoft Corporation, Redmond, Washington, USA

Correspondence should be addressed to Jiannan Lu, One Microsoft Way, Redmond, WA 98004, USA (email: jiannl@microsoft.com).Search for more papers by this author
Yixuan Qiu

Purdue University, West Lafayette, Indiana, USA

Search for more papers by this author
Alex Deng

Microsoft Corporation, Redmond, Washington, USA

Search for more papers by this author
First published: 23 March 2018
https://doi.org/10.1111/bmsp.12132
Citations: 7

The first two authors contributed equally to this work. This majority of the work was conducted when the second author was a research intern at Microsoft Corporation.

Get access to the full version of this article. View access options below.
Institutional Login
Loading institution options...
BPS Members

BPS Members: Click here to login

If you have previously obtained access with your personal account, .

Purchase Instant Access
    • View the article PDF and any associated supplements and figures for a period of 48 hours.
    • Article can not be printed.
    • Article can not be downloaded.
    • Article can not be redistributed.
    • Unlimited viewing of the article PDF and any associated supplements and figures.
    • Article can not be printed.
    • Article can not be downloaded.
    • Article can not be redistributed.
    • Unlimited viewing of the article/chapter PDF and any associated supplements and figures.
    • Article/chapter can be printed.
    • Article/chapter can be downloaded.
    • Article/chapter can not be redistributed.
Check out

Abstract

Motivated by the recent replication and reproducibility crisis, Gelman and Carlin (2014, Perspect. Psychol. Sci., 9, 641) advocated focusing on controlling for Type S/M errors, instead of the classic Type I/II errors, when conducting hypothesis testing. In this paper, we aim to fill several theoretical gaps in the methodology proposed by Gelman and Carlin (2014, Perspect. Psychol. Sci., 9, 641). In particular, we derive the closed‐form expression for the expected Type M error, and study the mathematical properties of the probability of Type S error as well as the expected Type M error, such as monotonicity. We demonstrate the advantages of our results through numerical and empirical examples.