Beginning in 2018, NISS organized a number of events: panels, presentations, discussions and even debates that focused on the topic of statistical significance and the issues surrounding these controversies.
NISS as Neutral Expert
NISS has served as an invited collaborator and consultant with the National Center for Education Statistics (NCES) for many years. The NCES collects data on the state of education via validated assessments, surveys and collections of administrative data. In September 2018 NISS convened an expert panel and published a report regarding the results that NCES should report as significant, how this significance is reported and most importantly, how it should this be explained to others. (See “Significance and Non-Significance NCES Reports”) As a follow-up to this initial report, in March 2019 NISS was again involved in convening an expert panel whose charge was to recommend how to report the importance of significant results clearly and accurately in a manner that moves away from the p < 0.05 standard in a way that is understandable to the public and acceptable to academic institutions. (See “Citing Significance in NCES Data Reporting”) The goal of both reports was twofold: to make the language and methodology of the information in the report consistent, and at the same time understandable for statisticians, policy makers, and the public.
NISS as Collaborator
NISS has also convened and hosted a series of more public panel discussions and webinars that brought to the stage individuals recognized for their contributions in this area.
In May of 2019 NISS hosted a webinar entitled, “Alternatives to the Traditional p-value.” This webinar featured Nicole Lazar (University of Georgia and Penn State), one of the editors of the ASA’s 2016 statement (mentioned above) who contended that the replicability / reproducibility crisis is something that statisticians and scientists need to address. Daniel Jeske (University of California, Riverside), having carefully reviewed each of the papers published in the special issue of The American Statistician, provided specific remarks highlighting those papers recommending alternatives to the p-value that he thought were especially intriguing. And Jessica Utts (University of California, Irvine) who provided specific advice on how to think about implementing change, sharing four recommendations for what statisticians and educators can do.
In November of 2019 a follow up webinar “Digging Deeper from the Radical to the Reasoned: p-value Alternatives Offered by Experts in NISS Webinar.” This event featured three authors whose papers were published in the special issue of The American Statistician. The titles of their talks provide a glimpse of their underlying opinions. These included Jim Berger (Arts and Sciences Professor of Statistics at Duke University), “Three recommendations to improve the use of p-values,” Sander Greenland (Emeritus Professor of Epidemiology and Statistics at the University of California, Los Angeles), “Statistics as a condemned building: A structural analysis and plans for demolition and reconstruction,” and Robert Matthews (Visiting Professor in the Department of Mathematics at Aston University, Birmingham, UK), “The analysis of credibility.”
In May of 2020 NISS hosted a third webinar that continued the conversation: “NISS Webinar Hosts Third Webinar on the Use of p-values for Making Decisions.” The speakers for this webinar were Yoav Benjamini, the Nathan and Lily Silver Professor of Applied Statistics at the Department of Statistics and Operations Research at Tel Aviv University, Alicia Laura Carriquiry, Distinguished Professor of Statistics at Iowa State University, and Hsien-Ming James Hung, Director of Division of Biometrics I, Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research (CDER), U.S. Food and Drug Administration (FDA). Again, the range of perspectives is evident in the titles of the talks. Whereas Yoav spoke on, “Replicability problems in science: It’s not the p-values’ fault”, Alicia shared an differing perspective, “P-values are rarely used in forensic science and the criminal justice system. That is too bad.” James, on the other hand, directed his comments at the role of multiplicity adjustments within regulatory applications.
NISS took a different approach to discussing p-values in October of 2020. While it is certainly interesting to get into the specific details of examples from individual presentations, NISS noticed that the interesting give and take of the question/answer time in the previous sessions always seemed too short for the number and kinds of questions that were being asked by attendees. So, Dan Jeske, moderator of The Statistics Debate devised a consistent, rolling format that provided each of the participants, Jim Berger (Duke University), Deborah Mayo (Virginia Tech), David Trafimow (New Mexico State University), with equal opportunities to be the first to respond to a question as well as respond or follow-up based on the remarks of the other participants. In fact, while the participants were given selected questions ahead of time so they had a chance to think about their responses, the group opted not to have a practice session, so their reactions to each other were extemporaneous and unscripted! The purpose of this event was to shed a spotlight on and get at the explanations of the main features (pro and con) that surround some of the controversies on the use of p-values.
NISS as Information Provider
Statistical significance is one area, which NISS has targeted in recent years through a variety of methods. Wherever possible, the resources associated with these projects or events are made public through the organization’s website. Here, you can find executive summaries or full text of papers, synopses of events, recordings of webinars as well as, with permission, copies of the slides that were used by the speakers.
NISS Affiliates are the driving force and resource for defining and implementing the direction of much of these efforts. Become a NISS Affiliate and get involved in determining the needs of tomorrow’s statisticians!