Exploring Leadership Synergies: Highlights from the COPSS-NISS Webinar on the Intersection of Statistics and Psychometrics

COPSS-NISS Leadership Webinar: Leadership at the Intersection of Statistics and Psychometrics

Date: Tuesday, November 26, 2024, 12-1pm ET/9-10am PT

The meeting focused on the intersection of statistics and psychometrics, with panelists Dr. Steven Andrew Culpepper and Dr. Sandip Sinharay sharing their journeys and perspectives on leadership. The discussion also covered the potential for stronger connections between the Psychometric Society and COPSS, the pursuit of methods that support learning for all students, and the increasing use of artificial intelligence and machine learning in education and psychometrics. The conversation ended with a discussion on future plans and potential research topics.

Leadership Webinar: Statistics and Psychometrics

Dr. Gongjun  Xu (Statistics at  and Director of Applied Statistics Master's Program at University of Michigan, and Co-Editor, Journal of Educational and Behavioral Statistics) opened the leadership webinar, discussing the intersection of statistics and psychometrics. He introduced out two panelists, Dr. Steven Andrew Culpepper and Dr. Sandip Sinharay, highlighting their professional achievements and research focus areas. Dr. Gongjun  then handed over to Dr. Culpepper to share his journey and path to leadership. The webinar was organized by NISS and COPSS, with plans for a panel discussion and audience questions at the end.

Psychometrics, Statistics, and Leadership Vision

Dr. Steven Culpepper discussed the historical overlap between psychometrics and statistics, and the potential for stronger connections between the Psychometric Society and COPSS. He shared his perspective on leadership, emphasizing the pursuit of progress with resilience, compassion, collaboration, and creativity. He highlighted his collaboration with Dr. Gongjun Xu as co-editors of the Journal of Educational Behavioral Statistics, and how they adapted to the challenges of the Covid pandemic. Dr. Steven also shared his vision for leadership at the intersection of education, psychometrics, and statistics, using the example of Jean-marc Côté's postcard from the 1910 World Exposition in Paris. He reflected on the nature of data, science, statistics, and psychometrics in education, and the potential for stronger connections between the two fields.

Pursuing Personalized Learning and Collaboration

Dr. Steven discussed the pursuit of methods that support learning for all students, not just ranking and sorting them. He emphasized the need for psychometric models, personalized learning algorithms, and strong methods for causal evidence to identify impactful interventions. Dr. Steven also highlighted the importance of creativity, collaboration, and compassion in this pursuit. He mentioned his collaborations with various experts and the development of new computational algorithms.

Dr. Sinharay's Journey in Psychometrics

Dr. Sandip Sinharay then introduced himself, sharing his educational background and how he joined the Educational Testing Service (ETS) after being encouraged by his professor, Hal Stern. Dr. Sandip Sinharay shared his journey in the field of psychometrics, highlighting key mentors and experiences that shaped his career. He credited Professor Paul Holland, Professor Shelby Haberman, Matthew Johnson, and Neil Duranz for their guidance and support. Dr. Sinharay also discussed his involvement in various publications, including editing the Handbook of Statistics, Journal of Educational Behavior and Behavioral Statistics, and Journal of Educational Measurement. He emphasized the importance of external funding and the role of colleagues like Chun Wang and Alison Chang in his success. Dr. Sinharay concluded by expressing his commitment to advancing quality and equity in education, continuing to take leadership roles, and mentoring younger colleagues. Dr. Gongjun thanked Dr. Sinharay for his insightful presentation.

Exploring Psychometrics and Interdisciplinary Research

Dr. Gongjun then initiated a discussion on the field of psychometrics and its relation to statistics. Dr. Steven emphasized the importance of finding collaborations and exploring research topics that one can contribute to. Dr. Sandip Sinharay described psychometrics as a field where statistics is applied to psychology and education, with a focus on latent variable models for individual differences. He advised students to understand the problem clearly, communicate effectively, and balance core skills with interdisciplinary research. Dr. Gongjun also asked about challenges faced by students coming from a traditional stats background in understanding psychometrics concepts, to which Dr. Sandip Sinharay responded that it's essential to understand the problem, communicate effectively, and balance core skills with interdisciplinary research.

Understanding Psychometrics and Emerging Trends

Dr. Steven emphasized the importance of understanding the jargon in the field of psychometrics to grasp the core mathematical ideas. Dr. Sandip Sinharay shared his experience of initially not knowing about item response theory and how he learned about different areas of research at ETS through discussions with colleagues and supervisors. He also highlighted the importance of finding problems to work on and publishing research to gain recognition. Dr. Sandip further discussed the need for simpler methods in practice, as complicated models may not be feasible for psychometricians. Dr. Gongjun then asked about emerging trends or research areas in the psychometrics field, to which Dr. Steven responded by mentioning the increasing use of causal methods and new designs.

AI in Education and Psychometrics

The webinar then focused on the potential applications of artificial intelligence (AI) and machine learning in education and psychometrics. Dr. Steven emphasized the importance of having a representative training sample to avoid bias in AI-based methods. Dr. Sandip Sinharay discussed the criticism of standardized tests and the need for new, less biased tests. He also mentioned the increasing use of AI in psychometrics, particularly in item development and scoring. Dr. Gongjun raised concerns about the interpretability of AI methods, and Dr. Steven suggested that identifiable models could help address this issue. Dr. Sandip Sinharay also highlighted the tension between replacing human test developers with AI and the potential for AI to produce incorrect results. The conversation ended with a discussion on the future of psychometrics, with Dr. Steven predicting an explosion of data available for analysis and Dr. Sandip Sinharay anticipating a push to reduce human work in favor of AI.

Future Plans and Research Topics

Dr. Gongjun opened the floor for questions and Dr. Steven and Dr. Sandip Sinharay discussed their future plans. Dr. Steven mentioned his plans to contribute to the psychometric society, including planning the meeting in Seoul, South Korea, and joining the meeting in Minneapolis in 2025. Dr. Sandip Sinharay shared his plans to work on personalized assessments and become the editor-in-chief of Elsevier's encyclopedia of measurement. Dr. Gongjun then asked Dr. Sandip Sinharay to provide an introduction about the types of papers that fit the journals he is associated with. Dr. Sandip Sinharay suggested looking at papers in statistical methods for psychology applications, journals like Journal of American Statistical Association, and the program book for the Psychometric Society Conference for potential research topics.

Watch Full Recording

 

Tuesday, November 26, 2024 by Megan Glenn