The National Center for Education Statistics (NCES) charged the National Institute of Statistical Sciences (NISS) with convening a panel of technical experts to consider the issues of accounting for missing data in educational surveys. In particular, the panel was asked to address the following questions:
The goal of NAEP is to provide high-quality indicators of performance for well-defined populations of students enrolled in selected grades of U.S. schools. Under current NAEP protocols, some students with disabilities (SD) and some English language learners (ELL) may be excluded from assessment, and inclusion rates differ across states.
This white paper is intended to lay out for the National Center for Education Statistics (NCES) available algorithms and technology solutions for secure, principled statistical analysis of distributed data, as well as to identify gaps in our current understanding.
Most National Center for Education Statistics (NCES) assessments and surveys are now conducted using electronic modes. In consequence, the data captured include not only responses but also time-stamps and click-by-click chronicles of the response process. These data offer unique insight into the cognitive processes involved in test-taking. They also have potential use for automated scoring and may be useful from a psychometric point of view in evaluating item properties. NCES is now committed to making these process data available to researchers.
The National Center for Education Statistics (NCES) is continually challenged by the problems and the opportunities as data gathering and data analysis evolve with the rapidity of technological change. Problems include rising rates of non-response and increasing need to reduce response burden. An alternative for the basic design of a survey or assessment was presented for consideration to NCES with the objective of remediating the problem of decreasing response rates at all levels and simultaneously providing robust estimates of the measured outcomes and unbiased variance estimators.
Empirical analysis requires access to data. For data about important policy and management issues, information organizations (IOs) - such as government statistical agencies - are the conduit between data providers and data users. However, data confidentiality is a concern for IOs as they work to disseminate products based on collected data that contribute legitimate information to their clients - e.g., government policy makers, individuals, firms, non-governmental organizations, the media, and interest groups.
The Postsecondary Choice Project was conducted by the National Institute of Statistical Sciences (NISS) for the National Center for Education Statistics (NCES) to address existing and emerging issues of postsecondary access and choice. The project culminated in a workshop held on 26 January 2009 at NCES in Washington, DC, where the papers that make up this report were presented.
Now that most National Center for Education Statistics (NCES) assessments and surveys are conducted using electronic modes, electronic data capture means that in addition to basic background information and final responses, data includes documentation of the process of responding. These process data comprise a time-stamped, click-by-click record of each student’s progress through the assessment or survey. To date these detailed process data have not been made available; however NCES is now preparing to establish a mechanism for release of process data for research purposes.
The role of technology in education is expanding and changing. Decisions about the utilization of technology for K-12 learners are made variously by teachers, schools, and all levels of governance. Decisions are enabled by funders, by school boards and communities, by parents. At the same time, barriers to successful implementation and utilization of technology for K-12 learning exist and disparities among US learners are great.
Redesigning a survey presents a rare opportunity to consider all aspects of the survey from the perspectives of both the producer and user. The National Center for Education Statistics (NCES) charged the National Institute of Statistical Sciences (NISS) with convening a panel of technical experts to consider the issues for design of the National Postsecondary Student Aid Study.
