Capturing the Conditions and the Impacts of Technology on US K-12 Education

The role of technology in education is expanding and changing. Decisions about the utilization of technology for K-12 learners are made variously by teachers, schools, and all levels of governance. Decisions are enabled by funders, by school boards and communities, by parents. At the same time, barriers to successful implementation and utilization of technology for K-12 learning exist and disparities among US learners are great. The starting point for examining equity of opportunity or for evaluating progress or for measuring the impacts of interventions and changes must be the accurate and thorough capturing of the condition and the impact of technology in US K-12 education.

Information on technology in education is variously gathered from National Center for Education Statistics (NCES) surveys and assessments of students, teachers and schools. However, it is not immediately clear how complete a contemporary picture can be drawn from these data. The NCES recognizes the need to give greater focus to technology equity and has established the Ed Tech Equity Initiative. To support this initiative, NISS was commissioned to convene a panel of experts to understand the various needs of different NCES constituencies for information about technology in K-12 education.

The charge to the panel of experts is to think in terms of the information that is needed now and the information that will be needed in five years and in ten years so to design a framework for information that will be robust and valid over time for answering the kinds of questions about the dynamic role(s) of technology in education and about the impact of that technology on learning that school officials, education researchers and education policy makers can be expected to ask. The panel unanimously agreed that the first requirement for the Framework is that it should be of continuing merit for the next decade and beyond.

The panel met via teleconference and then on 4-5 April met in person at NCES where NCES staff presentations outlined the work of the NCES Ed Tech Initiative as a basis for the panel’s discussions.

Panel’s Observations

Observation #1: There is a chain of events from: i) setting a policy to define the purpose and expectations and funding its implementation to ii) providing the hardware, software, internet access, etc. to iii) supporting the teacher to be effective in using the technology to iv) presenting the student with a learning opportunity to v) assessing the student’s competency. Breakdown at any point in this chain leaves the end-goal - student learning - in jeopardy.

Observation #2: The purpose for deciding to utilize technology may range.

Observation #3: The metric for the value-added by technology is student outcome. Decisions affect aggregates of students; and these can be evaluated based on student data, properly subsetted.

Observation #4: Questions about technology in US K-12 education, and the data to provide the answers, will be useful as baseline and subsequent observation over a 10-year horizon if these are framed independently of the specific hardware/software/technology.

Observation #5: To be useful, a national NCES data base must also include sufficiently detailed metadata, i.e., linked demographic and education information, to allow multiple ways of subsetting.

Observation #6: In addition, the technical information about the inventoried items needs to be made part of the metadata, to include the obvious information of dates used, respondent (teacher/student/administrative record/etc.), level of decision maker (teacher/school/local school board/superintendent/state department of education/federal government), whether item was tested (compliant with OMB requirement), psychometric properties.

Observation #7: There are important comparative examinations to be made about the condition and about the impacts of technology on K-12 learning. Whether these are questions of equity (evaluation of group-to-group disparities) or of progress (comparative study over time), these can be answered credibly if the data are unbiased, accurate and sufficiently detailed to define groups precisely and to provide rich enough information for real differences to emerge clearly.

See the report for additional details regarding the panel’s observations.

Recommendations

Technology for K-12 Education is so very broad a field that no one is a complete technology expert.

Recommendation #1: A roster of experts with varied expertise could provide valuable consultation to NCES on the gathering and reporting of data on technology in/for education. Using a flexible format rather than establishing a conventional advisory body or convening an expert panel, topics or even specific questions could be addressed expeditiously by small groups with the relevant expertise. Flexibility could extend to mode of meeting (in-person, video conference, teleconference) and also duration.

For the next task at hand, the inventory of items from existing sources and then identification of information gaps, practicality demands wide knowledge of both the kinds of questions that will be asked of the data base and equally wide knowledge of potential responses across the breadth of educational settings.

Recommendation #2: The knowledge bases required to evaluate existing items in NCES data collections or to design new ones are extremely varied across topic. Both experts on the intended purpose of items and experts on the details will need to be assembled for each component of the paradigm to undertake the review planned of the compilation of existing items and the identification of gaps in each component of the paradigm.

Recommendation #3: Today K-12 learning takes place in variety of settings. While the traditional public school is the most prevalent of these, private schools, religious schools, charter and other alternative schools, schools-within-schools, and home schooling educate growing numbers of K-12 learners. It is desirable to include knowledgeable representatives of the non-traditional schools among the resource experts, for example to take part in the next task of considering the current inventory of data item.