How do schools measure what matters? Many school leaders say they do not measure everything in their schools because “some things are not measurable.” Let’s put that idea to rest right now—all things we do in schools are measurable.
Many schools focus so much on accountability ratings and getting more students proficient that they miss how to understand their students, how to offer a better education for every student, and ultimately how to get better results. Schools try hard to move students to proficiency through interventions, tutoring, and credit recovery. However, they do not always develop a plan for school improvement that will guarantee better educational opportunities for all students.
Schools need to analyze multiple measures of data to understand where they are as a school, how they got there, and what they need to change to get better results.
Multiple Measures of Data
Schools need to review the following: demographic data to understand whom they have as students; student learning data to understand what students know and do not know, with respect to what we want them to know; perceptual data to understand what students want to do in the future and how they prefer to learn, plus gauging staff and parent perceptions of the learning environment; and school processes data to understand what processes have been implemented to obtain the results the school has been getting over time.
Looking across these four types of data, one will be able to:
- Know if the school is achieving its vision and goals.
- Understand the real reasons gaps and undesirable results exist.
- Evaluate what is working and what is not working.
- Predict and prevent failures.
- Predict and ensure successes.
Analyze Multiple Measures of Data
One high school analyzed multiple measures of data that shed light on the fact that school demographics had changed dramatically over the past four years. More students were living in poverty, were speaking English as a second language, and were mobile, with only 70 percent of students attending school 90 percent of the time. Proficiency percentages were decreasing each year. Looking at their school processes, staff concluded the school’s curriculum and instruction had not changed in seven years. The school was doing a great job of preparing the highest-achieving students for postsecondary education with AP and honors curricula. Beyond that, other students were not being prepared for the futures they desired and deserved. There were no pathway options for students. Few career tech options existed, even though most students reported on a questionnaire that they did not plan to pursue a two- or four-year academic postsecondary education. Students also stated they did not like the way they were taught and wanted school to be more fun and practical.
This comprehensive data analysis, conducted by staff, showed them how they could change their processes to provide a solid educational program that would better engage students in their learning and prepare them for productive futures.
The first step in measuring what matters in a school is to organize the school’s multiple measures of data so all staff can review the data in its entirety and quickly. A small group can do this organizing. The organization needs to begin with the most general data and move to the more specific data.
Demographic data is relatively easy to come by. Almost every school has a system that houses information about the students, the school’s staff, attendance, behavior, program participation, etc.
Student learning data, also easy to come by, includes a variety of measurements—norm-referenced tests, criterion-referenced tests, standards assessments, teacher-assigned grades, and authentic assessments—that show the impact of the education system on students. While you would review many of these data throughout the school year, when organizing student learning data for staff review of schoolwide data, hit the highlights (such as common state and standards accountability results over time disaggregated by grade level, gender, ethnicity, and other meaningful data) that will give staff pertinent information about student performance.
Perceptual data gathered through questionnaires, interviews, and observations helps you understand what students, parents, teachers, and the community think about the learning environment. People act according to what they believe about different topics, so if you want to change a group’s perceptions, you have to know about their beliefs. Make sure the instruments your school uses are valid (assess what you want them to assess).
School processes data includes the school’s programs, instructional strategies, assessment strategies, and classroom practices. School processes data are the measures over which we have the most control in the education setting. Schools cannot control who the students are, where they come from, or why they think the way they do. However, schools can control a major portion of the student learning results—through their processes (curriculum, instructional strategies, assessment practices, programs, environment). Prepare for staff analysis by making a list of the programs and processes operating in the school.
The next step is to gather staff, organized in small groups and mixed by content area and grade levels, for the analysis of data. Beginning with demographic data, have staff independently review the data in terms of what they see as strengths, challenges, and implications for a school improvement plan. Within small groups, have staff members share and chart the commonalities of their independent analyses. Then, combine the small group results to get a comprehensive set of strengths, challenges, and implications for a school improvement plan.
Diving Into Data Analysis
Use this process to analyze perceptions, school processes, and student learning data. The analysis of school processes can vary a little bit: Have staff review the list of programs and processes to make sure they are complete. Then, have staff color-code the programs and processes to determine whether each is important to the vision and the degree to which each is implemented with fidelity and integrity. Again, independent to small group to large group is a powerful approach. It is best to look at student learning results last, since this is where staff usually start (and end) their analyses.
After staff members have documented the school’s data strengths, challenges, and implications for a school improvement plan for demographics, student learning, perceptions, and school processes, and review the implications side by side. This alignment is important to help you see commonalities across the data and for staff to see the impact the other three types of data have on student learning results. The “must-haves” for a school improvement plan become very clear.
There are many benefits to this approach:
- Every staff member learns all the data.
- Staff are sharing challenges, not weaknesses, which helps them dive into hard issues.
- Staff members gain confidence in their analyses when they hear that other members of their small groups have seen the same things in the data.
- Staff members learn that together they see so much more than any one person can see alone.
- The process leads the staff to consensus.
- Most times, the real issues that need to be addressed are brought to the surface, discussed, and included in a school improvement plan.
- Because staff had a say in the suggested improvements, they will work to implement them. They see and understand the compelling “why.”
This type of schoolwide analysis is required only a couple times a year. If you can get your staff together once at the beginning of the year, you can get powerful information for the whole year. The data will help you plan to meet the needs of all students. It is rewarding to do another analysis at the end of the year to see the impact of the changes.
It takes strong leadership to inspire a shared vision and to ensure its implementation. It also takes a strong leader to ensure the analysis and use of data to verify the vision is doing what it needs to do for all students.
Victoria L. Bernhardt, PhD, is executive director of Education for the Future, a not-for-profit initiative based in Chico, CA, and the author of several data books, including Measuring What We Do in Schools: How to Know If What We Are Doing Is Making a Difference and Data Analysis for Continuous School Improvement.
Want more data on data? Tweet Victoria Bernhardt @ParadiseVickie.