Analysis of Evidence
The result of the initial analysis should be shared with the entire school staff. As problems are identified, those in a position to address them should do their own analysis of the data. For example, achievement gaps in fourth grade reading should probably be addressed by fourth and third grade teachers.
Conclusions reached should be descriptive, not judgmental and, while the analysis may suggest solutions, it is important to resist the instinct to leap at ways to narrow the gaps between goals and evidence of student learning and development. Problem identification and priority setting now; problem solving in Step 5.
In addition to simple tables and cross tabulations, graphical arrays are usually helpful in analyzing comparative data. For examples of tool for presenting data, CLICK HERE [link 4b]
There is no single way to examine the evidence. Here are some guidelines:
The primary goal is to identify where students are falling short of achieving goals and to be as specific as possible. Which goals, which students?
It is also important to identify successes so that these are recognized and rewarded and so they can be examined for possible lessons on how to solve problems.
Conclusions reached should be descriptive; no judgments about causes of short comings or successes at this point.
Analyses should be collaborative involving those who will be responsible for any improvements needed.
Statistical significance is not the standard for identifying successes or shortcomings. Professional discussion and judgment will suffice. Sometimes important information is overlooked if analyses become too technical.
Look for “interaction effects”. Insights may depend on looking at how “sub-sub-groups” perform. For example, breaking data down by race and gender so that one can look at how African American males compare with African American females may help in identifying issues to be addressed.
Be mindful of the difficulties of communication as different analyses are discussed.
Determine which problems deserve highest priority as major initiatives. The analyses undertaken in this Step will probably identify many student outcomes that need to be addressed. Of course, goals for which schools are held accountable and need to be addressed to forestall sanctions will be high priority. But, in highly effective schools addressing performance gaps related to other goals will be even more important to school staff. Driving school improvement primarily to avoid sanctions is not likely to stimulate ambitious initiatives that can sustain the commitment of substantial effort over time.
To learn more about using student data for school improvement CLICK HERE: [link 4c]
In most schools, the thorough examination of shortcomings illuminated by the analysis of student performance will not be without tension and conflict as valued goals are not met and apparent differences the effectiveness of individuals and groups within the school are implied. Indeed, it is important to the success of improvement efforts that different perspectives are encouraged and respectfully considered, especially around issues about which people feel strongly. In Step 3, the importance of developing trusting and collaborative school cultures was emphasized and that discussion might be revisited. For some suggestions about how facilitate communication and deal with conflict in the context of decisions making, CLICK HERE [link 4d]
NCREL, Guide to Using Data in School Improvement Efforts, December 2004. This is online document identifies the several steps involved in using data for school improvement. Note that the model for data based decision making is consistent with but different from the model developed by Hawley and Sykes that is in the learning resources for KEYS-CSI, Step 2. This web site, in which this document is presented, has a great deal of related information that will be useful to school leaders seeking to use data to facilitate school improvement. Available at www.ncrel.org/datause/howto/guidebook.php
National Staff Development Council, Guidelines for Analyzing Data, 2001. Ten propositions that outline do's and don’ts of analyzing data in collaborative settings. [link 4f]
National Education Association, 2004, A Process for Analyzing Data & Analysis Leading to Action. These two brief (one page each) sets of suggestions may help in organizing and drawing conclusions from your analysis of student performance. [link 4g]