Collecting Relevant Data
The information each school collects depends on its goals, missions and context. Listed below are examples of information most schools will find useful but it is not exhaustive.
In Step 5, several possible explanations for student performance are identified. To the extent possible, it will be useful to collect data related to these explanations both to facilitate diagnoses of problems and to examine alternative ways to address the challenges they represent. To see those explanations, CLICK HERE. [Link 2a].
It is critical that data be collected in the same categories over several years (ideally, five years). Looking at data from a single year can be instructive but trend lines and variations in student learning and experiences over time is much more useful for developing effective school improvement strategies that can be sustained and built upon. For example, it is not uncommon for classroom level test data for one year to be weak predictors of student performance in the next.
When data tell stories people don’t want to hear, it common to see the source and validity of the data being questioned. Thus, it is productive to engage those who have stake in the data in its collection, when this is feasible. Of course, some data, such as survey data, can only be obtained from all of those involved. In any case, the collection process and the data themselves should be available to the school community, except when issues of personal privacy that have been agreed upon at the outset of the data collection process are concerned.
Student Learning and Engagement Related to Priority Goals
- Performance on Tests of Academic Learning
- Standardized tests
- Locally developed assessments
- Examples of student work (e.g., portfolios)
- Grades
- Aptitude tests
- Disciplinary actions
- Attendance
- Completion of expected work
- Graduation rates
- Retention in grade
- Post-graduation (from high school) experiences
Student Characteristics that Might Account for Differences in Student Performance
- Race/Ethnicity (beyond federal definitions)
- Disability
- Language facility
- Gender
- Age
- Family income (beyond “free and reduced lunch”, if possible)
- Parental education
- Mobility (how often a student moves from school to school)
- Time in your school
School Programs, Curricula, Processes and Policies that Influence Student Learning and Engagement
- Variations in curricula among students
- Alignment of curricula with assessments
- Participation in special programs (AP, IB, honors, remedial, other interventions, etc.)
- Disciplinary rules
- Library utilization
- Extracurricular activities related to school programs
- Grouping practices
- Access to technology-facilitated instruction
- Teachers’
- Experience
- Qualifications (e.g., teaching in licensed field
- Professional development related to student learning goals
- Student load
- Attendance
- Teacher aide
- Classroom composition (characteristics of classroom peers)
- Variation in instructional strategies
- Homework practices
- Parent/family engagement
Perceptions of Faculty, Staff, Families and Students
- Safety
- Caring interpersonal relationships
- Trust among colleagues, and among students and teachers
- Beliefs that teachers are committed to high achievement for all students
- Shared responsibility for student success
- Professional collaboration
- Quality of learning resources
- Instructional strategies
- Quality and helpfulness of professional development
- Parental/family engagement
Some of this information will already be available at the school or district level while other information will have to be collected from scratch and regularly updated in a variety of ways including surveys, analyses of student work, systematic observations, records of participation, and teacher and program evaluations. The KEYS survey provides data on several of the topics (variables) listed above. See the overview of the KEYS survey below.
The KEYS survey is a rich source of perception data.
KEYS collects two sets of information. The first set involves school characteristics and is provided by school administrators in response to a structured questionnaire prior to administration of the survey. To view the KEYS questionnaire about school characteristics, CLICK HERE. [link 2b] The second set of data comes from teachers, staff and parents who complete the survey (there is a different version of the survey instrument for school personnel and for parents). While the survey responses describe perceptions, many of these perceptions can be taken as solid evidence of school conditions that influence teaching and learning.
How do you decide on the specific data of each type that you will find most useful? One answer to this question is that you will only know when you get involved in the KEYS-CSI process. But, Edie Holcomb* identifies five questions that can guide the selection of the data for school improvement:
What evidence would demonstrate that we are fulfilling the commitments embedded in our mission statement?
Do we have any existing, on-going goals that lack baseline data from which to measure progress?
Is there more than one source of evidence for this decision or more than one indicator of need for this goal?
What are the assumptions we make about students and their learning? What do we need to do to verify them?
What data might help resolve smoldering issues in our school?
*E. L. Holcomb, Getting Excited about Data, Thousand Oaks, CA: Corwin Press, 2004, p.77