Standardized Testing

Fact or Myth

• Students’ knowledge and skills can be assessed by the content included in standardized tests.
• High test scores of students at any particular school are reason enough to assume that the school has high student achievement and quality teaching.
• Test scores provide objective information about student achievement.
• Punishments or rewards to schools, teachers, or individual children based on their test scores motivate them to do better.
• Improvement in scores on “high stakes” tests is a sound indicator of improvement in learning.
• If it isn’t on the test, it isn’t valuable enough to spend classroom time on.
• A standardized test score is a better reflection of student learning than any other form of assessment.
• Standardized tests are the only form of assessment available.
• Moving to a district or attendance area with high test scores means high achievement and success in life for your children.

All of these statements are myths that can be debunked with sound evidence. Yet, we are at the time of year again when test scores are reported to the public and new tests are about to be taken. These test results will be used by parents and public officials to compare districts and determine the efficacy of their local school system. When the results are reported, districts will scramble to make changes in their instructional program to improve their “scores” on these assessments.

I find this interesting considering how much the individuals making these judgments and taking these actions do not know about the entire process. For example, can you answer any of the following questions?

1. Who makes the decisions regarding the items that will appear on state tests?
2. How are the items that are selected chosen?
3. Who develops the actual test questions?
4. What process is used to validate these questions?
5. Who determines what constitutes the correct answer for open-ended questions and essays?
6. What is the measurement error on this test and is that reported with the scores?
7. How are the categorical distinctions of partially proficient, proficient and advanced proficient determined? Is the identified measurement error factored into these distinctions?
8. Who actually scores the open-ended responses and essays?
9. How is consistency among those scoring these items monitored?
10. How much training and educational background is required of those who score the open-ended responses and essays?
11. Are there any incentives provided for increased quantity or rate of scoring?
12. Why are the test sections timed?
13. Is there any evidence that the scores on these tests predict future success in college or careers? Income level or eventual job satisfaction?

It seems to me that we accept test scores as an objective, valid measure of any student's success without conducting the same basic level of research we would do if we were purchasing a car or a television. That might be acceptable if the test results were used to accomplish the very limited aims they are capable of achieving. However, when decisions about a student's placement in basic skills or whether to add more time for writing and take away time from the arts, or is my child's school a good school are being made based upon state test results, don't you think we should strive to be more informed about the measure we are using to make these decisions?

My goal is to do just that. I will be conducting the third part of my Informed Parent Series in the upcoming months. The content of this presentation will be focused on helping parents and concerned citizens make informed decisions regarding the use of standardized testing in public education. We are at a time in the history of public schooling in which an informed citizenry may be the only hope for a common sense approach to providing the type of education our children need and deserve.

The Role of the Central Office

In his State of Education Address in 1987, Secretary of Education William Bennett attached the nickname "the blob" to administrators and the administrative system in public schools. The blob, he argued, is made up of people in the education system who work outside the classroom, soaking up resources and resisting reform without contributing to student achievement. In 1999, he reiterated his point in a book he coauthored. In the Educated Child, Bennett explained that the "B.L.O.B." referred to the "Bloated Education Bureaucracy" and included superintendents, district office staff, and local school board members.



Certainly one can find examples of local school district bureaucracies that stand in the way of efforts to improve schooling. But does this characterization apply to administration in general? Is district leadership really unrelated to student learning (at best) or detrimental to student learning (at worst)?



These were precisely the types of questions Marzano and Waters set out to answer in their book District Leadership that Works (2009). These researchers conducted a meta-analysis (synthesis of studies) to determine if there is a correlation between district-level leadership and average academic student achievement in a school district. Fourteen studies met the requirements for inclusion in their meta analysis. These fourteen studies included data from 1,210 school districts.



The computed correlation found was .24 and was statistically significant at the .05 level. To interpret this correlation, consider an average superintendent (50th percentile in terms of leadership skills). Also assume that this superintendent is leading a district where the average student achievement is also at the 50th percentile. Now, assume that the superintendent improves his or leadership abilities by one standard deviation (in this case, rising to the 84th percentile of district leaders). Given the correlation between district leadership and student achievement of .24, we would predict the average student achievement in the district would increase by 9.5% points. In other words, average student achievement in the district would rise to the 59.5th percentile.



These findings stand in sharp contrast to the notion that district administration is a part of an amorphous blob that soaks up valuable resoureces without adding value to a district's effectiveness. On the contrary, these findings suggest that when district leaders are carrying out their leadership responsibilities effectively, student achievement across the district is positively affected. So what specific leadership behaviors are associated with student achievement?



In response to this question, Marzano and Waters (2009) found five district-level leadership responsibilities with a statisticallys significant correlation with average student academic achievement. These responsibilities are:

1) Including central office staff, building-level administrators, board members and other relevant stakeholders in establishing nonnegotiable goals for the district.

2) Ensuring that the goal-setting process results in nonnegotiable goals (goals that all staff members must act on) in at least two areas: (a) student achievement (b) classroom instruction.

3) Creating board alignment with and support for the district's goals.

4) Monitoring progress towards the established achievement and instruction goals.

5) Allocating the necessary resources (time, money, personnel and materials) to accomplish the district's goals.



Two other findings from their study are worth noting. First, the relationship between the schools and central office should be based on defined autonomy. In other words, the central office administration expects building principals to lead within the boundaries defined by the district goals. Second, the longevity of the superintendent has a positive effect on the average academic achievement of students in the district. This is an alarming conclusion when one considers the frequency of leadership changes in school districts throughout New Jersey.



Certainly this study does not identify all of the roles and responsibilities of central office administration. However, it does provide perspective on the impact leadership can have as well as guidance for what responsibilities to emphasize as priorities.



On a separate note, during the holiday season more than ever, my thoughts turn gratefully to those who have made our progress possible. In this spirit, I say, simply but sincerely, thank you and best wishes for the holiday season and a Happy New Year!

Giving Thanks

It is the time of year when many of us gather with family and friends to give thanks for what life has blessed us with. This year, I am thankful for many things. In my professional life, I am thankful for the opportunity to wake up each morning knowing that I have the opportunity to make a difference. I believe in public education as a vehicle for creating a better future for all!

I am thankful for my colleagues. I have never worked with a group of more dedicated school administrators. Every day they come to work with the intention of making the school district a place where students can thrive.

I am thankful for our teaching staff. Teaching is a difficult profession that requires dedication, sacrifice and skill. I am proud to say that we have many professional staff members that possess these qualities.

I am thankful for our support staff. These talented, hard-working staff members are responsible for making our school district operate efficiently and effectively. They do this for the least amount of compensation and recognition in the whole school district.

I am thankful for our Board of Education. I often kid with them about why anyone in their right mind would want to volunteer for such a thankless task. They receive no compensation, are held to a higher standard than the general public, usually only hear the negative news and spend countless hours. I admire their dedication to public service and their willingness to share their viewpoints and talents.

I am thankful for the parents of students in our school district. The children you send to school each day are incredible! They are well-mannered and well-behaved. We are lucky to work in such an environment.

Thank you to all and I hope you have a wonderful Thanksgiving.

NCLB & AYP

Recently, I have had some inquiries related to the district's status regarding the No Child Left Behind Act. The inquiries have centered on the topic of our middle school being designated as a School in Need of Improvement. This blog is an attempt to explain why we have received this designation and what it means.

Each year the district's performance on the New Jersey Assessment of Student's Skills and Knowledge (grades 3-8) are disaggregated by subgroups for examination. There are 40 groups, including gender, ethnicity, socioeconomic status, etc. One of these subcategories is students with disabilities.

Of the 40 groups, two of ours did not make Adequate Yearly Progress according to the NCLB criteria. Those two categories were students with disabilities in both math and language arts. If any subgroup does not meet the Adequate Yearly Progress standard, then that school is designated as in need of improvement. This is the case, even though every other category including the total student population passes.

You might be asking, why this year for those two groups? In order for a subgroup to be reported annually, there must be 30 students in that subcategory across three grade spans. Alexandria Township has never met this requirement, so this subcategory was never reported upon. Last year, the district had one student with individualized educational plans move into the school district in grade 7. In addition, we graduated 5 students with individualized education plans. Eight students moved up from seventh to eighth grade. As a result, the district had 31 students with disabilities in grades 6,7 and 8. The previous year we had twenty-six students in these three grades.

Grade # of students 2008-2009 # of students 2009-2010
6 13 9
7 8 14
8 5 8
Totals 26 31

As a result of this increase in the subcategory of students with disabilities, the school district is now reported on in this category. If we had to report on this subcategory in any previous year, the school district would have received the designation of in need of improvement at that point in time. It is not that the average performance for the students has declined significantly, it is that the total number of students eligible to be reported on in this category has increased.

This does not mean that we do not dedicate ourselves to improving the performance of our students who have disabilities. However, as a professional educator with an extensive background in special education, I have serious reservations about using criterion-referenced standardized tests to measure the progress of our students with disabilities.

In some cases, the material on these tests is not the most important things for these students to learn. I do not expect my child study team or special education teachers to focus their instruction on improving test scores when the content is not truly what the student needs to learn. The I in the IEP stands for Individualized, a concept that seems to have been forgotten by the NCLB creators.

In some cases, the nature of the testing does not allow for our students with disabilities to present their full complement of knowledge and skills. Those who have the most difficulty with paper and pencil tasks are being required to complete an assessment that utilizes paper and pencil as the sole source of producing answers. Could our students with disabilities perform better if they were allowed to express their knowledge in other forms?

Standardized testing bases student performance on expected outcomes by chronological age. This assumes that all students can learn at the same rate of speed. Yet, brain research clearly informs educators of the developmental differences in learning. This is not to mention the clearly established differences among students with regards to the speed at which the brain processes information. I do believe that all children can learn, I just don't believe they can all learn at the same rate of speed or in the same way.

Do these flaws excuse us from expecting academic growth from our students with disabilities? Of course not! In fact, I argue that these students need more growth than their peers. That is why we have been in the process of implementing our own curriculum-based measurement program that will track the progress of students in resource center classrooms throughout the school district. Monthly we intend to use quick but reliable and valid "probes" to assess students' performance in reading, writing and math. We will use the information to inform instructional and program decisions. We are currently training our teachers in this program and anticipate having it started by the end of this school year.

I am a strong advocate of accountability for results with students. Those who know me, realize that I do not make excuses for not obtaining the results expected. However, I am also realistic and try my best to be fair minded. I do expect significant growth for all of our students, I just don't think that the best way to measure this growth is via a flawed standardized test that is taken once a year.

Two Types of Change in Schools

Since I have been an educator, and probably long before that, one of the constants of the K-12 educational system is that someone is always trying to change it. Some of the changes implemented are well thought out, well articulated, and even well researched. So why are they so short lived?

One reason may be the assumption that the magnitude of the change represented by all innovations are the same. This is definately not the case. Some changes are what is referred to as first-order change and others can be referred to as second-order change.

First-order change is incremental. It can be thought of as the next most obvious step to take in a class, school or district. Second-order change is anything but incremental. This type of change involves a radical departure from the norm.

It has been my experience that the common perspective used to address all problems in schools is to view them as first-order change issues. In other words, we tend to approach new problems from the perspective of our experiences. Unfortunately, solutions to the most pressing challenges facing our schools require a second-order perspective. We need deep, radical change to our schools if we are to maintain a competitive edge in the world.

Second-order change is never a small task. In fact, second-order change is so complex that it is not to be entered into lightly. Prestine (1992) cautions that second-order change cannot approach hesitantly. Sizer asserts that second-order change calls for decisive, swift action. Fullan (1993) states "I'm increasingly persuaded that schools that go slow and a little at a time end up doing so little that they succeed in only upsetting everything without accuring the benefits of the change."

Furthermore whether a change is considered first or second order varies based on the perceptions of individual staff members. To some, an innovation is a natural extension of everyday practice. To others, it is a radical departure from the past. For example, using computers to teach writing is easy for some, but is very scary for others.

In my most recent post, I described 21 responsibilities of the school principal. Interestingly, the importance of each responsibility varies according to the perception of the types of changes a school district is experiencing. Second-order change is associated with 7 of the 21 responsibilities I described. These include:
* Knowledge of curriculum, instruction and assessment.
* Optimizer
* Intellectual stimulation
* Change agent
* Monitoring/evaluating
* Flexibility
* Ideals/Beliefs
In other words, to be successful a school leader striving to provide leadership for a second-order change initiative must prioritize these aspects of leadership.

Unfortunately, second-order change initiatives negatively impact on four of the other principal responsibilities. These are:
* Culture
* Communication
* Order
* Input

A school leader must pay a certain price for the implementation of second-order change initiatives. Team spirit, cooperation, communication, order and routine and the level of input from all members of the staff typically deteriorate as a result of second-order change initiatives. Fullan (1993) states that "those individuals and organizations that are most effective do not experience fewer problems, less stressful situations, and greater fortune, they just deal with them differently."

To successfully implement a second-order change initiative, school leaders must ratchet up their idealism, energy and enthusiasm. Additionally, they must be willing to sustain these behaviors when confronted by frustration and even anger from some staff members. While this may take a personal toll on that school leader, this is the price we must pay for long-term progress.

The Importance of School Principals

Each day more than 53.6 million students walk into more than 94,000 K-12 schools in the hopes that the 13 years of schooling they will experience will dramatically enhance their chances of success in modern world. Indeed, evidence of income in 2001 supports these hopes. According to the U.S. Census Bureau, the earning potential of a student who graduates from high school is $19,900 compared with $11,864 for a student who does not. If the high school graduate completes college, that earning potential increases to $37,203. A master's degree increases the figure to $49,324. A doctorate or professional licensure reaches $71,606. School, then can be the door to advancement--at least financial advancement--in our complex society. For a particular school to be a launchpad to the levels of success sought by students, however, it must operate effectively.

Whether a school operates effectively or not will increase or decrease a student's chances of academic success. Marzano (2003) has shown that students in effective schools as opposed to ineffective schools have a 44% difference in their expected passing rate on a test that has a typical passing rate of 50 percent. To illustrate, consider two schools--School A and School B. Assume that the schools have a typical population of students--some with advantages in their home environment and background experiences, some with few if any advantages, most somewhere in the middle. If students in both schools take a test that has a typical passing rate of 50%, we would expect 72% of the students in the effective school to pass the test and only 28% in the ineffective school to pass. A difference of 44%.

To what extent does leadership play a role in whether a school is effective or ineffective? That is, how much of a school's impact on student achievement is due to the leadership displayed in that school? To answer this question, it is valuable to look at the results of a meta-analysis conducted by Marzano and associates (2003).

After examining 69 studies that involved 2,802 schools, they computed the correlation between the leadership behavior of the principal in the school and the average academic achievement of students in the school to be .25. To interpret this correlation, assume that the principal is hired into a district and assigned to a school that is at the 50th percentile in the average achievement of its students. Also assume that the principal is at the 50th percentile in his or her leadership ability.

Assuming the principal stays in the school for an extended period of time, the .25 correlation implies that over time, it would be expected that the average achievement of the school would remain at the 50th percentile. Interestingly, if you increase the principal's leadership behaviors by one standard deviation--from the 50th to the 84th percentile--you would expect to see the average achievement of the students in the school to rise to the 60th percentile. Increasing leadership behavior by two standard deviations--the 99th percentile--would allow us to predict student achievement to rise to 72nd percentile. Taken at face value, these findings are compelling. A highly effective school level leader can have a dramatic influence on the overall academic achievement of students. So what makes for an effective school level leader?

Examining 69 studies, looking for specific behaviors related to principal leadership has lead to the identification of 21 core responsibilities. These responsibilities and a description are:

Affirmation - Recognizes and celebrates accomplishments and acknowledges failures.
Change Agent - Willing to challenge and actively challenges the status quo.
Contingent Reward - Recognizes and rewards individual accomplishments.
Communication - Establishes strong lines of communication with and among teachers and students.
Culture - Fosters shared believes and a sense of community and cooperation.
Discipline - Protects teachers from issues and influences that would detract from their teaching time or focus.
Flexibility - Adapts his or her leadership behavior to the needs of the current situation and is comfortable with dissent.
Focus - Establishes clear goals and keeps these goals at the forefront of the school's attention.
Ideals/Beliefs - Communicates and operates from strong ideals and beliefs about schooling.
Input - Involves teachers in the design and implementation of important decisions.
Intellectual Stimulation - Ensures faculty and staff are aware of the most current theories and practices.
Involvement in Curriculum and Instruction - Is directly involved in the design and implementation of curriculum, instruction and assessment practices.
Knowledge of Curriculum, Instruction and Assessment - Is knowledgeable about current curriculum, instruction and assessment practices.
Monitoring/Evaluating - Monitors the effectiveness of school practices and their impact on student learning.
Optimizer - Inspires and leads new and challenging innovations.
Order - Establishes a set of standard operating procedures and routines.
Outreach - Is an advocate and spokesperson for the school to all stakeholders.
Relationships - Demonstrates an awareness of the personal aspects of teachers and staff.
Resources - Provides teachers with materials and professional development necessary for the successful execution of their jobs.
Situational Awareness - Is aware of the details and undercurrents in the running of the school.
Visibility - Has quality contact and interactions with teachers and students.

How does one person manage to demonstrate all of these behaviors? I am not sure they can. In my personal and professional life I have known many fine principals, and none of them could demonstrate all 21 of these characteristics. The best ones I have known were honest with themselves about their strengths and weaknesses and then used this information to delegate, compensate or improve.

Having only been a principal for a short period of time, I can tell you that I found it the most rewarding, yet most challenging job I have ever held. The rewards came from being able to make a direct impact on the lives of teachers and students. It is a job you could never understand unless you have done it yourself.

On a final note, I would like to address anonymous comments. While I certainly support and will continue to post the statements of those that disagree with my actions or decisions, I will no longer post anything that criticizes anyone else. I also wish that those who need to post a comment would consider contacting me directly. I cannot work through an issue or discuss a problem with someone when they provide feedback anonymously. It is hard to accept the validity of a comment if the person is unwilling to put their name to the statement. We are all trying to do the best we can for our students. If we keep that in mind, we can work constructively and even agree to disagree when necessary.

The Importance of Great Teachers

What influence does an individual teacher have in their classroom? Although most attempts to answer this question arrive at slightly different quantitative estimates, all researchers agree that the impact of the decisions and actions made by individual teachers is far greater than the impact of decisions made at the school or district level.

Reporting on their analysis of achievement scores from five subject areas for some 60,000 students across grades 3 through 5, Wright, Horn and Sanders (1997) note:
The most effective factor affecting student learning is the teacher. In addition, the results show wide variation in effectivness among teachers. Effective teachers appear to be effective with students of all achievement levels regardless of the level of diversity in their class.

Hayock (1998) describes the achievement difference between students who spend a year with a highly effective teacher as opposed to a less effective teacher.

On average, the most effective teachers produced gains of 53 percentage points in student achievement over one year, whereas the least effective teachers produced achievement gains of about 14 percentage points over one year. To understand these results, consider the fact that researchers estimate that students typically gain about 34 percentile points in achievement during one academic year. That is, a student who scores at the 50th percentile in mathematics in September will score at the 84th percentile on the same test given in May.

Even more startling about these numbers is the generally accepted notion that students gain about 6 percentage points simply from growing one year older and gleaning new knowledge and information through everyday life.

If the effect of attending the class of one of the least effective teachers for a year is not debilitating enough, the cumulative effect can be devastating. To illustrate, consider the findings from Haycock (1998). Over three years, students with the most effective teachers made an 83 percentile point gain. Over those same three years, students with the least effective teachers made a 29 percentile point gain. Differences of this magnitude are stunning! They can represent the differences between a "remedial label" and placement in the "accelerated" track.

So what makes for an effective teacher? One teacher level factor that affects student achievement is "instructional strategies". It is perhaps self-evident that more effective teachers use more effective instructional strategies. It is probably also true that effective teachers have more instructional strategies at their disposal. The expert teacher has acquired a wide array of instructional strategies along with the knowledge of when these strategies might be most useful.

Another teacher level factor is classroom management. It is mentioned in some form in virtually every major study of student achievement. This makes intuitive sense--a classroom that is chaotic as a result of poor management not only does not enhance student achievement, it might even inhibit it.

The final factor is "instructional planning". This may be the least addressed teacher-level factor. This lack of attention is unfortunate for two reasons. First, there is a strong and extensive research base that can be readily translated into practical suggestions and protocols for the construction of classroom curriculum. Second, many breakdowns in student learning may be a function of poor classroom curriculum design.

One of the most important responsibilities I have as the Superintendent of Schools is to hire, develop and retain the most effective instructional staff. I believe the purpose of school administration is to create the conditions under which a school staff can successfully educate all students. In a future blog post, I will describe what I believe those conditions are.

One thing I do not believe is that schools exist to meet adult needs. I value, respect and want to hire and retain staff members who understand and believe our job is to produce results with our students - whatever it takes. This is the type of person I would want teaching my child, so why would I expect anything less for your children?