Category Archives: gatekeepers

Third Grade Reading

Last month The Gazette published Between the Lines, an article with reporting on Iowa’s third grade retention law by Andrew Phillips.

One of the hardest things for me to grasp is what exactly defines a proficient third grade reader.

In Iowa, it is a third grader who can meet or exceed the benchmark or cut scores on the universal screening assessment (not the Iowa Assessments or whatever end of year accountability assessment ends up being used). These benchmark or cut scores have been set based on a prediction that a child meeting at least that score will meet a proficiency cut score on a statewide assessment. Presumably these predictions are state specific, but that isn’t entirely clear.

Iowa’s current proficiency cut scores on the Iowa Assessments are equivalent to a 41st percentile rank in the 2000 national sample. So (possibly) an Iowa third grader is a proficient reader, for purposes of the retention law, if the third grader’s performance on the universal screening assessment predicts that the child would score in the 41st percentile or higher on the Iowa Assessments as compared to the 2000 national sample. That would explain the results reported in The Gazette article Almost one in four Iowa third-graders failed new reading tests, data show. [Consider what the retention numbers might look like pegged to proficiency cut scores on the Smarter Balanced assessments. Yikes.]

If that still seems a bit abstract (and perhaps, arbitrary), The Gazette offers a look at the fluency portion of the universal screening assessment in another article, Quiz: Are you smarter than a third grader? Note that the orange, blue, and green lines mark the words a third grader would have to read to or beyond to earn a passing score on the assessment in the fall, winter, and spring assessment periods.

FAST Fluency

Are you confident that a third grader only reaching the word “blue” should be headed for retention, while a third grader reaching the word “with” shouldn’t be? I’m not.

To be fair, I don’t see any claims to the effect that the cut scores on the universal screening assessments are valid for the purposes of determining retention in third grade. See here, here, and here. And yet, we are poised to use them for retention purposes anyway. Consider what that says about state-level education leadership in Iowa.

ADDED: Current Iowa benchmark scores on universal screening assessments.

Advertisements

Cookbook Labs

In a recent post, Nicholas J lamented the requirement of the use of “humdrum labs” in his AP Chemistry class that have “shockingly predictable results.”

Nicholas J is not alone.  From the drafters of the Iowa Model Core:

The depth of understanding required of our students is not possible with lectures, readings, cookbook labs, and plug-and-chug problem solving. Students must be actively investigating: designing experiments, observing, questioning, exploring, making and testing hypotheses, making and comparing predictions, evaluating data, and communicating and defending conclusions.

Because of the Blogathon and, mostly, because the certainty of the phrase not possible is provocative, I offer this defense of the cookbook lab.

Father M liked to issue frequent grim reminders to us that “physics is fun,” while Mr. G chose to impress us early on with the dangers–eyewash station and emergency shower kind of dangers–inherent in the chemistry lab.

He wowed us with the sodium in water demonstration then warned us with the cautionary tale of the boy who tried to sneak sodium out of the chemistry lab only to burn a hole in his pants when he started to sweat.

Awesome.

In retrospect, I think that high school chemistry teachers must be made of stern stuff to venture into the laboratory with novice chemists year after year.

Cookbook labs provide students the opportunity to develop technical laboratory skills: how to measure with precision, how to work with laboratory glassware, and how to work with an open flame without lighting your sleeve on fire or singeing your hair.

Cookbook labs provide students with the opportunity to develop scientific habits: starting with a hypothesis, working with care and precision, making and recording observations, and writing lab reports.  In other words, the opportunity to do everything on the Model Core list except designing.

As a practical matter, the true virtue of a cookbook lab is that it can be reliably completed within the time allotted with predictable results.  That is to say, that the predictability of the cookbook lab is a virtue.

Why would that be so?

In The War Against Grammar (blogged about previously in this post), David Mulroy writes:

There is something to be said for hard-nosed formal instruction–for rote learning.  Knowing specified rules and definitions gives students autonomy.  When they are right, they are right.  They do not have to rely on a teacher’s subjective approval.

I think cookbook labs provide students of science a measure of autonomy too.  They can prove to themselves, through achieving the predicted results, that the things they have learned about the chemical properties of the elements and chemical reactions (or laws of physics for that matter) are true–not because their teacher or the textbook said so–but because they have been able to replicate the predicted results and observe it for themselves.

Keep in mind, this is how real scientists work too.  Advances in scientific knowledge are not accepted until the findings can be reliably replicated by someone else.

Leave It to the Professionals

I intended to set aside the issue of local control today and watch presentations and panel discussions from the Iowa Teacher and Principal Leadership Symposium.  So, I started with Marc Tucker’s presentation and the panel discussion that followed it (find them here).

Marc Tucker tells us that the only thing that matters is matching the achievements of the top performing countries.  Our students need to be at least as well educated but more innovative and creative.  So we’ll need to apply the top standards to every student.  Oh, and we’ll have to supply this elite level education to all students for less money.  This will require a new system of public education, but Tucker has it all figured out.

All we need to do is (1) implement world class standards, curriculum, and assessments [whatever those might be, he doesn’t say except to disparage computer scored, multiple choice tests], (2) hire the highest quality teachers, (3) pay them and train them like we pay and train engineers, (4) end the blue collar system of employment to be replaced by professional pay and working conditions, (5) do the same for principals, (6) start out students right and keep them on trajectory, (7) spend more on learning and less on management [apparently professionals don’t need management], (8) create a coherent, aligned system [local control is so not world class], and (9) implement, implement, implement.

He also noted that there is no evidence from PISA that more money, smaller class sizes, IT, choice, or market forces contribute to world-class student performance.

There is certainly a lot to talk about here, but I think I will comment more generally that there seems to be a sentiment underlying the move toward centralizing decision making that state level decision makers are both more competent in some sense and that some sort of data-driven/scientific management approach is required.

So, perhaps there are two points that can’t be repeated too often.

The first is that state-level decision makers have no special or superhuman ability to resist slick promotional materials and too-good-to-be-true sales pitches.  [My favorite example is the ongoing obsession with virtual reality—see here and here.]

The second is that evidence and science can only take us so far.  The evidence may have convinced Tucker that matching the achievements of the top performing countries is the only thing that matters, but it doesn’t prove that we would be wrong to prefer other goals for Iowa’s public schools.

Gatekeepers

Sometimes you stumble across just the right word that helps to crystallize a thought.

I previously wrote this about The Blueprint:

The Blueprint offers an assortment of fixes such as higher teacher pay, teacher mentors, Common Core standards, and new assessments.  None of these changes can substantially improve student achievement if built upon our current approaches to reading and mathematics instruction.

But in a recent post Dan Willingham provided me with just the right word for the idea I was trying to get at, which is gatekeeper.

As Dan Willingham explains, in the post and in the comments, teachers are too busy teaching to keep up with all possible research in all possible fields that may be relevant to their teaching (and, in fact, it is ridiculous to expect them to develop sufficient expertise in all potentially relevant fields such that they could evaluate whether the research and claims based on the research were sound or constitute “sham science.”)

 Teachers don’t need to learn neuroscience, or better put, teachers shouldn’t need to learn neuroscience–not to be protected from charlatans. Teachers need to learn things that will directly help their practice. Charlatan protection ought to come from institutions: from schools of education, from district central offices, and (potentially) from institutions of teachers’ own creation.

Dan Willingham also notes:

[T]here is virtually always someone in the district central office who is meant to be the resource person for professional development: is this PD session likely to be legit, or is this person selling snake oil?  If teachers are exposed to PD with sham science, the right response, it seems to me, is not to suggest that teachers learn some neuroscience. The right response is outrage directed at the person who brought the knucklehead in there to do the PD session.

I think the situation is even more complicated than Dan Willingham suggests, in that, at least in Iowa, the gatekeepers may often also be the policy makers.  This means that if the gatekeepers make a mistake in distinguishing the legitimate from the snake oil, the snake oil may be incorporated into policy.  In other words, even if teachers get better information on what is or isn’t legitimate from another source, they may still be required to implement snake oil in the classroom.

So, I think now what I would say is this:  instruction in Iowa will only be as good as our gatekeepers.

Before we focus so heavily on “fixing” the teachers, we might want to ask how well the gatekeepers in our colleges of education, our district central offices, our Area Education Agencies, and our Department of Education are protecting teachers (and their students) from charlatans?  And if they aren’t protecting teachers from charlatans, how are we going to fix that?

ADDED:  Improving Teachers:  Millions Spent, But Little Done to Make Sure It’s Working

HT: Superintendent Murley

Best Practices

Jay P. Greene has written about the pitfalls of relying on a best practices approach to crafting education policy in his review of Surpassing Shanghai: An Agenda for American Education Built on the World’s Leading Systems.  In a nutshell, the best practices approach is to make education policy recommendations based on imitating the policy choices of high-performing education systems.

The fundamental flaw of a “best practices” approach, as any student in a half-decent research-design course would know, is that it suffers from what is called “selection on the dependent variable.” If you only look at successful organizations, then you have no variation in the dependent variable: they all have good outcomes. When you look at the things that successful organizations are doing, you have no idea whether each one of those things caused the good outcomes, had no effect on success, or was actually an impediment that held organizations back from being even more successful.

Anyone following the Iowa education reform debate in the past year, will recognize this as the Branstad-Reynolds approach to world-class schools: choose policy recommendations imitating some (but not all) of the polices in higher-performing systems such as Finland, Florida, and Massachusetts.

The best practices discussion reminded me of a discussion about “cargo cult education” at kitchen table math.  [Click over and read it for an explanation, including the comments.]  Essentially, we can faithfully imitate features or behaviors of successful systems (or “good readers” or “good writers”) without achieving the desired result if we aren’t careful about understanding why the imitated feature or behavior seems to be working for the higher-performing system, school, reader, or writer.

For example, like Finland, we could be more selective about who enters teacher training programs in Iowa, but if we don’t bother to ask why selectivity seems to be working for Finland, we might have disappointing results.  That is, maybe we can replicate Finland’s success by implementing a 3.0 GPA requirement, but maybe, for starters, we should be asking questions like what preparation do prospective Finnish teachers have prior to applying for teacher training?  How do Finnish teacher training programs differ from Iowa teacher training programs?  Do Finnish teachers have more authority to make decisions about instruction and curriculum in their own classrooms than Iowa teachers do?  If so, is teacher autonomy an essential factor, along with selectivity, in Finnish success?  And so on.

In short, we need to be as appropriately skeptical about best practices as we are with other education research.  See Chris at A Blog About School for more on this topic here and here.