Category Archives: please send me the research

Third Grade Reading

Last month The Gazette published Between the Lines, an article with reporting on Iowa’s third grade retention law by Andrew Phillips.

One of the hardest things for me to grasp is what exactly defines a proficient third grade reader.

In Iowa, it is a third grader who can meet or exceed the benchmark or cut scores on the universal screening assessment (not the Iowa Assessments or whatever end of year accountability assessment ends up being used). These benchmark or cut scores have been set based on a prediction that a child meeting at least that score will meet a proficiency cut score on a statewide assessment. Presumably these predictions are state specific, but that isn’t entirely clear.

Iowa’s current proficiency cut scores on the Iowa Assessments are equivalent to a 41st percentile rank in the 2000 national sample. So (possibly) an Iowa third grader is a proficient reader, for purposes of the retention law, if the third grader’s performance on the universal screening assessment predicts that the child would score in the 41st percentile or higher on the Iowa Assessments as compared to the 2000 national sample. That would explain the results reported in The Gazette article Almost one in four Iowa third-graders failed new reading tests, data show. [Consider what the retention numbers might look like pegged to proficiency cut scores on the Smarter Balanced assessments. Yikes.]

If that still seems a bit abstract (and perhaps, arbitrary), The Gazette offers a look at the fluency portion of the universal screening assessment in another article, Quiz: Are you smarter than a third grader? Note that the orange, blue, and green lines mark the words a third grader would have to read to or beyond to earn a passing score on the assessment in the fall, winter, and spring assessment periods.

FAST Fluency

Are you confident that a third grader only reaching the word “blue” should be headed for retention, while a third grader reaching the word “with” shouldn’t be? I’m not.

To be fair, I don’t see any claims to the effect that the cut scores on the universal screening assessments are valid for the purposes of determining retention in third grade. See here, here, and here. And yet, we are poised to use them for retention purposes anyway. Consider what that says about state-level education leadership in Iowa.

ADDED: Current Iowa benchmark scores on universal screening assessments.

Have Fun, Start Now

Have fun, start now are the guiding principles in Dan Willingham’s latest book, Raising Kids Who Read: What Parents and Teachers Can Do.

Willingham focuses on three foundations of reading: decoding easily, comprehension, and motivation. These foundations are addressed by age groups, birth through preschool, kindergarten through second grade, and third grade and beyond. Willingham discusses what you might expect to be happening at school and what you can do at home. And, because it is Willingham, there are references to research, both what we already know and where further study could be helpful.

This book is aimed at helping you help your child to see the value and pleasure in reading for enjoyment or entertainment. Basically, kids who read have a self-concept of themselves as readers (“reading is one of the things that I do”) and Willingham offers suggestions for helping your child build that self-concept as well as positive encouragement (it’s never too late to start!).

Willingham also cautions about relying too much on your child’s school. From the Introduction:

If you want your child to value reading, schools can help, but you, the parent, have the greater influence and bear the greater responsibility. You can’t just talk about what a good idea reading is. Your child needs to observe that reading matters to you, that you live like a reader.

I found myself nodding along throughout much of this book, though there were two points at which I was stopped short. The first was at this passage, from Chapter Five:

I encourage you to be very cautious about providing reading instruction at home. There are studies showing that such teaching can help children learn to read, but in these studies, parents are trained in specific techniques by the researchers. If you’re not trained by researchers (or your child’s teacher), you’re either going to go with your gut instincts about how to teach (which is dicey) or you’ll choose one of the many products out there for parents to work on phonics with their kids. Many of these products are not sound in how they approach reading instruction, and most are terribly boring. (Emphasis in the original.)

I agree that many products (and advice) sold to parents may not be sound in how they approach reading instruction, but I am confident that that statement may too often be true with regard to materials (and advice) sold to schools as well. In other words, I’m less optimistic than Willingham that schools are mostly getting systematic phonics instruction right.

In addition, I think parents either trying to evaluate the quality of their child’s school’s reading instruction or listening to their child’s early efforts at decoding and offering effective corrective feedback may need to study up on systematic phonics instruction anyway, in which case, you’re most of the way to being prepared to teach reading to your own child (one-on-one). So while I might agree that there’s no reason to intervene if you are satisfied with your child’s reading instruction, I wouldn’t hesitate to intervene if I had concerns; just be careful who you take advice or purchase materials from.*

Case in point: here’s a portion of a literacy consultant’s blogpost, on the Iowa Reading Research Center website blog, highlighting a video, “Reading Solution: Don’t Give Me the Answer”, from the IRRC Family Resources page.

The video also shares that if a child is stuck on a word after a few tries, a parent can support the child by asking some questions. The parent in the video encourages the child to use the pictures in the book to see if the word makes sense. I decided to try this and it worked! Griffin stumbled on the word “tickets” in the sentence “Mom gets tickets.” I asked him to look at the mom and see what could be in her hand and to think about what they needed to get into the fair. Since he had figured out the first part “tic”, he was able to guess the rest of the word correctly. (Emphasis added.)

Let me suggest something different this parent could have said to her child instead. How about, “Griffin, each syllable in a word has a vowel sound. Let’s see if we can divide this word into syllables and try sounding it out again?” Having assisted Griffin in dividing “tickets” into two syllables (tick-ets), Griffin should have been able to sound out this word, not guess the word from the pictures (which, it should go without saying, won’t be a particularly good reading strategy in the long run).

The other passage that stopped me short was this one, from Chapter Six, following a discussion of the importance of broad general content knowledge:

In chapter 5, I encouraged you to count on your child’s teacher to get him reading, but when it comes to knowledge building, you can’t exhort the schools and hope for the best. This work will fall to you.

In this case, it wasn’t so much disagreement as discouragement about the general state of schools with regard to content knowledge that would cause Willingham to write this and what it means for local efforts to address the achievement gap. As we add more time to the elementary school day, I hope local school officials will take note that making time for building content knowledge (history, civics, science, art and more) is vital for supporting later reading comprehension (Chapter Six).

One other topic of particular interest to me is the role of electronic devices in reading and in schools. Willingham notes that it probably doesn’t matter if your child reads on an e-reader or not (Chapter Eight). However, in Chapter Eight, Willingham notes:

There is one qualification to that conclusion. If your child’s school is considering moving to electronic textbooks, be at least a little wary. Publishers are working to improve electronic textbooks, but with the current offerings, the research is pretty consistently negative.

Willingham discusses digital technology more extensively in Chapter Ten:

The consequence of long-term experience with digital technologies is not an inability to sustain attention. It’s impatience with boredom. It’s an expectation that I should always have something interesting to listen to, watch, or read and that creating an interesting experience should require little effort. . . . We’re not distractible. We just have a very low threshold for boredom.

But it’s not all bad news (e-readers can help make appealing reading material easily accessible) and Willingham offers a number of suggestions to encourage kids to choose to read.

All in all, a worthwhile and engaging read, and a book I’d like to see local teachers and school administrators find time to read.

*FWIW, I like the Montessori preschool writing to reading sequence of materials and activities. I also like Sound Steps to Reading (plus storybook) by Diane McGuinness.

Research Shows: Preschool Edition 2

The Iowa House Democrats are back again with more preschool research in their March 15th newsletter.  Here’s the key paragraph from the newsletter:

A new cost-benefit analysis indicates that up to $16 is returned long-term for every $1 invested in a high-quality early childhood program. Although, the cost of preschool is generally about $3,500 to $4,000 per child annually, big long-term payoffs on their initial investment will benefit taxpayers as well as student achievement.

Apparently, all preschool research leads back to the Perry Preschool Project, which is one of the programs used for this cost-benefit analysis.  But here’s what is missing from the House Democrats article:

  • The “payoff” or return comes from reduced spending later on special education, remedial education, fewer arrests, and fewer social social services.  Which means, presumably, that there isn’t much of a return from paying for preschool for children who are at low risk for needing special or remedial education services or social services, or are unlikely to be arrested.
  • The economist who presented this information at the capitol, Rob Grunewald, notes that “Based on costs used in previous studies and current programs for at-risk children, we estimate that total resources needed to fund an annual scholarship for a high-quality early-childhood-development program for an at-risk 3- or 4-year old would be about $10,000 to $15,000 for a full-day program that included parent mentoring.”  It isn’t clear that spending $3,500 to $4,000 per child for ten hours per week of preschool will generate the same returns as more intensive (and thus more expensive) programs.

In other words, this research doesn’t support expanding a universal, ten hour per week preschool program.  It supports funding a carefully targeted, intensive preschool program for at-risk children.

In a related vein, this week Scott McLeod takes on the Iowa DE and “evidence-based” educational policy.

 

Research Shows: Preschool Edition

The March 1, 2013 Iowa House Democrats newsletter includes the following item, titled “Research Shows Positive Effects of Preschool.”

With legislators considering an expansion of early childhood education this year, lawmakers received more research showing the positive effects of preschool for 3 and 4 year-old children from Early Childhood Iowa. Some of the highlights of the research include:

        • Studies show that a universal, high-quality preschool program would increase the employment rates of state residents by 1.3%.
        • A high-quality pre-kindergarten program that served both 3 and 4-year-olds would yield relatively quick budgetary savings and would begin to pay for itself through reduced special education costs and reduced juvenile justice costs.
        • Within 42 years, the total benefits of such a program, including reduced crime rates, higher income earnings by participants, higher tax revenues, savings from reduced grade retention and special education usage, would outweigh costs by a ratio of 8.4 to 1.

As usual, there is no research cited for these claims, but I think they probably refer to the Perry Preschool Project, a preschool program trial conducted in the 1960s involving 128 children (64 assigned to the control group and 64 assigned to the treatment group).

I am skeptical of claims promising that spending one dollar now will save us a specific number of dollars in the future.  Even so, there is no reason to think that the results of this particular research program (involving disadvantaged children) mean that universal preschool programs (involving all children) can produce these same results.  And there is no reason to think that the results of this particular research program (involving an average child-teacher ratio of 6:1 and 1.5 hour weekly home visits) mean that less intensive, universal preschool programs can produce these same results.

See the evidence of the effectiveness of the Perry Preschool Project at the Coalition for Evidence-Based Policy here, and note the disclaimer “that this was a demonstration project, and it is not yet known if the results can be replicated on a broader scale in typical classroom settings.”

There might be reasons to support expanded access to preschool (and reasons not to–like getting an earlier start on PBIS and high-stakes testing), but this research doesn’t seem to be one of them.

 

Cookbook Labs

In a recent post, Nicholas J lamented the requirement of the use of “humdrum labs” in his AP Chemistry class that have “shockingly predictable results.”

Nicholas J is not alone.  From the drafters of the Iowa Model Core:

The depth of understanding required of our students is not possible with lectures, readings, cookbook labs, and plug-and-chug problem solving. Students must be actively investigating: designing experiments, observing, questioning, exploring, making and testing hypotheses, making and comparing predictions, evaluating data, and communicating and defending conclusions.

Because of the Blogathon and, mostly, because the certainty of the phrase not possible is provocative, I offer this defense of the cookbook lab.

Father M liked to issue frequent grim reminders to us that “physics is fun,” while Mr. G chose to impress us early on with the dangers–eyewash station and emergency shower kind of dangers–inherent in the chemistry lab.

He wowed us with the sodium in water demonstration then warned us with the cautionary tale of the boy who tried to sneak sodium out of the chemistry lab only to burn a hole in his pants when he started to sweat.

Awesome.

In retrospect, I think that high school chemistry teachers must be made of stern stuff to venture into the laboratory with novice chemists year after year.

Cookbook labs provide students the opportunity to develop technical laboratory skills: how to measure with precision, how to work with laboratory glassware, and how to work with an open flame without lighting your sleeve on fire or singeing your hair.

Cookbook labs provide students with the opportunity to develop scientific habits: starting with a hypothesis, working with care and precision, making and recording observations, and writing lab reports.  In other words, the opportunity to do everything on the Model Core list except designing.

As a practical matter, the true virtue of a cookbook lab is that it can be reliably completed within the time allotted with predictable results.  That is to say, that the predictability of the cookbook lab is a virtue.

Why would that be so?

In The War Against Grammar (blogged about previously in this post), David Mulroy writes:

There is something to be said for hard-nosed formal instruction–for rote learning.  Knowing specified rules and definitions gives students autonomy.  When they are right, they are right.  They do not have to rely on a teacher’s subjective approval.

I think cookbook labs provide students of science a measure of autonomy too.  They can prove to themselves, through achieving the predicted results, that the things they have learned about the chemical properties of the elements and chemical reactions (or laws of physics for that matter) are true–not because their teacher or the textbook said so–but because they have been able to replicate the predicted results and observe it for themselves.

Keep in mind, this is how real scientists work too.  Advances in scientific knowledge are not accepted until the findings can be reliably replicated by someone else.

Please Send Me the Research

I just finished reading Daniel Willingham’s When Can You Trust the Experts?:  How to Tell Good Science From Bad in Education.

In Part One: Why We So Easily Believe Bad Science, Willingham explains why we are susceptible to believing particular things, even if they simply aren’t true.  He also explains why a scientific approach to education policy might be appealing and why the scientific method can often fall short as an approach to education policy.  For example, the scientific method doesn’t work well if we can’t measure whether we have obtained the expected results or not, and because science cannot tell us what we should value or prioritize in education.

In Part Two: The Shortcut Solution, Willingham offers a shortcut (shorter than becoming experts in a particular area of education research) for evaluating claims for why a particular educational program, method, or policy should be adopted.  It starts with simply stripping the claim down to “If I do X, there is a Y percent chance that Z will happen.”

This restatement of the claim removes appeals to authority, science, or emotion that may sway us to believe something that isn’t true.  But it also clarifies what exactly is being claimed (both what is to be done and what the expected outcomes are) so that we have a good starting point for evaluating whether it is likely to be true, is really supported by research, or is otherwise worth or not worth doing.

Once the claim is stripped down, Willingham offers questions and research tips to help non-experts make sense of claims, including just starting by asking the Persuader, “Please send me the research.”

As we prepare for another legislative session and another large education reform proposal from Governor Branstad’s administration, I wanted to highlight the following figure and discussion (Kindle Locations 3315-30).

Student thought  <—  Teacher  <—  Principal  <—  District  <—  State

At the far left of [the figure] are the thought processes that will drive learning, understanding, enthusiasm, and so forth.  The teacher tries to create an environment that will move the student’s thoughts in particular directions.  The school administration tries to support the teacher’s efforts, or the administration tries to get the teacher to teach in ways the administration thinks is most effective.  The district does the same, influencing school administrators.  The state legislature writes laws in an effort to influence how districts and schools are administered.

The point here is to emphasize that (1) Changes in the educational system are irrelevant if they don’t ultimately lead to changes in student thought; and (2) the further the Change from the student’s mind, the lower the likelihood that it will ultimately change student learning the way that people hope.

At the state level, we are likely to be considering expansions of the Iowa Core, a switch to the Smarter Balanced Assessments, changes in the teacher career ladder and pay, and changes in length of the school day or year.  We might want to ask some questions, like what specific student outcome is supposed to result from the particular policy change under consideration?  How likely is it that the particular outcome will, in fact, result from implementing the policy change?  Are the expected results good enough and likely enough to justify the costs of making the policy change?

All in all, it was an interesting and worthwhile read.  We’re allowed to ignore appeals to authority?  Our own experience might count for something?  Teaching phonics is non-negotiable?  Sate level decision-making might not be optimal?  There may be some confirmation bias at work, but I recommend taking the time to read this book.