Please Send Me the Research

I just finished reading Daniel Willingham’s When Can You Trust the Experts?:  How to Tell Good Science From Bad in Education.

In Part One: Why We So Easily Believe Bad Science, Willingham explains why we are susceptible to believing particular things, even if they simply aren’t true.  He also explains why a scientific approach to education policy might be appealing and why the scientific method can often fall short as an approach to education policy.  For example, the scientific method doesn’t work well if we can’t measure whether we have obtained the expected results or not, and because science cannot tell us what we should value or prioritize in education.

In Part Two: The Shortcut Solution, Willingham offers a shortcut (shorter than becoming experts in a particular area of education research) for evaluating claims for why a particular educational program, method, or policy should be adopted.  It starts with simply stripping the claim down to “If I do X, there is a Y percent chance that Z will happen.”

This restatement of the claim removes appeals to authority, science, or emotion that may sway us to believe something that isn’t true.  But it also clarifies what exactly is being claimed (both what is to be done and what the expected outcomes are) so that we have a good starting point for evaluating whether it is likely to be true, is really supported by research, or is otherwise worth or not worth doing.

Once the claim is stripped down, Willingham offers questions and research tips to help non-experts make sense of claims, including just starting by asking the Persuader, “Please send me the research.”

As we prepare for another legislative session and another large education reform proposal from Governor Branstad’s administration, I wanted to highlight the following figure and discussion (Kindle Locations 3315-30).

Student thought  <—  Teacher  <—  Principal  <—  District  <—  State

At the far left of [the figure] are the thought processes that will drive learning, understanding, enthusiasm, and so forth.  The teacher tries to create an environment that will move the student’s thoughts in particular directions.  The school administration tries to support the teacher’s efforts, or the administration tries to get the teacher to teach in ways the administration thinks is most effective.  The district does the same, influencing school administrators.  The state legislature writes laws in an effort to influence how districts and schools are administered.

The point here is to emphasize that (1) Changes in the educational system are irrelevant if they don’t ultimately lead to changes in student thought; and (2) the further the Change from the student’s mind, the lower the likelihood that it will ultimately change student learning the way that people hope.

At the state level, we are likely to be considering expansions of the Iowa Core, a switch to the Smarter Balanced Assessments, changes in the teacher career ladder and pay, and changes in length of the school day or year.  We might want to ask some questions, like what specific student outcome is supposed to result from the particular policy change under consideration?  How likely is it that the particular outcome will, in fact, result from implementing the policy change?  Are the expected results good enough and likely enough to justify the costs of making the policy change?

All in all, it was an interesting and worthwhile read.  We’re allowed to ignore appeals to authority?  Our own experience might count for something?  Teaching phonics is non-negotiable?  Sate level decision-making might not be optimal?  There may be some confirmation bias at work, but I recommend taking the time to read this book.


One thought on “Please Send Me the Research

  1. Chris Liebig

    “[A]nd because science cannot tell us what we should value or prioritize in education.”

    Bravo! But one has to wonder whether the whole reason policymakers are so fond of empirical assertions (“it’s evidence-based”!) is to conceal that fact from their listeners.

Comments are closed.