Category Archives: math

Smarter Balanced Assessments 30

Cut scores and proficiency labels.

SBAC announced cuts scores for each of the four achievement levels that will be used for reporting student performance on the Smarter Balanced Assessments earlier this week.

Education Week has coverage of the cut score announcement, the process used for setting the cut scores, and concerns raised about whether cut scores should have been set using only field-test data (PARCC will set cut scores after administering the operational test).

“It’s really bizarre to set cut scores based on field-test data,” said one state education department psychometrician. “You can’t possibly project” accurately what proportions of students will score at the four levels of the test. He and other assessment experts said that field-test data are not good predictors of performance on the operational test because students are unfamiliar with the test, and often, teachers have had less experience teaching the material that’s being tested.

And students might lack motivation to do their best on a field test, experts said.

Education Week also has coverage about the debate over use and reporting of test scores, particularly reporting test scores in performance (achievement-level) categories as opposed to reporting scale scores. Vermont abstained from voting to set SBAC cut scores and outlined concerns about the use of performance categories, and the lack of empirical evidence for the cut scores, in a memo to SBAC governing states. SBAC covered similar ground in a document titled Interpretation and Use of Scores and Achievement Levels.

Even though the predictions about student performance on the operational assessments may be flawed (based on field-test data only, Iowa students may outperform–or underperform–the multi-state averages), I thought it would be interesting to compare predicted performance on the Smarter Balanced Assessments to reported proficiency data from the Iowa Assessments.

I used SBACs performance predictions for Levels 3 and 4 (proficient) and Level 4 (college content ready at 11th grade) and Iowa Assessments intermediate and high performance levels (proficient) and the high performance level from the 2011-13 biennium (the most recent data reported by the state of Iowa). I chose grades four, eight, and eleven because those are the levels reported in The Annual Condition of Education Report (see pages 176-181).

Here’s how reading proficiency rates could change (predicted SBAC performance versus 2011-2013 Iowa Assessment performance):

Reading 2

Here’s how math proficiency rates could change (predicted SBAC performance versus 2011-2013 Iowa Assessment performance):

Math 2

Have Iowa proficiency standards been set too low? Are the Smarter Balanced Assessments proficiency standards set too high? Are the predictions grossly inaccurate? Who knows, but if the Iowa Legislature chooses the Smarter Balanced Assessments we had better be prepared for much lower reported proficiency rates, at least in the early years.

Just for fun: draft assessment tasks for the Next Generation Science Standards from Achieve (HT: Education Week). Iowa is expected to adopt the Next Generation Science Standards. If the Iowa Legislature chooses the Smarter Balanced Assessments, Iowa will need to choose a separate science assessment.

Just for fun 2: the State Board of Education will be submitting a recommendation about assessments to the Iowa Legislature. In a demonstration of minimal transparency, the State Board agenda for the November 19th meeting lists “assessment” as an agenda item. Tab M adds the following helpful background description:

This is a continuation of the conversation on the State Board’s priority on assessment. Possible ideas for Board positions and recommendations on assessment will be provided. Opportunities for interaction around these topics will be provided throughout.

I guess we’ll just have to wait for the minutes to be posted following the January 22, 2015 meeting to learn more.

New Iowa Core Website

The Iowa Department of Education has launched a new Iowa Core website.

The website includes a series of parent’s guides to the Iowa Core for kindergarten through grade eight, and high school.

The educator resources page includes links to Iowa Core mathematics support materials, which include documents explaining the Iowa Core mathematics content shifts from prior practice for high school and grades six through eight, and content and practice shifts for kindergarten through grade five. At first glance, these documents look more accessible than reading the standards themselves, at least for understanding how things might be changing under the Iowa Core, as well as for understanding how the Iowa DE is interpreting the the common core standards.

The educator resources also include a link to the IowaLearns digital library of teaching and learning resources, thousands of which are available to the general public through a guest login.

With the inclusion of an Iowa Core Spotlight containing statements from Iowa Core supporters, the new website is clearly intended to promote a positive view of the Iowa Core, but it does seem to be an improvement over prior websites in terms of pulling Iowa Core resources together, in a way that they may be more easily found.



Math and Reading are Fundamental

Jack has a new, thought- and blog post-provoking blogpost entitled Why Math & Reading are Overrated.

I read the post largely as a call for broader assessment of students, both in rating schools and college admissions. I understand Jack’s points, but I would like to suggest that there are good reasons for maintaining a narrow focus on math and reading in large-scale assessments.

Math and reading are fundamental, necessary if not sufficient for success in a broad range of academic pursuits. To deprive children of the opportunity to master reading and elementary mathematics is to severely limit their future options, both in school and in the workforce. To use Jack’s example, weak public speaking skills aren’t likely to prevent a person from successfully completing an engineering degree, but weak math skills certainly will.

I’m no longer sure that there are any other academic subjects or skills, except to some extent writing, for which this is more or less universally true. That is, for any other academic subject, we can identify many successful people with little knowledge of it–geography, history, government, science, art, music, literature, or foreign languages.

Moreover, if something is hard to measure well in a large-scale assessment, we certainly shouldn’t be measuring it poorly with high stakes attached to the results, no matter how highly we value the thing that is hard to measure well.

Consider the case of large-scale writing assessments and the unfortunate effect they have had on writing instruction. There is an inherent difficulty in ensuring consistent scoring for thousands or tens of thousands or hundreds of thousands of writing samples (or extended constructed responses, if you prefer), necessitating a detailed scoring rubric, followed by teaching to the test (scoring rubric, really), with the horrible result of not excellence, but tedious and formulaic writing (see the five paragraph essay or the 3-5-3).

No doubt that attempts at large-scale assessments of 21st century skills would result in teaching students to produce similarly tedious and formulaic evidence of their capacities for “critical thinking”, “creativity”, “innovation”, “collaboration”, and public speaking.

Which means, I think, that a narrow focus on math and reading is preferable to more expansive large-scale, high stakes tests in order to allow local districts and individual students as much leeway as possible to shape school programs according to their own educational values and goals.

More than that, I think that a narrow focus on assessments of basic competency in math and reading is preferable to more expansive, high stakes tests that purport to be able to measure higher order thinking/depth of understanding or other 21st century skills. I think H. Wu offers a compelling argument for a driver’s license style, basic competency assessment for math and reading. His article is worth reading in its entirety, not just for the details of his suggested alternative, but also for his overview of the limitations of large-scale assessments.

Would the assessments measure all of the things that Jack thinks are important? Nope. But perhaps they could change the conversation we’re currently having about public education and mitigate the worst effects of large-scale, high stakes assessment.

Letters from . . . Barry Garelick

I read Letters From John Dewey/Letters From Huck Finn: A Look at Math Education From the Inside by Barry Garelick last night.

I thoroughly enjoyed them; Garelick knows which side of the math wars he’s on and writes about it with a mix of humor and seriousness.

Garelick’s introduction provides a brief overview of the math wars, if you need a primer on the subject. The letters relate a few of his experiences in ed school (training for a second career in teaching mathematics to follow his retirement from a federal government job), student teaching, and substitute teaching.

There were too many good quotes to choose from, so I’ll direct you to the Out In Left Field blog where most of the Huck Finn letters were originally published, and I’ll leave you with this observation worth remembering [from Kindle Locations 352-61]:

The teachers in both videos were extremely good at what they were doing, which brought home an unsettling realization to me: You can be very good at doing something that is absolutely horrible.