Tutors question ‘nonsense word’ literacy assessment

During a training for Reading Corps coaches this fall, Sue Vang noticed something strange about one of the literacy assessments for kindergarteners and first graders. The so-called “Nonsense Words Benchmark Probe,” which is meant to test students’ reading skills, was full of common Hmong words.

Reading Corps is an AmeriCorps program that works in schools to provide one-on-one tutoring for students who need extra help learning to read. The nonsense words probe is one of four assessments the program uses to gather information about students’ reading abilities, according to Lisa Winkler, vice president of external relations for ServeMinnesota, which oversees Reading Corps. It instructs students to sound out letter combinations that do not make English words, such as “zof” and “jex.” This allows Reading Corps tutors to see whether students can decode words as opposed to just recognizing them in context.

Vang, a Reading Corps internal coach and bilingual literacy specialist at Hmong International Academy in North Minneapolis, is concerned about how the assessment could impact the bilingual students in her school, which is 72 percent Hmong. She found that 19 out of 50 of the “nonsense” letter combinations in the first grade probe and 10 out of 50 in the kindergarten probe are Hmong words.

“[Hmong International Academy is] promoting language development and literacy and bilingualism,” she explains. “But when we show this to our [Hmong] kids, it’s telling them that your language is not real, it’s nonsense, it’s made up…We’re setting up the students already at an early age to see a subdivision between different groups of people.”

Vang also worries that calling Hmong words nonsense could send mixed messages to young bilingual students, who do not yet have a solid foundation in either English or Hmong literacy. “We don’t want the students … switching back and forth between those languages. We want students to be secure and concrete in any language,” she said. 

Dr. Cheryl Bostrom, a Reading Corps internal coach at Hmong College Prep Academy in St. Paul, also attended the September training and shares Vang’s concerns. “What does that communicate to [our Hmong students and Reading Corps tutors] when we call their language nonsense?”

Vang and Bostrom brought the issue to the attention of the Reading Corps trainers at the September training. They suggested Reading Corps could remove the word “nonsense” as a descriptor for the assignment, and replace the Hmong words with letter combinations that make neither English nor Hmong words. 

Following the training, Vang and Bostrom received an email from Dr. Erin Haley Strub, director of Strategic Implementation with ServeMinnesota. In the email, Strub said that Reading Corps would communicate their feedback to FastBridge, the company that developed the assessment. However, Strub explained that Reading Corps could not take independent steps to change the assessment because “Reading Corps has an agreement with FastBridge…[that] does not allow us to modify copyrighted materials that are not owned by us, without their permission or consent.” In an interview, Lisa Winkler with ServeMinnesota added, “Because [Reading Corps] is so research based and tested, we definitely want FastBridge to weigh in on this and make sure the assessments we’re using are tested and used with fidelity.”

In the meantime, Strub offered two options to address the coaches’ concerns. The first option was to “use white out for the word ‘nonsense’ in the header of the probes…[and] take out the word ‘pretend’ in the standard directions.” Option two was to “use white out for the words that are Hmong words…[and] use an adjusted formula to get a score.”

For Vang, this doesn’t solve the problem. “That’s a fast Band-Aid for our school but it’s not a solution that’s going to be impacting the rest of Minnesota.” After consulting with the first grade teachers and the administrative team at Hmong International Academy, Vang decided not to use the assessment until she hears whether Reading Corps will modify it. Bostrom is doing the same.

As of the first week in November, ServeMinnesota reached out to all of Minnesota Reading Corps’ internal coaches informing them of the situation.

Minnesota’s Hmong population is the second largest in the country, with just over 66,000 residents as of the 2010 census. The Nonsense Words Benchmark Probe is currently being used in over 550 schools statewide that use Reading Corps’ K-3 program, according to Strub with ServeMinnesota.

Not all students at these schools are given the assessment, however. According to Bostrom, Reading Corps uses probes like this one to see whether students with mid-range standardized test scores may need additional support through the program.

Winkler also points out that nonsense words assignments such as this one are not unique to Reading Corps. “There are other education companies that have a similar assessment. That term ‘nonsense words’ was not developed from the FastBridge team.”

Dao Lor, director of operations and community relations for Hmong College Prep Academy, shares Vang and Bostrom’s concerns but acknowledges that there is no perfect solution. “The United States is so diverse and we have so many different cultures…Even if we change it for Hmong, we might not be able to change it for other groups.”

Still, ServeMinnesota is working to improve the assessment Reading Corps uses. “The research director [with ServeMinnesota] has been in fairly regular contact with [FastBridge] since we learned about it,” says Winkler. “I don’t know what the timeline is but I know that it’s definitely on their high priority list.”

For her part, Bostrom remains optimistic. “I just wanted to go on record acknowledging that they are trying to be culturally responsive. And it takes time. So I want to give them the time to fix it and make it right.”