Skip to main contentSkip to main content
You are the owner of this article.
You have permission to edit this article.
alert top story

NAU researcher developing tool to help language learners with pronunciation

  • 0

A Northern Arizona University researcher is in the process of developing a feedback tool to help second-language learners with their pronunciation.

Okim Kang, a professor of applied linguistics at the university, received a $300,000 National Science Foundation (NSF) grant last year to develop language acquisition technology.

The grant is from the Early-Concept Grants for Exploratory Research program (EAGER), which according to NSF’s website, “supports exploratory work in its early stages on untested, but potentially transformative, research ideas or approaches.”

John Hansen from the University of Texas - Dallas and Stephen Looney from Penn State University are co-principal investigators on the project alongside Kang.

Called Objective Speech Intelligibility Measures (OSIM), the project focuses on creating a program that can analyze language learners' speech and provide useful feedback to improve pronunciation.

Kang's hope is to create a free program eventually that could give learners feedback on speaking in a language they are trying to learn, as well as practical methods to help them improve.

The team will be submitting a follow-up proposal for an additional $850,000 of funding in October, with the hope for the project to take four years.

Kang said in the next three years she is trying to make a “freely available, individualized pronunciation feedback program” for language learners.

The team is currently developing one for English and might expand to other languages if it is successful.

“My goal is to develop this specific computer program that can evaluate or assess students’ overall speaking skills, but especially on pronunciation and provide specific pronunciation features along with the intelligibility score. This type of information can be presented to both teachers and students so that they know what they are doing,” Kang said.

There are more objective assessments for the other primary language skills: reading, writing and listening. Tests for speaking skills, however, often involve teachers making an assessment based on their speaking with students.

In the year since receiving the first grant, Kang has been working primarily with international teaching assistants, a group that typically has a high level of proficiency with English, but may have trouble with intelligibility.

The researchers chose a variety of speech features -- stress, pausing, speed -- and worked with the assistants to give feedback and see the ways they improved or changed their pronunciation in response.

“It's not like one thing can determine your accent or pronunciation. Many of the speech components can play a role,” Kang said.

When it comes to being understood, some features are also more important than others.

“When you and I are communicating, I want to emphasize certain words because that word is important," she said. "So I pronounce it strongly and more slowly. ... This type of information and this kind of feature is way more important than teaching one consonant.”

The tool compares a learner’s speech to that of second-language learners with the same native language, something Kang said was unique in automatic speech recognition (ASR) feedback programs. Rather than making a comparison to native speakers’ speech, it focuses on improving how well learners can be understood.

“The learner’s target goal is not native speech, but a very, very intelligible speech level, so it’s much more achievable and it’s much more specific for them,” she said.

Kang developed a patented pronunciation model years ago, focusing on 35 features that determine a speaker’s accent. The model for this project is more simplified, focusing on the features that have the biggest impacts on speech and based on the computer’s capabilities.

Two sets of data are put into the model: linguistics information on speech properties and human perceptual information. The first comes from biolinguistic analysis of speech, and the second from consultation with linguistic experts and teachers listening to the same speech files.

“Eventually with the sets of data … like, lots and lots of speech data, we ultimately train the technology to produce the outcome that we want," she said.

The model can then also be used as the base for a program focusing on learners of a different language, or for those who have a different native language, using additional data sets to train the computer. 

The finished product could be used for both individual study and as a supplementary material in language classrooms.

Language learning classrooms often use textbooks and informal conversation between students to teach speaking skills. The approach, and gaps in teacher training, can cause several issues.

On July 6, Meta announced a new AI capable of translating hundreds of languages.

“Having taught classes myself, I know teachers are looking for a tool like this because, especially teaching a pronunciation or speaking class, evaluating learners’ progress is extremely hard ... . It’s almost this instant, intuitive decision made by teachers," Kang said.

She said she hopes OSIM will be the first step in realizing her "big idea": a new kind of teaching tool for language learners. She has been working with pronunciation for years, and said she'd tried for this particular NSF grant several times.

Eventually, she hopes the project will lead to a personal AI tutor program that can work one-on-one with language learners on language skills, including pronunciation.

“Traditional classrooms are important, but I realize nowadays, more and more people rely on technology,” she said. “They’re more individualized and ... undocumented immigrants, they can’t even go to classes and their resources are extremely limited. So I want a freely available language tutor program.”

More about Kang’s research is available here.


* I understand and agree that registration on or use of this site constitutes agreement to its user agreement and privacy policy.

Related to this story

Most Popular

Get up-to-the-minute news sent straight to your device.


News Alerts

Breaking News

Breaking News (FlagLive!)