Language Learning Analytics

“Dictionary” is a language learning analytic tool for students to view lists of vocabulary they and the community they belong have used over time. This is embedded in an online discussion platform. When students submit their posts, the system compares the words they have used with those in actual dictionaries loaded in the system, and if the words are matched, they will be registered as “their words” and if they don’t these words appear as “erroneous words.” The system registers both correctly and incorrectly spelled words in students’ dictionaries, in which they can view their growing vocabulary in use.

This is the initial concept of “Dictionary” and I have been working on refining the design through design research.

Challenges

(Teacher perspective) It’s great students improve their writing through idea building; taking students’ focus away from subject-matter may not be a good idea. It might risk interfering with building ideas on primary topics teachers want them to focus.

(Student perspective) They see a growing list of vocabulary, which might help them get motivated to write more but then what? The initial model shows only correct and incorrect words, which might or might not prompt them to fix them. It is based on deficit model.

Following an initial data analysis study to investigate gaps in expert and novice students’ linguistic and conceptual knowledge built up over time, I conducted a pilot study with a teacher and two students (grade 3 and 6) in Japan for four weeks in 2021. With insights drawn from the data analysis study as rationales, I explored how novice writers could improve their English writing and conceptual understanding simultaneously by using technologies such as Dictionary and translators.

Dictionary Design Used

This is the initial Dictionary design used for this pilot study. It shows a list of all words with sentences students have used in their posts as well as word counts. The green and orange round-cornered boxes show the total and unique numbers of words each has used.

When students click "Display Option" the filter option appears. Students could filter the results by the type (general or science vocabulary), users, and word levels (the first 1000, 2000, 3000 most frequent words, and anything beyond). For word levels, most frequent English words were used as an indicator because literature shows less frequently used words are more advanced or sophisticated. There was also an option to remove any "stop words" such as "and" and "or" because these words can be meaningless to track for some students.

Data Collection Methods

• Observation
• Text-Mining
• Qualitative Interview
• Reflective Journal
• Pre- and Post-Tests

Data Analysis Methods

• Discourse Analysis

Major Findings

Translators worked like magic to engage students in writing their ideas in English, leveraging their knowledge for their 1st language (Japanese). They used translators as a tool to examine structures of languages through experimenting with input-output patterns. They explored, experimented with, and simulated language and content through interactions with translators. (for those who are interested in reading about this study more, you can do so here)

Example Scenario:

A: “This sentence generates this sentence in English. What about if I change this part of the sentence to xx?”

B: “OK. So, changing this part changes this part in English. Then, xx in the original sentence corresponds to xx in this sentence.”

Problems: current design of Dictionary didn’t engage students

• They had very little interest in checking their vocabulary on Dictionary
• They used translators to write and because of that, they felt that some words they wrote from translators in the earlier sessions did not belong to them
• No meaningful connection between form (language) and content (thought / idea) visible to users; again, nothing beyond a list of correct and incorrect words – no significant improvement from the original design

What if we ... ?

• Let interested users examine structures of their outputs (so that they might better understand texts and the knowledge returns to their writing)
• Let interested users experiment with / simulate language outputs (so that they might strive for more sophisticated ways of writing their ideas)
• Let interested users see patterns in form and content (so that writers might go back and forth between two layers of their writing and language and content compliment each other)

Next question… How might we do these?

New Dictionary design intends to let users to do simple and quick explorations of their words with sentences that build up over time as they write (going beyond “correct” and “incorrect” categories). The purpose of using this is to support students to search for better ways of writing and understanding ideas. This could be done by letting them see connections between form and content. For example, functions such as filtering sentences by semantically similar texts might help them see how the same or similar idea could be said in different ways. Also on the preposition there will be a new implementation on the platform side that allows students to write multilingual entries, it might be helpful students could view a list of words and sentences used by individual students and the entire community in different languages (if they wrote in other languages).