Identification of Behavioral Variables for Efficient Representation of Difficulty in Vocabulary Learning Systems
Abstract
This study focuses on foreign language vocabulary learning in computerized medium and seeks behavioral variables that best reflect the difficulty level of learning material. In this respect, we employ a spaced repetition flashcard software and display English vocabulary belonging to various word classes as well as difficulty levels to a set of participants and determine the most efficient variables in representation of task difficulty and estimation of required effort. Based on an analysis of activity logs, we propose a set of behavioral variables, which have a potential relation to task difficulty. Subsequently, we examine the correlation between these behavioral variables and task difficulty levels. Our results indicate that variables at deck level have a stronger relation to difficulty than those at card level. In addition, when the correlation of the proposed variables with the difficulty level is contrasted to users’ self assessment of difficulty, the proposed variables are found to be more reliable indicators of difficulty. Such variables are promising for improving the performance in user-adaptation, scheduling of learning tasks, and estimation of learning effort, motivation and engagement among others.
References
A. Beinicke and T. Bipp, “Evaluating training outcomes in corporate e-learning and classroom training,” Vocations and Learning, p. 501–528, 2018.
E. O’Donnell, S. Lawless, M. Sharp, and V. P. Wade, “A review of personalised elearning: Towards supporting learner diversity,” International Journal of Distance Education Technologies, vol. 13, no. 1, pp. 22–47, 2015.
G. M. Piskurich, “Online learning: E-learning. Fast, cheap, and good,” Performance Improvement, vol. 45, no. 1, pp. 18–24, 2006.
M. A. A. Dewan, M. Murshed, and F. Lin, “Engagement detection in online learning: a review,” Smart Learning Environments, vol. 6, no. 1, p. 1, 2019.
J. Keller and K. Suzuki, “Learner motivation and e-learning design: A multinationally validated process,” Journal of Educational Multimedia and Hypermedia , vol. 29, no. 3, pp. 229–239, 2004.
J. A. Fredricks, P. C. Blumenfeld, and A. H. Paris, “School engagement: Potential of the concept, state of the evidence,” Review of Educational Research, vol. 74, no. 1, pp. 59–109, 2004.
M. Cocea and S. Weibelzahl, “Can log files analysis estimate learners’ level of motivation?,” in LWA, pp. 1–4, 2006.
C. R. Henrie, L. R. Halverson, and C. R. Graham, “Measuring student engagement in technology-mediated learning,” Computers & Education, vol. 90, pp. 36–53, 2015.
G. Ben-Zadok, M. Leiba, and R. Nachmias, “Drills, Games or Tests? Evaluating Students’ Motivation in Different Online Learning Activities, Using Log File Analysis,” Interdisciplinary Journal of E-Learning and Learning Objects, vol. 7, no. 1, pp. 235–248, 2011.
L. Juhanˇak, J. Zounek, and L. Rohl ´ ´ıkova, “Using process mining to analyze students’ ´ quiz-taking behavior patterns in a learning management system,” Computers in Human Behavior, vol. 92, pp. 496–506, 2019.
A. Hershkovitz and R. Nachmias, “Learning about online learning processes and students’ motivation through web usage mining,” Interdisciplinary Journal of E-Learning and Learning Objects, vol. 5, no. 1, pp. 197–214, 2009.
S. Chen, A. Lippert, G. Shi, Y. Fang, and A. C. Graesser, “Disengagement detection within an intelligent tutoring system,” in Proc. International Conference on Intelligent Tutoring Systems, p. 127, 2018.
Y. Ma, L. Agnihotri, R. Baker, and S. Mojarad, “Effect of student ability and question difficulty on duration,” Journal of Educational Data Mining, pp. 135–142, 2016.
J. Beck, Engagement tracing: using response times to model student disengagement, vol. 125. Ios Press, 2005.
M. Cocea and S. Weibelzahl, “Cross-system validation of engagement prediction from log files,” in Proc. European Conference on Technology Enhanced Learning, pp. 14–25, Springer, 2007.
M. Cocea and S. Weibelzahl, “Log file analysis for disengagement detection in elearning environments,” Journal of User Modeling and User-Adapted Interaction, vol. 19, no. 4, pp. 341–385, 2009.
S. Reddy, I. Labutov, S. Banerjee, and T. Joachims, “Unbounded human learning: Optimal scheduling for spaced repetition,” in Proc. ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1815–1824, ACM, 2016.
N. Kornell, “Optimising learning using flashcards: Spacing is more effective than cramming,” Applied Cognitive Psychology, vol. 23, no. 9, pp. 1297–1317, 2009.
D. Keder, Computer-assisted language learning using spaced repetition. PhD thesis, Masarykova univerzita, 2009.
G. Vigliocco, D. P. Vinson, J. Druks, H. Barber, and S. F. Cappa, “Nouns and verbs in the brain: a review of behavioural, electrophysiological, neuropsychological and imaging studies,” Neuroscience & Biobehavioral Review, vol. 35, no. 3, pp. 407–426, 2011.
Z. Yucel, P. Supitayakul, A. Monden, and P. Leelaprute, “An algorithm for automatic ¨ collation of vocabulary decks based on word frequency,” IEICE Transactions on Information and Systems, vol. E103-D, no. 8, pp. 1–10, 2020.
D. S. Kerby, “The simple difference formula: An approach to teaching nonparametric correlation,” Comprehensive Psychology, vol. 3, pp. 11–IT, 2014.
D. Elmes, “Anki - friendly, intelligent flashcards.” https://ankiweb.net/about, 2019. [Accessed 2019-03-15].
C. Altiner, “Integrating a computer-based flashcard program into academic vocabulary learning,” Master’s thesis, Iowa State University, 2011.
U. Olsson, F. Drasgow, and N. J. Dorans, “The polyserial correlation coefficient,” Psychometrika, vol. 47, no. 3, pp. 337–347, 1982.
V. Arkorful and N. Abaidoo, “The role of e-learning, advantages and disadvantages of its adoption in higher education,” International Journal of Distance Education Technologies, vol. 12, no. 1, pp. 29–42, 2015.
