Adriana Padilla
Master of Science Capstone Project, December 2020
Content recommendation has garnered increasing attention in recent years. While there are many approaches to modeling this problem, one well-known way is as a balancing of “exploration” and “exploitation”, where “exploitation” selects content with known positive feedback, and “exploration” chooses new or less familiar content in order to better understand it. The exploration-exploitation trade-off can be effectively modelled with a bandit-based approach, also known as the Multi-Armed Bandit problem (MAB), which represents available options as bandits with corresponding rewards. Solutions for MAB problems aim to maximize the accumulated rewards through the balancing of exploitation and exploration of bandits.
New language learning has been greatly facilitated by many mobile applications. Applications for vocabulary learning typically offer users a set of preselected words that can be learned through various engaging activities. An approach to further engage the user in word learning would be to leverage their physical location to recommend relevant words. Such a system can be modeled as a contextual MAB problem.
This report details the modeling of an application that recommends vocabulary based on location as a contextual MAB problem, and describes an approach to solving the problem based on the Thompson Sampling algorithm. In order to consider the location dimension, the Thompson Sampling algorithm is invoked to process words that are categorized according to the location of the device. Additionally, with the goal of providing users with a way to learn the recommended words, the application offers a set of vocabulary learning activities. The process of selecting words for the activities is also formulated as a recommendation system, where the user’s familiarity with a word is the target of exploitation-exploration trade-off.
Based on this system, over the course of three weeks, data was collected from a group of 12 testers who were asked to rate the relevance of the recommended words. Ratings were also compared between one user who started using the app in the first week of testing, and another who started after the three-week testing had ended. Overall, the results show that the system was able to weed out words with low relevance ratings, while repeatedly recommending those with high rating scores. As a result, it is observed that the average relevancy ratings had increased over time.
These positive results demonstrate that the MAB algorithms can be effectively applied to solve the problem of location-based word recommendation, and that other context-based recommendation systems could also benefit from these algorithms.