Personalization, especially in recommender systems, is ubiquitous, from personalized streams in social media to recommendations of what to buy in online shops. The algorithms that drive personalization today are very good in exploiting bottom-up information, such as user clicks or ratings. However, bottom-up approaches are not able to capture top-down knowledge, which is embedded in the human cognition. For example, the bottom-up information is a manifestation of human behavior, which is driven by decisions. However, decisions are complex cognitive processes that a single click cannot capture entirely. Hence, we need to complement bottom-up approaches with top-down cognitive models. My research interest is to use theories and models from psychology to (i) provide further insight into the behavior of users, (ii) devise methodologies for the acquisition of novel features from available data and (iii) improve the performance of personalized systems.
My doctoral and post-doctoral work focused on combining the psychological theories on personality and emotions and recommender systems. I showed that both the usage of personality and emotions has benefits on personalized recommender systems. In psychology research, personality has shown to account for many individual differences in the way we behave (John & Srivastava, 1999) and in what kind of content we prefer (Rentfrow & Gosling, 2003). Furthermore, decision making research devised models where it has been shown that emotions drive our decisions in a substantial way (Kahneman, 2003). However, these models from psychology have not yet been exploited to model the preferences of users for personalization of online content.
My approach in filling this gap was to map the personality and emotion theories in recommender systems scenarios. I used user studies to collect data, image processing and natural language processing to generate features, and machine learning techniques to predict the preferences and perform recommendations. Throughout a series of experiments, I built the elements of a chain that takes user data unobtrusively, infers emotions and personality and then uses these in user models for personalizing recommendations.
First, I showed that personality improves the accuracy of recommender systems in cold-start conditions. By devising a personality based user similarity measure I showed that such measure yields better recommendation than the standard rating-based user similarity measures (Tkalčič, Kunaver, Košir, & Tasič, 2011).
Second, I complemented the personality-based recommender algorithm with a method for the unobtrusive inference of personality from social media. To be more precise, I used Instagram and Twitter and applied image processing techniques to extract feature from posted images and natural language processing to extract features from posts. I trained a machine learning regressor for the prediction of personality traits (Skowron, Ferwerda, Tkalčič, & Schedl, 2016).
Third, I showed that people differ in that which emotions they pursue when consuming content. I used this information to profile users and built a recommender system based on the induced emotions of the content (Tkalčič, Burnik, & Košir, 2010).
Fourth, I complemented the emotion-based recommender system with an algorithm that detects the emotional state of the user from the web camera feed. I extracted Gabor features from the videos of facial expressions of users and trained a kNN classifier to predict dichotomized emotions in the valence-arousal-dominance space (Tkalčič, Odić, & Košir, 2013).
I am currently investigating the relationship between music preferences and emotional responses to music. More specifically, how emotional responses to music-listening predicts pairwise music preferences.
I also work on a project where I am devising a personality-based user similarity measure in the domain of online purchases.
My future work will primarily focus on extending existing data-driven user models with more psychological models. I will study how the behavior of users, which is manifested through log data of online applications, can be modeled with theory-driven models.
If you are further interested in my work, please download my CV.