| Abstract [eng] |
Visual emotion recognition (VER) is a task in affective computing that aims to automatically recognize emotion in images. In general-purpose images, emotion can be influenced not only on depicted objects but also on color, texture, composition, and scene context. The task is difficult because the same visual elements may be associated with different emotions depending on how they appear in the whole image and how they are perceived by different viewers. In this dissertation, VER is studied in general-purpose images using an eight-category labeling scheme aligned with the employed datasets. This dissertation proposes a VER model based on a convolutional neural network (CNN). To complement the main visual representation learned by the CNN, Gram matrix modules are added to capture stylistic information from intermediate feature maps, such as texture and color-related patterns. In addition, contrastive-center loss is incorporated into training to improve the separation of emotion classes in the learned feature space and to enhance classification performance on general-purpose visual emotion datasets. Model performance is evaluated using standard classification metrics, together with feature-space analysis based on dimensionality reduction and clustering quality evaluations. The dissertation also proposes a top-2 cross-sentiment measure for assessing prediction consistency without requiring ground-truth labels. The proposed model, training strategy, and consistency measure contribute to more reliable VER in general-purpose images. |
