The Experts below are selected from a list of 360 Experts worldwide ranked by ideXlab platform
Judith Redi - One of the best experts on this subject based on the ideXlab platform.
-
A Reliable Methodology to Collect Ground Truth Data of Image Aesthetic Appeal
IEEE Transactions on Multimedia, 2016Co-Authors: Ernestasia Siahaan, Alan Hanjalic, Judith RediAbstract:Recognizing what makes an image Aesthetically pleasing is crucial to the effectiveness of many multimedia systems. Several works have attempted to build image Aesthetic Appeal predictors, and created their own set of ground truth data for the purpose, either by using rated images from photo sharing websites, or by asking a pool of users to rate images in lab or crowdsourcing experiments. Literature has shown that the way these experiments are conducted can influence their results: poor experimental setup can result in poorly reliable outcomes (i.e., highly imprecise Aesthetic Appeal measures). A question then arises whether the different choices made to collect ground truth of Aesthetic Appeal data are appropriate. In this paper, we propose a systematic study that looks into how different experimental environments and rating scales used to collect image Aesthetic Appeal ground truth data influence the reliability and repeatability of Aesthetic Appeal assessments. Our findings show that discrete and continuous scales with five-point absolute category rating labels yield more reliable results, with the continuous scale being more reliable for abstract images. We also show that image Aesthetic Appeal assessments could be repeatable across different experimental environments (i.e., lab and crowdsourcing). We finally formulate concrete recommendations to guide the collection of large sets of ground truth data for training models of Aesthetic Appeal appreciation.
-
when the crowd challenges the lab lessons learnt from subjective studies on image Aesthetic Appeal
ACM Multimedia, 2015Co-Authors: Judith Redi, Pavel Korshunov, Ernestasia Siahaan, Julian Habigt, Tobias HossfeldAbstract:Crowdsourcing gives researchers the opportunity to collect subjective data quickly, in the real-world, and from a very diverse pool of users. In a long-term study on image Aesthetic Appeal, we challenged the crowdsourced assessments with typical lab methodologies in order to identify and analyze the impact of crowdsourcing environment on the reliability of subjective data. We identified and conducted three types of crowdsourcing experiments that helped us perform an in-depth analysis of factors influencing reliability and reproducibility of results in uncontrolled crowdsourcing environments. We provide a generalized summary of lessons learnt for future research studies which will try to port lab-based evaluation methodologies into crowdsourcing, so that they can avoid the typical pitfalls in design and analysis of crowdsourcing experiments.
-
the Aesthetic Appeal of depth of field in photographs
Quality of Multimedia Experience, 2014Co-Authors: Tingting Zhang, Judith Redi, Harold T Nefs, Iej Ingrid HeynderickxAbstract:We report here how depth of field (DOF) affects the Aesthetic Appeal of photographs for different content categories. 339 photographs spanning eight categories were selected from Flickr, Google+, and personal collections. First, we classified the 339 photographs into three levels of depth of field: small, medium, and large. Then, we asked participants to rate the Aesthetic Appeal of these photographs in random order. We found that Aesthetic Appeal is only influenced significantly by the content category and by depth of field for animal and sport related photographs. Therefore, we conclude that depth of field should not be regarded as a common criterion for judging Aesthetic Appeal in different semantic content categories.
-
beauty is in the scale of the beholder comparison of methodologies for the subjective assessment of image Aesthetic Appeal
Quality of Multimedia Experience, 2014Co-Authors: Ernestasia Siahaan, Judith Redi, Alan HanjalicAbstract:A first step towards creating automatic measures of image Aesthetic Appeal is understanding its appreciation via subjective testing. Nevertheless, reliably setting up such tests appears to be challenging, as Aesthetic Appeal is proven to be influenced by a number of subjective factors. In this paper we investigate four scale types for Aesthetic Appeal rating and assess their ability to provide general quantification of Aesthetic Appeal, as well as repeatable judgment across experiments. We asked 24 users to assess a representative image set constructed to uniformly cover a wide range of Aesthetic Appeal. Our experiments show that the Absolute Category Rating (ACR) 5-point scale provides the most consistent ratings across participants, which are also repeatable across different experiments.
-
crowdsourcing for rating image Aesthetic Appeal better a paid or a volunteer crowd
Proceedings of the 2014 International ACM Workshop on Crowdsourcing for Multimedia, 2014Co-Authors: Judith Redi, Isabel PovoaAbstract:Crowdsourcing has the potential to become a preferred tool to study image Aesthetic Appeal preferences of users. Nevertheless, some reliability issues still exist, partially due to the sometimes doubtful commitment of paid workers to perform the rating task properly. In this paper we compare the reliability in scoring image Aesthetic Appeal of both a paid and a volunteer crowd. We recruit our volunteers through Facebook and our paid users via Microworkers. We conclude that, whereas volunteer participants are more likely to leave the rating task unfinished, when they complete it they do so more reliably than paid users.
Sine Mcdougall - One of the best experts on this subject based on the ideXlab platform.
-
mood moderates the effect of Aesthetic Appeal on performance
Cognition & Emotion, 2021Co-Authors: Irene Reppa, Sine Mcdougall, Andreas Sonderegger, William C SchmidtAbstract:Aesthetically Appealing stimuli can improve performance in demanding target localisation tasks compared to unAppealing stimuli. Two search-and-localisation experiments were carried out to examine the possible underlying mechanism mediating the effects of Appeal on performance. Participants (N = 95) were put in a positive or negative mood prior to carrying out a visual target localisation task with Appealing and unAppealing targets. In both experiments, positive mood initially led to faster localisation of Appealing compared to unAppealing stimuli, while an advantage for Appealing over unAppealing stimuli emerged over time in negative mood participants. The findings are compatible with the idea that Appealing stimuli may be inherently rewarding, with Aesthetic Appeal overcoming the detrimental effects of negative mood on performance.
-
when the going gets tough the beautiful get going Aesthetic Appeal facilitates task performance
Psychonomic Bulletin & Review, 2015Co-Authors: Irene Reppa, Sine McdougallAbstract:The current studies examined the effect of Aesthetic Appeal on performance. According to one hypothesis, Appeal would lead to overall decrements or enhancements in performance [e.g. Sonderegger & Sauer, (Applied Ergonomics, 41, 403–410, 2010)]. Alternatively, Appeal might influence performance only in problem situations, such as when the task is difficult [e.g. Norman, (2004)]. The predictions of these hypotheses were examined in the context of an icon search-and-localisation task. Icons were used because they are well-defined stimuli and pervasive to modern everyday life. When search was made difficult using visually complex stimuli (Experiment 1), or abstract and unfamiliar stimuli (Experiment 2), icons that were Appealing were found more quickly than their unAppealing counterparts. These findings show that in a low-level visual processing task, with demand characteristics related to Appeal eliminated, Appeal can influence performance, especially under duress. Electronic supplementary material The online version of this article (doi:10.3758/s13423-014-0794-z) contains supplementary material, which is available to authorized users.
-
visual Aesthetic Appeal speeds processing of complex but not simple icons
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2008Co-Authors: Irene Reppa, David Playfoot, Sine McdougallAbstract:Over the last decade there has been a shift in emphasis from interface usability to interface Appeal. Very few studies, however, have examined the link between the two. The current study examined the possibility that Aesthetic Appeal may affect user performance. In a visual search task designed to mimic user searches of interface displays, participants were asked to search for a target icon in an array of distractors. Target icons were varied orthogonally along two dimensions, complexity (which is known to affect visual search for icons in displays) and Aesthetic Appeal. The results showed that visually simple icons were found faster than visually complex icons, replicating previous findings. More importantly, Aesthetic Appeal interacted with icon complexity, significantly reducing search times for complex but not simple icons. These findings provide empirical evidence to support the idea that Aesthetic Appeal can influence performance. In recent years considerable emphasis has been given to the Aesthetic Appeal of user interfaces in the hope that Aesthetic Appeal will bolster interface usability. Studies to date suggest that our perceptions of Appeal affect the effort we are likely to make in learning how to use an interface (e.g., Kurosu & Kashimura, 1995; Lingaard & Dudek, 2003; Tractinsky, 1997; Tractinsky, Katz, & Ikar, 2000; Wiedenbeck, 1999). Although a general relationship between Appeal and performance has been identified, and research suggests that perceptions of Appeal can influence user performance, no research has examined the precise nature of the Appeal-performance relationship or the mechanisms which may underpin it. The study reported here examines one such relationship: the relationship between Aesthetic Appeal and attention. Two prior strands of research suggest that Aesthetic Appeal could have a direct influence on behaviour. The first
-
why do i like it the relationships between icon characteristics user performance and Aesthetic Appeal
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2008Co-Authors: Sine Mcdougall, Irene ReppaAbstract:Until recently the guiding tenet in human-computer interaction was that any interface must be easy to learn and use. However, it has been increasingly recognized that the Appeal of the interface to the user and their enjoyment of it is also important. The aim of the current study was to examine the nature of the relationships between icon characteristics, user performance, and Aesthetic Appeal. When participants were asked to rate the Appeal of a corpus of icons, it was found that the same icon characteristics predicted Appeal as those predicting user performance. The theoretical and practical implications of the remarkable similarity in the factors determining Appeal and usability are discussed.
Irene Reppa - One of the best experts on this subject based on the ideXlab platform.
-
mood moderates the effect of Aesthetic Appeal on performance
Cognition & Emotion, 2021Co-Authors: Irene Reppa, Sine Mcdougall, Andreas Sonderegger, William C SchmidtAbstract:Aesthetically Appealing stimuli can improve performance in demanding target localisation tasks compared to unAppealing stimuli. Two search-and-localisation experiments were carried out to examine the possible underlying mechanism mediating the effects of Appeal on performance. Participants (N = 95) were put in a positive or negative mood prior to carrying out a visual target localisation task with Appealing and unAppealing targets. In both experiments, positive mood initially led to faster localisation of Appealing compared to unAppealing stimuli, while an advantage for Appealing over unAppealing stimuli emerged over time in negative mood participants. The findings are compatible with the idea that Appealing stimuli may be inherently rewarding, with Aesthetic Appeal overcoming the detrimental effects of negative mood on performance.
-
when the going gets tough the beautiful get going Aesthetic Appeal facilitates task performance
Psychonomic Bulletin & Review, 2015Co-Authors: Irene Reppa, Sine McdougallAbstract:The current studies examined the effect of Aesthetic Appeal on performance. According to one hypothesis, Appeal would lead to overall decrements or enhancements in performance [e.g. Sonderegger & Sauer, (Applied Ergonomics, 41, 403–410, 2010)]. Alternatively, Appeal might influence performance only in problem situations, such as when the task is difficult [e.g. Norman, (2004)]. The predictions of these hypotheses were examined in the context of an icon search-and-localisation task. Icons were used because they are well-defined stimuli and pervasive to modern everyday life. When search was made difficult using visually complex stimuli (Experiment 1), or abstract and unfamiliar stimuli (Experiment 2), icons that were Appealing were found more quickly than their unAppealing counterparts. These findings show that in a low-level visual processing task, with demand characteristics related to Appeal eliminated, Appeal can influence performance, especially under duress. Electronic supplementary material The online version of this article (doi:10.3758/s13423-014-0794-z) contains supplementary material, which is available to authorized users.
-
visual Aesthetic Appeal speeds processing of complex but not simple icons
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2008Co-Authors: Irene Reppa, David Playfoot, Sine McdougallAbstract:Over the last decade there has been a shift in emphasis from interface usability to interface Appeal. Very few studies, however, have examined the link between the two. The current study examined the possibility that Aesthetic Appeal may affect user performance. In a visual search task designed to mimic user searches of interface displays, participants were asked to search for a target icon in an array of distractors. Target icons were varied orthogonally along two dimensions, complexity (which is known to affect visual search for icons in displays) and Aesthetic Appeal. The results showed that visually simple icons were found faster than visually complex icons, replicating previous findings. More importantly, Aesthetic Appeal interacted with icon complexity, significantly reducing search times for complex but not simple icons. These findings provide empirical evidence to support the idea that Aesthetic Appeal can influence performance. In recent years considerable emphasis has been given to the Aesthetic Appeal of user interfaces in the hope that Aesthetic Appeal will bolster interface usability. Studies to date suggest that our perceptions of Appeal affect the effort we are likely to make in learning how to use an interface (e.g., Kurosu & Kashimura, 1995; Lingaard & Dudek, 2003; Tractinsky, 1997; Tractinsky, Katz, & Ikar, 2000; Wiedenbeck, 1999). Although a general relationship between Appeal and performance has been identified, and research suggests that perceptions of Appeal can influence user performance, no research has examined the precise nature of the Appeal-performance relationship or the mechanisms which may underpin it. The study reported here examines one such relationship: the relationship between Aesthetic Appeal and attention. Two prior strands of research suggest that Aesthetic Appeal could have a direct influence on behaviour. The first
-
why do i like it the relationships between icon characteristics user performance and Aesthetic Appeal
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2008Co-Authors: Sine Mcdougall, Irene ReppaAbstract:Until recently the guiding tenet in human-computer interaction was that any interface must be easy to learn and use. However, it has been increasingly recognized that the Appeal of the interface to the user and their enjoyment of it is also important. The aim of the current study was to examine the nature of the relationships between icon characteristics, user performance, and Aesthetic Appeal. When participants were asked to rate the Appeal of a corpus of icons, it was found that the same icon characteristics predicted Appeal as those predicting user performance. The theoretical and practical implications of the remarkable similarity in the factors determining Appeal and usability are discussed.
Ernestasia Siahaan - One of the best experts on this subject based on the ideXlab platform.
-
subject bias in image Aesthetic Appeal ratings
Data Science: Journal of Computing and Applied Informatics, 2017Co-Authors: Ernestasia Siahaan, Esther NababanAbstract:Automatic prediction of image Aesthetic Appeal is an important part of multimedia and computer vision research, as it contributes to providing better content quality to users. Various features and learning methods have been proposed in the past to predict image Aesthetic Appeal more accurately. The effectiveness of these proposed methods often depend on the data used to train the predictor. Since Aesthetic Appeal is a subjective construct, factors that influence the subjectivity in Aesthetic Appeal data need to be understood and addressed. In this paper, we look into the subjectivity of Aesthetic Appeal data, and how it relates with image characteristics that are often used in Aesthetic Appeal prediction. We use subject bias and confidence interval to measure subjectivity, and check how they might be influenced by image content category and features.
-
A Reliable Methodology to Collect Ground Truth Data of Image Aesthetic Appeal
IEEE Transactions on Multimedia, 2016Co-Authors: Ernestasia Siahaan, Alan Hanjalic, Judith RediAbstract:Recognizing what makes an image Aesthetically pleasing is crucial to the effectiveness of many multimedia systems. Several works have attempted to build image Aesthetic Appeal predictors, and created their own set of ground truth data for the purpose, either by using rated images from photo sharing websites, or by asking a pool of users to rate images in lab or crowdsourcing experiments. Literature has shown that the way these experiments are conducted can influence their results: poor experimental setup can result in poorly reliable outcomes (i.e., highly imprecise Aesthetic Appeal measures). A question then arises whether the different choices made to collect ground truth of Aesthetic Appeal data are appropriate. In this paper, we propose a systematic study that looks into how different experimental environments and rating scales used to collect image Aesthetic Appeal ground truth data influence the reliability and repeatability of Aesthetic Appeal assessments. Our findings show that discrete and continuous scales with five-point absolute category rating labels yield more reliable results, with the continuous scale being more reliable for abstract images. We also show that image Aesthetic Appeal assessments could be repeatable across different experimental environments (i.e., lab and crowdsourcing). We finally formulate concrete recommendations to guide the collection of large sets of ground truth data for training models of Aesthetic Appeal appreciation.
-
when the crowd challenges the lab lessons learnt from subjective studies on image Aesthetic Appeal
ACM Multimedia, 2015Co-Authors: Judith Redi, Pavel Korshunov, Ernestasia Siahaan, Julian Habigt, Tobias HossfeldAbstract:Crowdsourcing gives researchers the opportunity to collect subjective data quickly, in the real-world, and from a very diverse pool of users. In a long-term study on image Aesthetic Appeal, we challenged the crowdsourced assessments with typical lab methodologies in order to identify and analyze the impact of crowdsourcing environment on the reliability of subjective data. We identified and conducted three types of crowdsourcing experiments that helped us perform an in-depth analysis of factors influencing reliability and reproducibility of results in uncontrolled crowdsourcing environments. We provide a generalized summary of lessons learnt for future research studies which will try to port lab-based evaluation methodologies into crowdsourcing, so that they can avoid the typical pitfalls in design and analysis of crowdsourcing experiments.
-
beauty is in the scale of the beholder comparison of methodologies for the subjective assessment of image Aesthetic Appeal
Quality of Multimedia Experience, 2014Co-Authors: Ernestasia Siahaan, Judith Redi, Alan HanjalicAbstract:A first step towards creating automatic measures of image Aesthetic Appeal is understanding its appreciation via subjective testing. Nevertheless, reliably setting up such tests appears to be challenging, as Aesthetic Appeal is proven to be influenced by a number of subjective factors. In this paper we investigate four scale types for Aesthetic Appeal rating and assess their ability to provide general quantification of Aesthetic Appeal, as well as repeatable judgment across experiments. We asked 24 users to assess a representative image set constructed to uniformly cover a wide range of Aesthetic Appeal. Our experiments show that the Absolute Category Rating (ACR) 5-point scale provides the most consistent ratings across participants, which are also repeatable across different experiments.
Isabel Povoa - One of the best experts on this subject based on the ideXlab platform.
-
crowdsourcing for rating image Aesthetic Appeal better a paid or a volunteer crowd
Proceedings of the 2014 International ACM Workshop on Crowdsourcing for Multimedia, 2014Co-Authors: Judith Redi, Isabel PovoaAbstract:Crowdsourcing has the potential to become a preferred tool to study image Aesthetic Appeal preferences of users. Nevertheless, some reliability issues still exist, partially due to the sometimes doubtful commitment of paid workers to perform the rating task properly. In this paper we compare the reliability in scoring image Aesthetic Appeal of both a paid and a volunteer crowd. We recruit our volunteers through Facebook and our paid users via Microworkers. We conclude that, whereas volunteer participants are more likely to leave the rating task unfinished, when they complete it they do so more reliably than paid users.
-
Evaluating the impact of digital filters on the Aesthetic Appeal of photographs: A crowdsourcing based approach
2014Co-Authors: Isabel PovoaAbstract:In particular grouping by Aesthetics and quality of the media has brought along new challenges for Computational Aesthetics research such as what makes an image beautiful, what means beautiful and how do you quantify beautiful. to meet those challenges, researchers have tried to come up with several algorithms based in different metrics to bridge the gap between the quantitative aspects of what is called beauty and what people call beauty. In order to fill part of this gap we studied the effect of digital filters in photographic Aesthetics so widely used in the social networks nowadays. Taking in consideration the popularity of digital filters among many social network users, it was a surprise to understand that most participants in the experiment preferred the images with no filter. In any case measuring what is beautiful always requires collecting Aesthetics scores from people. Doing that collection process in a laboratory environment is the most effective approach. The main reasons are the highly controlled environment that leads to good data quality. The downside is cost, time and restriction of participants to the people available nearby. Therefore another issue addressed in the study was the use of crowdsourcing to minimize time and cost, as well as to expand the scope of participation, in the process of collecting image scores from users. To test that possibility a 4 step process step was designed and implemented. First preference scores were collected in a lab environment over a previous selected dataset. Afterwards the crowdsourcing experiment was planned what included an optimization of the dataset (ground truth dataset). Subsequently three digital filters were then applied to the collection and an online experiment followed to once again collect preference scores. In phase, we developed the experiment in the context of Microworkers and as a Facebook app interface enriched with a playful visual interface. The last step included a process to filter the suspicious participants and check results consistency. The results show that implementing an experiment to collect preferences of image quality in social media is a good methodology for Computational Aesthetics, if appropriate planning and management is adopted.
-
the role of visual attention in the Aesthetic Appeal of consumer images a preliminary study
Visual Communications and Image Processing, 2013Co-Authors: Judith Redi, Isabel PovoaAbstract:Predicting the Aesthetic Appeal of images is of great interest for a number of applications, from image retrieval to visual quality optimization. In this paper, we report a preliminary study on the relationship between visual attention deployment and Aesthetic Appeal judgment. In particular, we seek to validate through a scientific approach those simplicity and compositional rules of thumb that have been applied by photographers and modeled by computer vision scientists in computational Aesthetics algorithms. Our results provide a confirmation that both simplicity and composition matter for Aesthetic Appeal of images, and indicate effective ways to compute them directly from the saliency distribution of an image.
-
crowdsourcing based multimedia subjective evaluations a case study on image recognizability and Aesthetic Appeal
ACM Multimedia, 2013Co-Authors: Judith Redi, Tobias Hosfeld, Pavel Korshunov, Filippo Mazza, Isabel Povoa, Christian KeimelAbstract:Research on Quality of Experience (QoE) heavily relies on subjective evaluations of media. An important aspect of QoE concerns modeling and quantifying the subjective notions of 'beauty' (Aesthetic Appeal) and 'something well-known' (content recognizability), which are both subject to cultural and social effects. Crowdsourcing, which allows employing people worldwide to perform short and simple tasks via online platforms, can be a great tool for performing subjective studies in a time and cost-effective way. On the other hand, the crowdsourcing environment does not allow for the degree of experimental control which is necessary to guarantee reliable subjective data. To validate the use of crowdsourcing for QoE assessments, in this paper, we evaluate Aesthetic Appeal and recognizability of images using the Microworkers crowdsourcing platform and compare the outcomes with more conventional evaluations conducted in a controlled lab environment. We find high correlation between crowdsourcing and lab scores for recognizability but not for Aesthetic Appeal, indicating that crowdsourcing can be used for QoE subjective assessments as long as the workers' tasks are designed with extreme care to avoid misinterpretations.