The Experts below are selected from a list of 9738 Experts worldwide ranked by ideXlab platform
Laura Mickes - One of the best experts on this subject based on the ideXlab platform.
-
theoretical vs empirical Discriminability the application of roc methods to eyewitness identification
Cognitive Research: Principles and Implications, 2018Co-Authors: John T Wixted, Laura MickesAbstract:Receiver operating characteristic (ROC) analysis was introduced to the field of eyewitness identification 5 years ago. Since that time, it has been both influential and controversial, and the debate has raised an issue about measuring Discriminability that is rarely considered. The issue concerns the distinction between empirical Discriminability (measured by area under the ROC curve) vs. underlying/theoretical Discriminability (measured by d’ or variants of it). Under most circumstances, the two measures will agree about a difference between two conditions in terms of Discriminability. However, it is possible for them to disagree, and that fact can lead to confusion about which condition actually yields higher Discriminability. For example, if the two conditions have implications for real-world practice (e.g., a comparison of competing lineup formats), should a policymaker rely on the area-under-the-curve measure or the theory-based measure? Here, we illustrate the fact that a given empirical ROC yields as many underlying Discriminability measures as there are theories that one is willing to take seriously. No matter which theory is correct, for practical purposes, the singular area-under-the-curve measure best identifies the diagnostically superior procedure. For that reason, area under the ROC curve informs policy in a way that underlying theoretical Discriminability never can. At the same time, theoretical measures of Discriminability are equally important, but for a different reason. Without an adequate theoretical understanding of the relevant task, the field will be in no position to enhance empirical Discriminability.
-
roc analysis measures objective Discriminability for any eyewitness identification procedure
Journal of applied research in memory and cognition, 2015Co-Authors: John T Wixted, Laura MickesAbstract:Abstract Which eyewitness identification procedure better enables eyewitnesses to discriminate between innocent and guilty suspects? In other words, which procedure better enables eyewitnesses to sort innocent and guilty suspects into their correct categories? The answer to that objective, theory-free question is what policymakers need to know, and it is precisely the information that ROC analysis provides. Wells et al. largely ignore that question and focus instead on whether ROC analysis accurately measures underlying (theoretical) Discriminability for lineups. They argue that the apparent Discriminability advantage for lineups over showups is an illusion caused by “filler siphoning.” Here, we demonstrate that, both objectively and theoretically, the ability of eyewitnesses to discriminate innocent from guilty suspects is higher for lineups compared to showups, just as the ROC data suggest. Intuitions notwithstanding, filler siphoning does not account for the Discriminability advantage for lineups. An actual theory of Discriminability is needed to explain that interesting phenomenon.
-
a signal detection based diagnostic feature detection model of eyewitness identification
Psychological Review, 2014Co-Authors: John T Wixted, Laura MickesAbstract:The theoretical understanding of eyewitness identifications made from a police lineup has long been guided by the distinction between absolute and relative decision strategies. In addition, the accuracy of identifications associated with different eyewitness memory procedures has long been evaluated using measures like the diagnosticity ratio (the correct identification rate divided by the false identification rate). Framed in terms of signal-detection theory, both the absolute/relative distinction and the diagnosticity ratio are mainly relevant to response bias while remaining silent about the key issue of diagnostic accuracy, or Discriminability (i.e., the ability to tell the difference between innocent and guilty suspects in a lineup). Here, we propose a signal-detection-based model of eyewitness identification, one that encourages the use of (and helps to conceptualize) receiver operating characteristic (ROC) analysis to measure Discriminability. Recent ROC analyses indicate that the simultaneous presentation of faces in a lineup yields higher Discriminability than the presentation of faces in isolation, and we propose a diagnostic feature-detection hypothesis to account for that result. According to this hypothesis, the simultaneous presentation of faces allows the eyewitness to appreciate that certain facial features (viz., those that are shared by everyone in the lineup) are non-diagnostic of guilt. To the extent that those non-diagnostic features are discounted in favor of potentially more diagnostic features, the ability to discriminate innocent from guilty suspects will be enhanced.
John T Wixted - One of the best experts on this subject based on the ideXlab platform.
-
theoretical vs empirical Discriminability the application of roc methods to eyewitness identification
Cognitive Research: Principles and Implications, 2018Co-Authors: John T Wixted, Laura MickesAbstract:Receiver operating characteristic (ROC) analysis was introduced to the field of eyewitness identification 5 years ago. Since that time, it has been both influential and controversial, and the debate has raised an issue about measuring Discriminability that is rarely considered. The issue concerns the distinction between empirical Discriminability (measured by area under the ROC curve) vs. underlying/theoretical Discriminability (measured by d’ or variants of it). Under most circumstances, the two measures will agree about a difference between two conditions in terms of Discriminability. However, it is possible for them to disagree, and that fact can lead to confusion about which condition actually yields higher Discriminability. For example, if the two conditions have implications for real-world practice (e.g., a comparison of competing lineup formats), should a policymaker rely on the area-under-the-curve measure or the theory-based measure? Here, we illustrate the fact that a given empirical ROC yields as many underlying Discriminability measures as there are theories that one is willing to take seriously. No matter which theory is correct, for practical purposes, the singular area-under-the-curve measure best identifies the diagnostically superior procedure. For that reason, area under the ROC curve informs policy in a way that underlying theoretical Discriminability never can. At the same time, theoretical measures of Discriminability are equally important, but for a different reason. Without an adequate theoretical understanding of the relevant task, the field will be in no position to enhance empirical Discriminability.
-
roc analysis measures objective Discriminability for any eyewitness identification procedure
Journal of applied research in memory and cognition, 2015Co-Authors: John T Wixted, Laura MickesAbstract:Abstract Which eyewitness identification procedure better enables eyewitnesses to discriminate between innocent and guilty suspects? In other words, which procedure better enables eyewitnesses to sort innocent and guilty suspects into their correct categories? The answer to that objective, theory-free question is what policymakers need to know, and it is precisely the information that ROC analysis provides. Wells et al. largely ignore that question and focus instead on whether ROC analysis accurately measures underlying (theoretical) Discriminability for lineups. They argue that the apparent Discriminability advantage for lineups over showups is an illusion caused by “filler siphoning.” Here, we demonstrate that, both objectively and theoretically, the ability of eyewitnesses to discriminate innocent from guilty suspects is higher for lineups compared to showups, just as the ROC data suggest. Intuitions notwithstanding, filler siphoning does not account for the Discriminability advantage for lineups. An actual theory of Discriminability is needed to explain that interesting phenomenon.
-
a signal detection based diagnostic feature detection model of eyewitness identification
Psychological Review, 2014Co-Authors: John T Wixted, Laura MickesAbstract:The theoretical understanding of eyewitness identifications made from a police lineup has long been guided by the distinction between absolute and relative decision strategies. In addition, the accuracy of identifications associated with different eyewitness memory procedures has long been evaluated using measures like the diagnosticity ratio (the correct identification rate divided by the false identification rate). Framed in terms of signal-detection theory, both the absolute/relative distinction and the diagnosticity ratio are mainly relevant to response bias while remaining silent about the key issue of diagnostic accuracy, or Discriminability (i.e., the ability to tell the difference between innocent and guilty suspects in a lineup). Here, we propose a signal-detection-based model of eyewitness identification, one that encourages the use of (and helps to conceptualize) receiver operating characteristic (ROC) analysis to measure Discriminability. Recent ROC analyses indicate that the simultaneous presentation of faces in a lineup yields higher Discriminability than the presentation of faces in isolation, and we propose a diagnostic feature-detection hypothesis to account for that result. According to this hypothesis, the simultaneous presentation of faces allows the eyewitness to appreciate that certain facial features (viz., those that are shared by everyone in the lineup) are non-diagnostic of guilt. To the extent that those non-diagnostic features are discounted in favor of potentially more diagnostic features, the ability to discriminate innocent from guilty suspects will be enhanced.
Andrew M Smith - One of the best experts on this subject based on the ideXlab platform.
-
fair lineups are better than biased lineups and showups but not because they increase underlying Discriminability
Law and Human Behavior, 2017Co-Authors: Andrew M Smith, Gary L Wells, R C L Lindsay, Steven D PenrodAbstract:Receiver Operating Characteristic (ROC) analysis has recently come in vogue for assessing the underlying Discriminability and the applied utility of lineup procedures. Two primary assumptions underlie recommendations that ROC analysis be used to assess the applied utility of lineup procedures: (a) ROC analysis of lineups measures underlying Discriminability, and (b) the procedure that produces superior underlying Discriminability produces superior applied utility. These same assumptions underlie a recently derived diagnostic-feature detection theory, a theory of Discriminability, intended to explain recent patterns observed in ROC comparisons of lineups. We demonstrate, however, that these assumptions are incorrect when ROC analysis is applied to lineups. We also demonstrate that a structural phenomenon of lineups, differential filler siphoning, and not the psychological phenomenon of diagnostic-feature detection, explains why lineups are superior to showups and why fair lineups are superior to biased lineups. In the process of our proofs, we show that computational simulations have assumed, unrealistically, that all witnesses share exactly the same decision criteria. When criterial variance is included in computational models, differential filler siphoning emerges. The result proves dissociation between ROC curves and underlying Discriminability: Higher ROC curves for lineups than for showups and for fair than for biased lineups despite no increase in underlying Discriminability. (PsycINFO Database Record
-
roc analysis of lineups does not measure underlying Discriminability and has limited value
Journal of applied research in memory and cognition, 2015Co-Authors: Gary L Wells, Laura Smalarz, Andrew M SmithAbstract:Abstract Some researchers have been arguing that eyewitness identification data from lineups should be analyzed using Receiver Operating Characteristic (ROC) analysis because it purportedly measures underlying Discriminability. But ROC analysis, which was designed for 2 × 2 tasks, does not fit the 3 × 2 structure of lineups. Accordingly, ROC proponents force lineup data into a 2 × 2 structure by treating false-positive identifications of lineup fillers as though they were rejections. Using data from lineups versus showups, we illustrate how this approach misfires as a measure of underlying Discriminability. Moreover, treating false-positive identifications of fillers as if they were rejections hides one of the most important phenomena in eyewitness lineups, namely filler siphoning. Filler siphoning reduces the risk of mistaken identification by drawing false-positive identifications away from the innocent suspect and onto lineup fillers. We show that ROC analysis confuses filler siphoning with an improvement in underlying Discriminability, thereby fostering misleading theoretical conclusions about how lineups work.
Joseph H Ricker - One of the best experts on this subject based on the ideXlab platform.
-
the california verbal learning test in the detection of incomplete effort in neuropsychological evaluation
Psychological Assessment, 1995Co-Authors: Scott R Millis, Joseph H Ricker, Steven H Putnam, Kenneth M AdamsAbstract:This study determined whether performance patterns on four California Verbal Learning Test variables (CVLT ; Trials 1-5 List A, Discriminability, recognition hits, and long-delay cued recall) could differentiate participants with moderate and severe brain injuries from those with mild head injuries who were giving incomplete effort. Litigating mild head injury participants (n = 23) performing at chance level or worse on a forced-choice test obtained significantly lower scores on the four CVLT variables than participants with moderate and severe brain injuries (n = 23). The linear discriminant function accurately classified 91%, and the quadratic function, 96%. The Discriminability cutoff score accurately classified 93% of the cases ; recognition hits, 89% ; long-delay cued recall, 87% ; and CVLT total, 83%.
Laurent Lessard - One of the best experts on this subject based on the ideXlab platform.
-
semantic Discriminability for visual communication
IEEE Transactions on Visualization and Computer Graphics, 2021Co-Authors: Karen B Schloss, Zachary Leggon, Laurent LessardAbstract:To interpret information visualizations, observers must determine how visual features map onto concepts. First and foremost, this ability depends on perceptual Discriminability; observers must be able to see the difference between different colors for those colors to communicate different meanings. However, the ability to interpret visualizations also depends on semantic Discriminability, the degree to which observers can infer a unique mapping between visual features and concepts, based on the visual features and concepts alone (i.e., without help from verbal cues such as legends or labels). Previous evidence suggested that observers were better at interpreting encoding systems that maximized semantic Discriminability (maximizing association strength between assigned colors and concepts while minimizing association strength between unassigned colors and concepts), compared to a system that only maximized color-concept association strength. However, increasing semantic Discriminability also resulted in increased perceptual distance, so it is unclear which factor was responsible for improved performance. In the present study, we conducted two experiments that tested for independent effects of semantic distance and perceptual distance on semantic Discriminability of bar graph data visualizations. Perceptual distance was large enough to ensure colors were more than just noticeably different. We found that increasing semantic distance improved performance, independent of variation in perceptual distance, and when these two factors were uncorrelated, responses were dominated by semantic distance. These results have implications for navigating trade-offs in color palette design optimization for visual communication.
-
semantic Discriminability for visual communication
arXiv: Human-Computer Interaction, 2020Co-Authors: Karen B Schloss, Zachary Leggon, Laurent LessardAbstract:To interpret information visualizations, observers must determine how visual features map onto concepts. First and foremost, this ability depends on perceptual Discriminability; e.g., observers must be able to see the difference between different colors for those colors to communicate different meanings. However, the ability to interpret visualizations also depends on semantic Discriminability, the degree to which observers can infer a unique mapping between visual features and concepts, based on the visual features and concepts alone (i.e., without help from verbal cues such as legends or labels). Previous evidence suggested that observers were better at interpreting encoding systems that maximized semantic Discriminability (maximizing association strength between assigned colors and concepts while minimizing association strength between unassigned colors and concepts), compared to a system that only maximized color-concept association strength. However, increasing semantic Discriminability also resulted in increased perceptual distance, so it is unclear which factor was responsible for improved performance. In the present study, we conducted two experiments that tested for independent effects of semantic distance and perceptual distance on semantic Discriminability of bar graph data visualizations. Perceptual distance was large enough to ensure colors were more than just noticeably different. We found that increasing semantic distance improved performance, independent of variation in perceptual distance, and when these two factors were uncorrelated, responses were dominated by semantic distance. These results have implications for navigating trade-offs in color palette design optimization for visual communication.
