The Experts below are selected from a list of 3285 Experts worldwide ranked by ideXlab platform

Arun R. Vemury - One of the best experts on this subject based on the ideXlab platform.

  • Demographic Effects in Facial Recognition and Their Dependence on Image Acquisition: An Evaluation of Eleven Commercial Systems
    IEEE Transactions on Biometrics Behavior and Identity Science, 2019
    Co-Authors: Cynthia M. Cook, John J. Howard, Yevgeniy B. Sirotin, Jerry L. Tipton, Arun R. Vemury
    Abstract:

    We examined the effect of demographic factors on the performance of 11 commercial face biometric systems tested as part of the 2018 U.S. Department of Homeland Security, Science and Technology Directorate biometric technology rally. Each system that participated in this evaluation was tasked with acquiring face images from a diverse population of 363 subjects in a controlled environment. Biometric performance was assessed using a systematic, repeatable test process measuring both efficiency (transaction times), and accuracy (mated similarity scores) using a leading commercial algorithm. Prior works have documented differences in biometric algorithm performance across demographic categories and proposed that skin phenotypes offer a superior explanation for these differences. To test this concept, we developed an automatic method for measuring relative facial skin reflectance using subjects' enrollment images and quantified the effect of this metric and other demographic covariates on performance using linear modeling. Both the efficiency and accuracy of the tested acquisition systems were significantly affected by multiple demographic covariates, including skin reflectance, gender, age, eyewear, and height. Skin reflectance had the strongest net linear effect on performance. Linear modeling showed that lower (darker) skin reflectance was associated with lower efficiency (higher transaction times) and accuracy (lower mated similarity scores). Skin reflectance was also a statistically better predictor of these effects than self-identified race labels. Unlike other covariates, the degree to which skin reflectance altered accuracy varied between systems. We show that the size of this skin reflectance effect was inversely related to the overall accuracy of the system such that the effect was almost negligible for the system with the highest overall accuracy. These results suggest that, in evaluations of biometric accuracy, the magnitude of measured demographic effects depends on image acquisition.

  • On Efficiency and Effectiveness Tradeoffs in High-Throughput Facial Biometric Recognition Systems
    2018 IEEE 9th International Conference on Biometrics Theory Applications and Systems (BTAS), 2018
    Co-Authors: John J. Howard, Jacob A. Hasselgren, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun R. Vemury
    Abstract:

    This research discusses the evaluation of biometric systems that are designed to process hundreds to tens of thousands of individuals in short time spans. We propose a method for evaluating a system's performance across capture attempts for the purpose of identifying characteristics that are advantageous in these high-throughput environments. We also present a novel modification to the traditionally accepted biometric performance metrics of failure-to-acquire, and true-match rate. Namely, this paradigm shift holds that these metrics are a function of time and, as such, vary with the time available for a biometric system to interact with a user. This research demonstrates the utility of these time-based metrics in evaluating the performance of multiple, commercially available, high-throughput systems. We show that different biometric systems have notably different time-based performance curves using a corpus of data collected during the 2018 Department of Homeland Security, Science and Technology Directorate (DHS S&T) Biometric Technology Rally. These curves and the deviations between them are useful when quantifying the suitability of a technology, evaluated via scenario testing, for deployment in an operational environment where the throughput of the target population is a key performance parameter.

  • An Investigation of High-Throughput Biometric Systems: Results of the 2018 Department of Homeland Security Biometric Technology Rally
    2018 IEEE 9th International Conference on Biometrics Theory Applications and Systems (BTAS), 2018
    Co-Authors: John J. Howard, Jacob A. Hasselgren, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun R. Vemury
    Abstract:

    The 2018 Biometric Technology Rally was an evaluation, sponsored by the U.S. Department of Homeland Security, Science and Technology Directorate (DHS S&T), that challenged industry to provide face or face/iris systems capable of unmanned, traveler identification in a high-throughput Security environment. Selected systems were installed at the Maryland Test Facility (MdTF), a DHS S&T affiliated bio-metrics testing laboratory, and evaluated using a population of 363 naive human subjects recruited from the general public. The performance of each system was examined based on measured throughput, capture capability, matching capability, and user satisfaction metrics. This research documents the performance of unmanned face and face/iris systems required to maintain an average total subject interaction time of less than 10 seconds. The results highlight discrepancies between the performance of biometric systems as anticipated by the system designers and the measured performance, indicating an incomplete understanding of the main determinants of system performance. Our research shows that failure-to-acquire errors, unpredicted by system designers, were the main driver of non-identification rates instead of failure-to-match errors, which were better predicted. This outcome indicates the need for a renewed focus on reducing the failure-to-acquire rate in high-throughput, unmanned biometric systems.

John J. Howard - One of the best experts on this subject based on the ideXlab platform.

  • Demographic Effects in Facial Recognition and Their Dependence on Image Acquisition: An Evaluation of Eleven Commercial Systems
    IEEE Transactions on Biometrics Behavior and Identity Science, 2019
    Co-Authors: Cynthia M. Cook, John J. Howard, Yevgeniy B. Sirotin, Jerry L. Tipton, Arun R. Vemury
    Abstract:

    We examined the effect of demographic factors on the performance of 11 commercial face biometric systems tested as part of the 2018 U.S. Department of Homeland Security, Science and Technology Directorate biometric technology rally. Each system that participated in this evaluation was tasked with acquiring face images from a diverse population of 363 subjects in a controlled environment. Biometric performance was assessed using a systematic, repeatable test process measuring both efficiency (transaction times), and accuracy (mated similarity scores) using a leading commercial algorithm. Prior works have documented differences in biometric algorithm performance across demographic categories and proposed that skin phenotypes offer a superior explanation for these differences. To test this concept, we developed an automatic method for measuring relative facial skin reflectance using subjects' enrollment images and quantified the effect of this metric and other demographic covariates on performance using linear modeling. Both the efficiency and accuracy of the tested acquisition systems were significantly affected by multiple demographic covariates, including skin reflectance, gender, age, eyewear, and height. Skin reflectance had the strongest net linear effect on performance. Linear modeling showed that lower (darker) skin reflectance was associated with lower efficiency (higher transaction times) and accuracy (lower mated similarity scores). Skin reflectance was also a statistically better predictor of these effects than self-identified race labels. Unlike other covariates, the degree to which skin reflectance altered accuracy varied between systems. We show that the size of this skin reflectance effect was inversely related to the overall accuracy of the system such that the effect was almost negligible for the system with the highest overall accuracy. These results suggest that, in evaluations of biometric accuracy, the magnitude of measured demographic effects depends on image acquisition.

  • Operational Tradeoffs in the 2018 Department of Homeland Security Science and Technology Directorate Biometric Technology Rally
    2018 IEEE International Symposium on Technologies for Homeland Security (HST), 2018
    Co-Authors: Jacob A. Hasselgren, John J. Howard, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun Vemury
    Abstract:

    The 2018 Biometric Technology Rally was an evaluation, sponsored by the U.S. Department of Homeland Security (DHS), Science and Technology (S&T) Directorate, that challenged industry to provide face or face/iris systems capable of unmanned, traveler identification in a high-throughput Security environment. Eleven selected systems were installed at the Maryland Test Facility (MdTF), a DHS S&T affiliated biometrics testing laboratory, and evaluated using a sample of 363 naive human subjects recruited from the general public. The performance of each system was examined based on measured throughput (efficiency), matching capability (effectiveness), and user satisfaction. This research documents the operational tradeoffs between these three measures of system performance. Specifically, we perform two tradeoff analyses: efficiency versus effectiveness and satisfaction versus both efficiency and effectiveness. These tradeoff analyses allow us to determine how and if these three performance measures are related in the various kinds of biometric systems we tested. For example, are higher throughput systems also more effective? Do people prefer systems that are faster or more effective? Our results show there is no clear relationship between how quickly a system can process a user and how well it can identify the user. Furthermore, there was also no significant relationship observed between how quickly a system can process a user and how satisfied the user is with the system. However, there was a strong relationship between how well a system identifies a user and how satisfied the user is with the system. These outcomes suggest that some systems could benefit by leveraging additional collection time to collect a higher quality image. Users did not tend to prefer faster systems but did prefer a system they thought was working as intended. Finally, these results also show that in regards to public acceptance, systems designers should focus on correctly identifying larger populations of users rather than how quickly a given user can be processed.

  • On Efficiency and Effectiveness Tradeoffs in High-Throughput Facial Biometric Recognition Systems
    2018 IEEE 9th International Conference on Biometrics Theory Applications and Systems (BTAS), 2018
    Co-Authors: John J. Howard, Jacob A. Hasselgren, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun R. Vemury
    Abstract:

    This research discusses the evaluation of biometric systems that are designed to process hundreds to tens of thousands of individuals in short time spans. We propose a method for evaluating a system's performance across capture attempts for the purpose of identifying characteristics that are advantageous in these high-throughput environments. We also present a novel modification to the traditionally accepted biometric performance metrics of failure-to-acquire, and true-match rate. Namely, this paradigm shift holds that these metrics are a function of time and, as such, vary with the time available for a biometric system to interact with a user. This research demonstrates the utility of these time-based metrics in evaluating the performance of multiple, commercially available, high-throughput systems. We show that different biometric systems have notably different time-based performance curves using a corpus of data collected during the 2018 Department of Homeland Security, Science and Technology Directorate (DHS S&T) Biometric Technology Rally. These curves and the deviations between them are useful when quantifying the suitability of a technology, evaluated via scenario testing, for deployment in an operational environment where the throughput of the target population is a key performance parameter.

  • An Investigation of High-Throughput Biometric Systems: Results of the 2018 Department of Homeland Security Biometric Technology Rally
    2018 IEEE 9th International Conference on Biometrics Theory Applications and Systems (BTAS), 2018
    Co-Authors: John J. Howard, Jacob A. Hasselgren, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun R. Vemury
    Abstract:

    The 2018 Biometric Technology Rally was an evaluation, sponsored by the U.S. Department of Homeland Security, Science and Technology Directorate (DHS S&T), that challenged industry to provide face or face/iris systems capable of unmanned, traveler identification in a high-throughput Security environment. Selected systems were installed at the Maryland Test Facility (MdTF), a DHS S&T affiliated bio-metrics testing laboratory, and evaluated using a population of 363 naive human subjects recruited from the general public. The performance of each system was examined based on measured throughput, capture capability, matching capability, and user satisfaction metrics. This research documents the performance of unmanned face and face/iris systems required to maintain an average total subject interaction time of less than 10 seconds. The results highlight discrepancies between the performance of biometric systems as anticipated by the system designers and the measured performance, indicating an incomplete understanding of the main determinants of system performance. Our research shows that failure-to-acquire errors, unpredicted by system designers, were the main driver of non-identification rates instead of failure-to-match errors, which were better predicted. This outcome indicates the need for a renewed focus on reducing the failure-to-acquire rate in high-throughput, unmanned biometric systems.

Jacob A. Hasselgren - One of the best experts on this subject based on the ideXlab platform.

  • Operational Tradeoffs in the 2018 Department of Homeland Security Science and Technology Directorate Biometric Technology Rally
    2018 IEEE International Symposium on Technologies for Homeland Security (HST), 2018
    Co-Authors: Jacob A. Hasselgren, John J. Howard, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun Vemury
    Abstract:

    The 2018 Biometric Technology Rally was an evaluation, sponsored by the U.S. Department of Homeland Security (DHS), Science and Technology (S&T) Directorate, that challenged industry to provide face or face/iris systems capable of unmanned, traveler identification in a high-throughput Security environment. Eleven selected systems were installed at the Maryland Test Facility (MdTF), a DHS S&T affiliated biometrics testing laboratory, and evaluated using a sample of 363 naive human subjects recruited from the general public. The performance of each system was examined based on measured throughput (efficiency), matching capability (effectiveness), and user satisfaction. This research documents the operational tradeoffs between these three measures of system performance. Specifically, we perform two tradeoff analyses: efficiency versus effectiveness and satisfaction versus both efficiency and effectiveness. These tradeoff analyses allow us to determine how and if these three performance measures are related in the various kinds of biometric systems we tested. For example, are higher throughput systems also more effective? Do people prefer systems that are faster or more effective? Our results show there is no clear relationship between how quickly a system can process a user and how well it can identify the user. Furthermore, there was also no significant relationship observed between how quickly a system can process a user and how satisfied the user is with the system. However, there was a strong relationship between how well a system identifies a user and how satisfied the user is with the system. These outcomes suggest that some systems could benefit by leveraging additional collection time to collect a higher quality image. Users did not tend to prefer faster systems but did prefer a system they thought was working as intended. Finally, these results also show that in regards to public acceptance, systems designers should focus on correctly identifying larger populations of users rather than how quickly a given user can be processed.

  • On Efficiency and Effectiveness Tradeoffs in High-Throughput Facial Biometric Recognition Systems
    2018 IEEE 9th International Conference on Biometrics Theory Applications and Systems (BTAS), 2018
    Co-Authors: John J. Howard, Jacob A. Hasselgren, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun R. Vemury
    Abstract:

    This research discusses the evaluation of biometric systems that are designed to process hundreds to tens of thousands of individuals in short time spans. We propose a method for evaluating a system's performance across capture attempts for the purpose of identifying characteristics that are advantageous in these high-throughput environments. We also present a novel modification to the traditionally accepted biometric performance metrics of failure-to-acquire, and true-match rate. Namely, this paradigm shift holds that these metrics are a function of time and, as such, vary with the time available for a biometric system to interact with a user. This research demonstrates the utility of these time-based metrics in evaluating the performance of multiple, commercially available, high-throughput systems. We show that different biometric systems have notably different time-based performance curves using a corpus of data collected during the 2018 Department of Homeland Security, Science and Technology Directorate (DHS S&T) Biometric Technology Rally. These curves and the deviations between them are useful when quantifying the suitability of a technology, evaluated via scenario testing, for deployment in an operational environment where the throughput of the target population is a key performance parameter.

  • An Investigation of High-Throughput Biometric Systems: Results of the 2018 Department of Homeland Security Biometric Technology Rally
    2018 IEEE 9th International Conference on Biometrics Theory Applications and Systems (BTAS), 2018
    Co-Authors: John J. Howard, Jacob A. Hasselgren, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun R. Vemury
    Abstract:

    The 2018 Biometric Technology Rally was an evaluation, sponsored by the U.S. Department of Homeland Security, Science and Technology Directorate (DHS S&T), that challenged industry to provide face or face/iris systems capable of unmanned, traveler identification in a high-throughput Security environment. Selected systems were installed at the Maryland Test Facility (MdTF), a DHS S&T affiliated bio-metrics testing laboratory, and evaluated using a population of 363 naive human subjects recruited from the general public. The performance of each system was examined based on measured throughput, capture capability, matching capability, and user satisfaction metrics. This research documents the performance of unmanned face and face/iris systems required to maintain an average total subject interaction time of less than 10 seconds. The results highlight discrepancies between the performance of biometric systems as anticipated by the system designers and the measured performance, indicating an incomplete understanding of the main determinants of system performance. Our research shows that failure-to-acquire errors, unpredicted by system designers, were the main driver of non-identification rates instead of failure-to-match errors, which were better predicted. This outcome indicates the need for a renewed focus on reducing the failure-to-acquire rate in high-throughput, unmanned biometric systems.

Andrew J. Blanchard - One of the best experts on this subject based on the ideXlab platform.

  • Operational Tradeoffs in the 2018 Department of Homeland Security Science and Technology Directorate Biometric Technology Rally
    2018 IEEE International Symposium on Technologies for Homeland Security (HST), 2018
    Co-Authors: Jacob A. Hasselgren, John J. Howard, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun Vemury
    Abstract:

    The 2018 Biometric Technology Rally was an evaluation, sponsored by the U.S. Department of Homeland Security (DHS), Science and Technology (S&T) Directorate, that challenged industry to provide face or face/iris systems capable of unmanned, traveler identification in a high-throughput Security environment. Eleven selected systems were installed at the Maryland Test Facility (MdTF), a DHS S&T affiliated biometrics testing laboratory, and evaluated using a sample of 363 naive human subjects recruited from the general public. The performance of each system was examined based on measured throughput (efficiency), matching capability (effectiveness), and user satisfaction. This research documents the operational tradeoffs between these three measures of system performance. Specifically, we perform two tradeoff analyses: efficiency versus effectiveness and satisfaction versus both efficiency and effectiveness. These tradeoff analyses allow us to determine how and if these three performance measures are related in the various kinds of biometric systems we tested. For example, are higher throughput systems also more effective? Do people prefer systems that are faster or more effective? Our results show there is no clear relationship between how quickly a system can process a user and how well it can identify the user. Furthermore, there was also no significant relationship observed between how quickly a system can process a user and how satisfied the user is with the system. However, there was a strong relationship between how well a system identifies a user and how satisfied the user is with the system. These outcomes suggest that some systems could benefit by leveraging additional collection time to collect a higher quality image. Users did not tend to prefer faster systems but did prefer a system they thought was working as intended. Finally, these results also show that in regards to public acceptance, systems designers should focus on correctly identifying larger populations of users rather than how quickly a given user can be processed.

  • On Efficiency and Effectiveness Tradeoffs in High-Throughput Facial Biometric Recognition Systems
    2018 IEEE 9th International Conference on Biometrics Theory Applications and Systems (BTAS), 2018
    Co-Authors: John J. Howard, Jacob A. Hasselgren, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun R. Vemury
    Abstract:

    This research discusses the evaluation of biometric systems that are designed to process hundreds to tens of thousands of individuals in short time spans. We propose a method for evaluating a system's performance across capture attempts for the purpose of identifying characteristics that are advantageous in these high-throughput environments. We also present a novel modification to the traditionally accepted biometric performance metrics of failure-to-acquire, and true-match rate. Namely, this paradigm shift holds that these metrics are a function of time and, as such, vary with the time available for a biometric system to interact with a user. This research demonstrates the utility of these time-based metrics in evaluating the performance of multiple, commercially available, high-throughput systems. We show that different biometric systems have notably different time-based performance curves using a corpus of data collected during the 2018 Department of Homeland Security, Science and Technology Directorate (DHS S&T) Biometric Technology Rally. These curves and the deviations between them are useful when quantifying the suitability of a technology, evaluated via scenario testing, for deployment in an operational environment where the throughput of the target population is a key performance parameter.

  • An Investigation of High-Throughput Biometric Systems: Results of the 2018 Department of Homeland Security Biometric Technology Rally
    2018 IEEE 9th International Conference on Biometrics Theory Applications and Systems (BTAS), 2018
    Co-Authors: John J. Howard, Jacob A. Hasselgren, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun R. Vemury
    Abstract:

    The 2018 Biometric Technology Rally was an evaluation, sponsored by the U.S. Department of Homeland Security, Science and Technology Directorate (DHS S&T), that challenged industry to provide face or face/iris systems capable of unmanned, traveler identification in a high-throughput Security environment. Selected systems were installed at the Maryland Test Facility (MdTF), a DHS S&T affiliated bio-metrics testing laboratory, and evaluated using a population of 363 naive human subjects recruited from the general public. The performance of each system was examined based on measured throughput, capture capability, matching capability, and user satisfaction metrics. This research documents the performance of unmanned face and face/iris systems required to maintain an average total subject interaction time of less than 10 seconds. The results highlight discrepancies between the performance of biometric systems as anticipated by the system designers and the measured performance, indicating an incomplete understanding of the main determinants of system performance. Our research shows that failure-to-acquire errors, unpredicted by system designers, were the main driver of non-identification rates instead of failure-to-match errors, which were better predicted. This outcome indicates the need for a renewed focus on reducing the failure-to-acquire rate in high-throughput, unmanned biometric systems.

Yevgeniy B. Sirotin - One of the best experts on this subject based on the ideXlab platform.

  • Demographic Effects in Facial Recognition and Their Dependence on Image Acquisition: An Evaluation of Eleven Commercial Systems
    IEEE Transactions on Biometrics Behavior and Identity Science, 2019
    Co-Authors: Cynthia M. Cook, John J. Howard, Yevgeniy B. Sirotin, Jerry L. Tipton, Arun R. Vemury
    Abstract:

    We examined the effect of demographic factors on the performance of 11 commercial face biometric systems tested as part of the 2018 U.S. Department of Homeland Security, Science and Technology Directorate biometric technology rally. Each system that participated in this evaluation was tasked with acquiring face images from a diverse population of 363 subjects in a controlled environment. Biometric performance was assessed using a systematic, repeatable test process measuring both efficiency (transaction times), and accuracy (mated similarity scores) using a leading commercial algorithm. Prior works have documented differences in biometric algorithm performance across demographic categories and proposed that skin phenotypes offer a superior explanation for these differences. To test this concept, we developed an automatic method for measuring relative facial skin reflectance using subjects' enrollment images and quantified the effect of this metric and other demographic covariates on performance using linear modeling. Both the efficiency and accuracy of the tested acquisition systems were significantly affected by multiple demographic covariates, including skin reflectance, gender, age, eyewear, and height. Skin reflectance had the strongest net linear effect on performance. Linear modeling showed that lower (darker) skin reflectance was associated with lower efficiency (higher transaction times) and accuracy (lower mated similarity scores). Skin reflectance was also a statistically better predictor of these effects than self-identified race labels. Unlike other covariates, the degree to which skin reflectance altered accuracy varied between systems. We show that the size of this skin reflectance effect was inversely related to the overall accuracy of the system such that the effect was almost negligible for the system with the highest overall accuracy. These results suggest that, in evaluations of biometric accuracy, the magnitude of measured demographic effects depends on image acquisition.

  • Operational Tradeoffs in the 2018 Department of Homeland Security Science and Technology Directorate Biometric Technology Rally
    2018 IEEE International Symposium on Technologies for Homeland Security (HST), 2018
    Co-Authors: Jacob A. Hasselgren, John J. Howard, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun Vemury
    Abstract:

    The 2018 Biometric Technology Rally was an evaluation, sponsored by the U.S. Department of Homeland Security (DHS), Science and Technology (S&T) Directorate, that challenged industry to provide face or face/iris systems capable of unmanned, traveler identification in a high-throughput Security environment. Eleven selected systems were installed at the Maryland Test Facility (MdTF), a DHS S&T affiliated biometrics testing laboratory, and evaluated using a sample of 363 naive human subjects recruited from the general public. The performance of each system was examined based on measured throughput (efficiency), matching capability (effectiveness), and user satisfaction. This research documents the operational tradeoffs between these three measures of system performance. Specifically, we perform two tradeoff analyses: efficiency versus effectiveness and satisfaction versus both efficiency and effectiveness. These tradeoff analyses allow us to determine how and if these three performance measures are related in the various kinds of biometric systems we tested. For example, are higher throughput systems also more effective? Do people prefer systems that are faster or more effective? Our results show there is no clear relationship between how quickly a system can process a user and how well it can identify the user. Furthermore, there was also no significant relationship observed between how quickly a system can process a user and how satisfied the user is with the system. However, there was a strong relationship between how well a system identifies a user and how satisfied the user is with the system. These outcomes suggest that some systems could benefit by leveraging additional collection time to collect a higher quality image. Users did not tend to prefer faster systems but did prefer a system they thought was working as intended. Finally, these results also show that in regards to public acceptance, systems designers should focus on correctly identifying larger populations of users rather than how quickly a given user can be processed.

  • On Efficiency and Effectiveness Tradeoffs in High-Throughput Facial Biometric Recognition Systems
    2018 IEEE 9th International Conference on Biometrics Theory Applications and Systems (BTAS), 2018
    Co-Authors: John J. Howard, Jacob A. Hasselgren, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun R. Vemury
    Abstract:

    This research discusses the evaluation of biometric systems that are designed to process hundreds to tens of thousands of individuals in short time spans. We propose a method for evaluating a system's performance across capture attempts for the purpose of identifying characteristics that are advantageous in these high-throughput environments. We also present a novel modification to the traditionally accepted biometric performance metrics of failure-to-acquire, and true-match rate. Namely, this paradigm shift holds that these metrics are a function of time and, as such, vary with the time available for a biometric system to interact with a user. This research demonstrates the utility of these time-based metrics in evaluating the performance of multiple, commercially available, high-throughput systems. We show that different biometric systems have notably different time-based performance curves using a corpus of data collected during the 2018 Department of Homeland Security, Science and Technology Directorate (DHS S&T) Biometric Technology Rally. These curves and the deviations between them are useful when quantifying the suitability of a technology, evaluated via scenario testing, for deployment in an operational environment where the throughput of the target population is a key performance parameter.

  • An Investigation of High-Throughput Biometric Systems: Results of the 2018 Department of Homeland Security Biometric Technology Rally
    2018 IEEE 9th International Conference on Biometrics Theory Applications and Systems (BTAS), 2018
    Co-Authors: John J. Howard, Jacob A. Hasselgren, Yevgeniy B. Sirotin, Andrew J. Blanchard, Arun R. Vemury
    Abstract:

    The 2018 Biometric Technology Rally was an evaluation, sponsored by the U.S. Department of Homeland Security, Science and Technology Directorate (DHS S&T), that challenged industry to provide face or face/iris systems capable of unmanned, traveler identification in a high-throughput Security environment. Selected systems were installed at the Maryland Test Facility (MdTF), a DHS S&T affiliated bio-metrics testing laboratory, and evaluated using a population of 363 naive human subjects recruited from the general public. The performance of each system was examined based on measured throughput, capture capability, matching capability, and user satisfaction metrics. This research documents the performance of unmanned face and face/iris systems required to maintain an average total subject interaction time of less than 10 seconds. The results highlight discrepancies between the performance of biometric systems as anticipated by the system designers and the measured performance, indicating an incomplete understanding of the main determinants of system performance. Our research shows that failure-to-acquire errors, unpredicted by system designers, were the main driver of non-identification rates instead of failure-to-match errors, which were better predicted. This outcome indicates the need for a renewed focus on reducing the failure-to-acquire rate in high-throughput, unmanned biometric systems.