The Experts below are selected from a list of 149526 Experts worldwide ranked by ideXlab platform
Mj Schuemie - One of the best experts on this subject based on the ideXlab platform.
-
empirical assessment of case based methods for Drug Safety alert identification in the french national healthcare system database snds methodology of the alcapone project
Pharmacoepidemiology and Drug Safety, 2020Co-Authors: N Thurin, Mj Schuemie, R Lassalle, Marine Penichon, Joshua J Gagne, Jeremy A Rassen, Jacques Benichou, Alain Weill, P Blin, Nicholas MooreAbstract:OBJECTIVES To introduce the methodology of the ALCAPONE project. BACKGROUND The French National Healthcare System Database (SNDS), covering 99% of the French population, provides a potentially valuable opportunity for Drug Safety alert generation. ALCAPONE aimed to assess empirically in the SNDS case-based designs for alert generation related to four health outcomes of interest. METHODS ALCAPONE used a reference set adapted from observational medical outcomes partnership (OMOP) and Exploring and Understanding Adverse Drug Reactions (EU-ADR) project, with four outcomes-acute liver injury (ALI), myocardial infarction (MI), acute kidney injury (AKI), and upper gastrointestinal bleeding (UGIB)-and positive and negative Drug controls. ALCAPONE consisted of four main phases: (1) data preparation to fit the OMOP Common Data Model and select the Drug controls; (2) detection of the selected controls via three case-based designs: case-population, case-control, and self-controlled case series, including design variants (varying risk window, adjustment strategy, etc.); (3) comparison of design variant performance (area under the ROC curve, mean square error, etc.); and (4) selection of the optimal design variants and their calibration for each outcome. RESULTS Over 2009-2014, 5225 cases of ALI, 354 109 MI, 12 633 AKI, and 156 057 UGIB were identified using specific definitions. The number of detectable Drugs ranged from 61 for MI to 25 for ALI. Design variants generated more than 50 000 points estimates. Results by outcome will be published in forthcoming papers. CONCLUSIONS ALCAPONE has shown the interest of the empirical assessment of pharmacoepidemiological approaches for Drug Safety alert generation and may encourage other researchers to do the same in other databases.
-
defining a reference set to support methodological research in Drug Safety
Drug Safety, 2013Co-Authors: Mj Schuemie, Patrick B Ryan, Emily Welebob, Jon D Duke, Sarah Valentine, Abraham G HartzemaAbstract:Background Methodological research to evaluate the performance of methods requires a benchmark to serve as a referent comparison. In Drug Safety, the performance of analyses of spontaneous adverse event reporting databases and observational healthcare data, such as administrative claims and electronic health records, has been limited by the lack of such standards.
-
defining a reference set to support methodological research in Drug Safety
Drug Safety, 2013Co-Authors: Mj Schuemie, Patrick B Ryan, Emily Welebob, Sarah Valentine, Abraham G Hartzema, Jon DukeAbstract:Methodological research to evaluate the performance of methods requires a benchmark to serve as a referent comparison. In Drug Safety, the performance of analyses of spontaneous adverse event reporting databases and observational healthcare data, such as administrative claims and electronic health records, has been limited by the lack of such standards. To establish a reference set of test cases that contain both positive and negative controls, which can serve the basis for methodological research in evaluating methods performance in identifying Drug Safety issues. Systematic literature review and natural language processing of structured product labeling was performed to identify evidence to support the classification of Drugs as either positive controls or negative controls for four outcomes: acute liver injury, acute kidney injury, acute myocardial infarction, and upper gastrointestinal bleeding. Three-hundred and ninety-nine test cases comprised of 165 positive controls and 234 negative controls were identified across the four outcomes. The majority of positive controls for acute kidney injury and upper gastrointestinal bleeding were supported by randomized clinical trial evidence, while the majority of positive controls for acute liver injury and acute myocardial infarction were only supported based on published case reports. Literature estimates for the positive controls shows substantial variability that limits the ability to establish a reference set with known effect sizes. A reference set of test cases can be established to facilitate methodological research in Drug Safety. Creating a sufficient sample of Drug-outcome pairs with binary classification of having no effect (negative controls) or having an increased effect (positive controls) is possible and can enable estimation of predictive accuracy through discrimination. Since the magnitude of the positive effects cannot be reliably obtained and the quality of evidence may vary across outcomes, assumptions are required to use the test cases in real data for purposes of measuring bias, mean squared error, or coverage probability.
-
a reference standard for evaluation of methods for Drug Safety signal detection using electronic healthcare record databases
Drug Safety, 2013Co-Authors: P M Coloma, Mj Schuemie, Paul Avillach, Carmen Ferrajolo, Antoine Pariente, Francesco Salvo, Annie FourrierreglatAbstract:Background The growing interest in using electronic healthcare record (EHR) databases for Drug Safety surveillance has spurred development of new methodologies for signal detection. Although several Drugs have been withdrawn postmarketing by regulatory authorities after scientific evaluation of harms and benefits, there is no definitive list of confirmed signals (i.e. list of all known adverse reactions and which Drugs can cause them). As there is no true gold standard, prospective evaluation of signal detection methods remains a challenge.
-
electronic healthcare databases for active Drug Safety surveillance is there enough leverage
Pharmacoepidemiology and Drug Safety, 2012Co-Authors: P M Coloma, Mj Schuemie, Gianluca Trifiro, Rosa Gini, Ron M C Herings, Julia Hippisleycox, Giampiero Mazzaglia, Gino Picelli, Giovanni CorraoAbstract:Purpose To provide estimates of the number and types of Drugs that can be monitored for Safety surveillance using electronic healthcare databases. Methods Using data from eight European databases (administrative claims, medical records) and in the context of a cohort study, we determined the amount of Drug exposure required for signal detection across varying magnitudes of relative risk (RR). We provide estimates of the number and types of Drugs that can be monitored as a function of actual use, minimal detectable RR, and empirically derived incidence rates for the following adverse events: (i) acute myocardial infarction; (ii) acute renal failure; (iii) anaphylactic shock; (iv) bullous eruptions; (v) rhabdomyolysis; and (vi) upper gastrointestinal bleeding. We performed data simulation to see how expansion of database size would influence the capabilities of such system. Results Data from 19647452 individuals (59594132person-years follow-up) who used 2289 Drugs in the EU-ADR network show that for a frequent event such as acute myocardial infarction, there are 531 Drugs (23% of total) for which an association with RR=2, if present, can be investigated.For a rareevent suchas rhabdomyolysis, thereare 19 Drugs (1%)for which an association ofsame magnitude canbe investigated. Conclusion Active surveillance using healthcare data-based networks for signal detection is feasible, although the leverage to do so may be low for infrequently used Drugs and for rare outcomes. Extending database network size to include data from heterogeneous populations and increasing follow-up time are warranted to maximize leverage of these surveillance systems. Copyright © 2012 John Wiley & Sons, Ltd. key words—active Drug Safety surveillance; Drug Safety monitoring; signal detection; electronic healthcare records; electronic healthcare databases; EU-ADR
Fiona Measham - One of the best experts on this subject based on the ideXlab platform.
-
city checking piloting the uk s first community based Drug Safety testing Drug checking service in 2 city centres
British Journal of Clinical Pharmacology, 2020Co-Authors: Fiona MeashamAbstract:Aims To explore the feasibility of delivering community-based Drug Safety testing (Drug checking), to trial service design characteristics and to compare with festival-based testing. Methods In total, 171 substances of concern were submitted on 5 dates at 3 venues in 2 UK cities and tested using up to 6 analytical techniques. Test results and harm reduction advice were distributed directly to over 200 service users through 144 tailored healthcare consultations, to stakeholders, and through early warning systems, media and social media alerts. Results The 171 samples were submitted and identified as MDMA (43.3%), cocaine (12.9%), ketamine (12.9%), various psychedelics submitted by students, and heroin and a synthetic cannabinoid submitted by rough sleeping communities, with 76% of samples' test results as expected. The 144 primary service users identified as 91.7% white, 68.1% male, with an average age of 26.7 years. Reported harm reduction intentions included alerting friends and acquaintances (37.5%), being more careful mixing that substance (35.4%), lowered dosage (27.8%), disposal of further substances (6.9%) and additionally 2.8% handed over further substances for verified destruction. Conclusion Community-based Drug Safety testing (Drug checking) was piloted for the first time in the UK-within a Drugs service, a community centre and a church-with consideration given to meso-level operational feasibility and micro-level behavioural outcomes. Service design characteristics such as venue, day of week, prior publicity, service provider, and direct and indirect dissemination of results all may impact on outcomes. Future studies should consider cost-benefit analyses of community and event-based testing and context-appropriate macro, meso and micro-level evaluations.
-
Drug Safety testing disposals and dealing in an english field exploring the operational and behavioural outcomes of the uk s first onsite Drug checking service
International Journal of Drug Policy, 2019Co-Authors: Fiona MeashamAbstract:Abstract Background In a year when UK Drug-related deaths and festival Drug-related deaths reached their highest on record, a pilot festival Drug Safety testing service was introduced with the aim of reducing Drug-related harm. This paper describes the operational and behavioural outcomes of this pilot and explores the relationship between Drug use, supply and policing within festival grounds. Methods Chemists in a temporary laboratory analysed 247 substances submitted by the public to a free, confidential testing service across four days at a UK festival in July 2016. Test results were returned to service users embedded in 230 healthcare consultations delivered to approximately 900 festival-goers (one in five Drug using festival-goers) that included harm reduction advice and the opportunity to use a disposal service for further substances of concern. Consultation data were collected at point of care, matched with test results, coded and analysed using SPSS Results Test results revealed that one in five substances was not as sold or acquired. One in five service users utilised the disposal service for further substances of concern in their possession and another one in six moderated their consumption. Two thirds of those whose sample was missold disposed of further substances, compared with under one in ten whose sample was as sold. Service users who acquired substances onsite at the festival were more than twice as likely to have been missold them as those acquired offsite, were nearly twice as likely to use the disposal service and were on average two years younger. Women were more likely to be using the Drug for the first time and more likely to use the disposal service. Test results were shared with emergency services; alerts issued across site and an unanticipated feedback loop occurred to some Drug suppliers. Conclusion This pilot suggests that festival-goers engage productively with onsite Drug Safety testing services when given the opportunity, such services can access harder-to-reach and new user groups and can play a part in reducing Drug-related harm by identifying and informing service users, emergency services and offsite Drug using communities about substances of concern. Disposals to the testing service for onward police destruction provide an externally corroborated measure of impact, reducing harm to the individual and others by removing such substances from site. Evidence of differential dealing onsite and its potential negative consequences has implications for future research and policing.
Nigam H Shah - One of the best experts on this subject based on the ideXlab platform.
-
a curated and standardized adverse Drug event resource to accelerate Drug Safety research
Scientific Data, 2016Co-Authors: Juan M Banda, Lee Evans, Rami Vanguri, Nicholas P Tatonetti, Patrick B Ryan, Nigam H ShahAbstract:Identification of adverse Drug reactions (ADRs) during the post-marketing phase is one of the most important goals of Drug Safety surveillance. Spontaneous reporting systems (SRS) data, which are the mainstay of traditional Drug Safety surveillance, are used for hypothesis generation and to validate the newer approaches. The publicly available US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) data requires substantial curation before they can be used appropriately, and applying different strategies for data cleaning and normalization can have material impact on analysis results. We provide a curated and standardized version of FAERS removing duplicate case records, applying standardized vocabularies with Drug names mapped to RxNorm concepts and outcomes mapped to SNOMED-CT concepts, and pre-computed summary statistics about Drug-outcome relationships for general consumption. This publicly available resource, along with the source code, will accelerate Drug Safety research by reducing the amount of time spent performing data management on the source FAERS reports, improving the quality of the underlying data, and enabling standardized analyses using common vocabularies.
-
a curated and standardized adverse Drug event resource to accelerate Drug Safety research
Scientific Data, 2016Co-Authors: Juan M Banda, Lee Evans, Rami Vanguri, Nicholas P Tatonetti, Patrick B Ryan, Nigam H ShahAbstract:Identification of adverse Drug reactions (ADRs) during the post-marketing phase is one of the most important goals of Drug Safety surveillance. Spontaneous reporting systems (SRS) data, which are the mainstay of traditional Drug Safety surveillance, are used for hypothesis generation and to validate the newer approaches. The publicly available US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) data requires substantial curation before they can be used appropriately, and applying different strategies for data cleaning and normalization can have material impact on analysis results. We provide a curated and standardized version of FAERS removing duplicate case records, applying standardized vocabularies with Drug names mapped to RxNorm concepts and outcomes mapped to SNOMED-CT concepts, and pre-computed summary statistics about Drug-outcome relationships for general consumption. This publicly available resource, along with the source code, will accelerate Drug Safety research by reducing the amount of time spent performing data management on the source FAERS reports, improving the quality of the underlying data, and enabling standardized analyses using common vocabularies.
-
annotation analysis for testing Drug Safety signals using unstructured clinical notes
Journal of Biomedical Semantics, 2012Co-Authors: Paea Lependu, Srinivasan Iyer, Cedrick Fairon, Nigam H ShahAbstract:The electronic surveillance for adverse Drug events is largely based upon the analysis of coded data from reporting systems. Yet, the vast majority of electronic health data lies embedded within the free text of clinical notes and is not gathered into centralized repositories. With the increasing access to large volumes of electronic medical data—in particular the clinical notes—it may be possible to computationally encode and to test Drug Safety signals in an active manner. We describe the application of simple annotation tools on clinical text and the mining of the resulting annotations to compute the risk of getting a myocardial infarction for patients with rheumatoid arthritis that take Vioxx. Our analysis clearly reveals elevated risks for myocardial infarction in rheumatoid arthritis patients taking Vioxx (odds ratio 2.06) before 2005. Our results show that it is possible to apply annotation analysis methods for testing hypotheses about Drug Safety using electronic medical records.
Andrew Bate - One of the best experts on this subject based on the ideXlab platform.
-
an evaluation of the thin database in the omop common data model for active Drug Safety surveillance
Drug Safety, 2013Co-Authors: Xiaofeng Zhou, Andrew Bate, Qing Liu, Sundaresan Murugesan, Harshvinder Bhullar, Bing Cai, Chuck WentworthAbstract:There has been increased interest in using multiple observational databases to understand the Safety profile of medical products during the postmarketing period. However, it is challenging to perform analyses across these heterogeneous data sources. The Observational Medical Outcome Partnership (OMOP) provides a Common Data Model (CDM) for organizing and standardizing databases. OMOP’s work with the CDM has primarily focused on US databases. As a participant in the OMOP Extended Consortium, we implemented the OMOP CDM on the UK Electronic Healthcare Record database—The Health Improvement Network (THIN). The aim of the study was to evaluate the implementation of the THIN database in the OMOP CDM and explore its use for active Drug Safety surveillance. Following the OMOP CDM specification, the raw THIN database was mapped into a CDM THIN database. Ten Drugs of Interest (DOI) and nine Health Outcomes of Interest (HOI), defined and focused by the OMOP, were created using the CDM THIN database. Quantitative comparison of raw THIN to CDM THIN was performed by execution and analysis of OMOP standardized reports and additional analyses. The practical value of CDM THIN for Drug Safety and pharmacoepidemiological research was assessed by implementing three analysis methods: Proportional Reporting Ratio (PRR), Univariate Self-Case Control Series (USCCS) and High-Dimensional Propensity Score (HDPS). A published study using raw THIN data was selected to examine the external validity of CDM THIN. Overall demographic characteristics were the same in both databases. Mapping medical and Drug codes into the OMOP terminology dictionary was incomplete: 25 % medical codes and 55 % Drug codes in raw THIN were not listed in the OMOP terminology dictionary, representing 6 % condition occurrence counts, 4 % procedure occurrence counts and 7 % Drug exposure counts in raw THIN. Seven DOIs had <0.3 % and three DOIs had 1 % of unmapped Drug exposure counts; each HOI had at least one definition with no or minimal (≤0.2 %) issues with unmapped condition occurrence counts, except for the upper gastrointestinal (UGI) ulcer hospitalization cohort. The application of PRR, USCCS and HDPS found, respectively, a sensitivity of 67, 78 and 50 %, and a specificity of 68, 59 and 76 %, suggesting that Safety issues defined as known by the OMOP could be identified in CDM THIN, with imperfect performance. Similar PRR scores were produced using both CDM THIN and raw THIN, while the execution time was twice as fast on CDM THIN. There was close replication of demographic distribution, death rate and prescription pattern and trend in the published study population and the cohort of CDM THIN. This research demonstrated that information loss due to incomplete mapping of medical and Drug codes as well as data structure in the current CDM THIN limits its use for all possible epidemiological evaluation studies. Current HOIs and DOIs predefined by the OMOP were constructed with minimal loss of information and can be used for active surveillance methodological research. The OMOP CDM THIN can be a valuable tool for multiple aspects of pharmacoepidemiological research when the unique features of UK Electronic Health Records are incorporated in the OMOP library.
-
large scale regression based pattern discovery the example of screening the who global Drug Safety database
Statistical Analysis and Data Mining, 2010Co-Authors: Ola Caster, Niklas G Noren, David Madigan, Andrew BateAbstract:Most measures of interestingness for patterns of co-occurring events are based on data projections onto contingency tables for the events of primary interest. As an alternative, this article presents the first implementation of shrinkage logistic regression for large-scale pattern discovery, with an evaluation of its usefulness in real-world binary transaction data. Regression accounts for the impact of other covariates that may confound or otherwise distort associations. The application considered is international adverse Drug reaction (ADR) surveillance, in which large collections of reports on suspected ADRs are screened for interesting reporting patterns worthy of clinical follow-up. Our results show that regression-based pattern discovery does offer practical advantages. Specifically it can eliminate false positives and false negatives due to other covariates. Furthermore, it identifies some established Drug Safety issues earlier than a measure based on contingency tables. While regression offers clear conceptual advantages, our results suggest that methods based on contingency tables will continue to play a key role in ADR surveillance, for two reasons: the failure of regression to identify some established Drug Safety concerns as early as the currently used measures, and the relative lack of transparency of the procedure to estimate the regression coefficients. This suggests shrinkage regression should be used in parallel to existing measures of interestingness in ADR surveillance and other large-scale pattern discovery applications. Copyright © 2010 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 3: 197-208, 2010
-
extending the methods used to screen the who Drug Safety database towards analysis of complex associations and improved accuracy for rare events
Statistics in Medicine, 2006Co-Authors: Niklas G Noren, Roland Orre, Andrew Bate, Ralph I EdwardsAbstract:Extending the methods used to screen the WHO Drug Safety database towards analysis of complex associations and improved accuracy for rare events
-
extending the methods used to screen the who Drug Safety database towards analysis of complex associations and improved accuracy for rare events
Statistics in Medicine, 2006Co-Authors: Niklas G Noren, Roland Orre, Andrew Bate, Ralph I EdwardsAbstract:Post-marketing Drug Safety data sets are often massive, and entail problems with heterogeneity and selection bias. Nevertheless, quantitative methods have proven a very useful aid to help clinical experts in screening for previously unknown associations in these data sets. The WHO international Drug Safety database is the world's largest data set of its kind with over three million reports on suspected adverse Drug reaction incidents. Since 1998, an exploratory data analysis method has been in routine use to screen for quantitative associations in this data set. This method was originally based on large sample approximations and limited to pairwise associations, but in this article we propose more accurate credibility interval estimates and extend the method to allow for the analysis of more complex quantitative associations. The accuracy of the proposed credibility intervals is evaluated through comparison to precise Monte Carlo simulations. In addition, we propose a Mantel-Haenszel-type adjustment to control for suspected confounders.
-
a hit miss model for duplicate detection in the who Drug Safety database
Knowledge Discovery and Data Mining, 2005Co-Authors: Niklas G Noren, Roland Orre, Andrew BateAbstract:The WHO Collaborating Centre for International Drug Monitoring in Uppsala, Sweden, maintains and analyses the world's largest database of reports on suspected adverse Drug reaction incidents that occur after Drugs are introduced on the market. As in other post-marketing Drug Safety data sets, the presence of duplicate records is an important data quality problem and the detection of duplicates in the WHO Drug Safety database remains a formidable challenge, especially since the reports are anonymised before submitted to the database. However, to our knowledge no work has been published on methods for duplicate detection in post-marketing Drug Safety data. In this paper, we propose a method for probabilistic duplicate detection based on the hit-miss model for statistical record linkage described by Copas & Hilton. We present two new generalisations of the standard hit-miss model: a hit-miss mixture model for errors in numerical record fields and a new method to handle correlated record fields. We demonstrate the effectiveness of the hit-miss model for duplicate detection in the WHO Drug Safety database both at identifying the most likely duplicate for a given record (94.7% accuracy) and at discriminating duplicates from random matches (63% recall with 71% precision). The proposed method allows for more efficient data cleaning in post-marketing Drug Safety data sets, and perhaps other applications throughout the KDD community.
Patrick B Ryan - One of the best experts on this subject based on the ideXlab platform.
-
a curated and standardized adverse Drug event resource to accelerate Drug Safety research
Scientific Data, 2016Co-Authors: Juan M Banda, Lee Evans, Rami Vanguri, Nicholas P Tatonetti, Patrick B Ryan, Nigam H ShahAbstract:Identification of adverse Drug reactions (ADRs) during the post-marketing phase is one of the most important goals of Drug Safety surveillance. Spontaneous reporting systems (SRS) data, which are the mainstay of traditional Drug Safety surveillance, are used for hypothesis generation and to validate the newer approaches. The publicly available US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) data requires substantial curation before they can be used appropriately, and applying different strategies for data cleaning and normalization can have material impact on analysis results. We provide a curated and standardized version of FAERS removing duplicate case records, applying standardized vocabularies with Drug names mapped to RxNorm concepts and outcomes mapped to SNOMED-CT concepts, and pre-computed summary statistics about Drug-outcome relationships for general consumption. This publicly available resource, along with the source code, will accelerate Drug Safety research by reducing the amount of time spent performing data management on the source FAERS reports, improving the quality of the underlying data, and enabling standardized analyses using common vocabularies.
-
a curated and standardized adverse Drug event resource to accelerate Drug Safety research
Scientific Data, 2016Co-Authors: Juan M Banda, Lee Evans, Rami Vanguri, Nicholas P Tatonetti, Patrick B Ryan, Nigam H ShahAbstract:Identification of adverse Drug reactions (ADRs) during the post-marketing phase is one of the most important goals of Drug Safety surveillance. Spontaneous reporting systems (SRS) data, which are the mainstay of traditional Drug Safety surveillance, are used for hypothesis generation and to validate the newer approaches. The publicly available US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) data requires substantial curation before they can be used appropriately, and applying different strategies for data cleaning and normalization can have material impact on analysis results. We provide a curated and standardized version of FAERS removing duplicate case records, applying standardized vocabularies with Drug names mapped to RxNorm concepts and outcomes mapped to SNOMED-CT concepts, and pre-computed summary statistics about Drug-outcome relationships for general consumption. This publicly available resource, along with the source code, will accelerate Drug Safety research by reducing the amount of time spent performing data management on the source FAERS reports, improving the quality of the underlying data, and enabling standardized analyses using common vocabularies.
-
defining a reference set to support methodological research in Drug Safety
Drug Safety, 2013Co-Authors: Mj Schuemie, Patrick B Ryan, Emily Welebob, Jon D Duke, Sarah Valentine, Abraham G HartzemaAbstract:Background Methodological research to evaluate the performance of methods requires a benchmark to serve as a referent comparison. In Drug Safety, the performance of analyses of spontaneous adverse event reporting databases and observational healthcare data, such as administrative claims and electronic health records, has been limited by the lack of such standards.
-
defining a reference set to support methodological research in Drug Safety
Drug Safety, 2013Co-Authors: Mj Schuemie, Patrick B Ryan, Emily Welebob, Sarah Valentine, Abraham G Hartzema, Jon DukeAbstract:Methodological research to evaluate the performance of methods requires a benchmark to serve as a referent comparison. In Drug Safety, the performance of analyses of spontaneous adverse event reporting databases and observational healthcare data, such as administrative claims and electronic health records, has been limited by the lack of such standards. To establish a reference set of test cases that contain both positive and negative controls, which can serve the basis for methodological research in evaluating methods performance in identifying Drug Safety issues. Systematic literature review and natural language processing of structured product labeling was performed to identify evidence to support the classification of Drugs as either positive controls or negative controls for four outcomes: acute liver injury, acute kidney injury, acute myocardial infarction, and upper gastrointestinal bleeding. Three-hundred and ninety-nine test cases comprised of 165 positive controls and 234 negative controls were identified across the four outcomes. The majority of positive controls for acute kidney injury and upper gastrointestinal bleeding were supported by randomized clinical trial evidence, while the majority of positive controls for acute liver injury and acute myocardial infarction were only supported based on published case reports. Literature estimates for the positive controls shows substantial variability that limits the ability to establish a reference set with known effect sizes. A reference set of test cases can be established to facilitate methodological research in Drug Safety. Creating a sufficient sample of Drug-outcome pairs with binary classification of having no effect (negative controls) or having an increased effect (positive controls) is possible and can enable estimation of predictive accuracy through discrimination. Since the magnitude of the positive effects cannot be reliably obtained and the quality of evidence may vary across outcomes, assumptions are required to use the test cases in real data for purposes of measuring bias, mean squared error, or coverage probability.
