Societal Impact

14,000,000 Leading Edge Experts on the ideXlab platform

Scan Science and Technology

Contact Leading Edge Experts & Companies

Scan Science and Technology

Contact Leading Edge Experts & Companies

The Experts below are selected from a list of 62316 Experts worldwide ranked by ideXlab platform

Lutz Bornmann - One of the best experts on this subject based on the ideXlab platform.

  • can altmetrics reflect Societal Impact considerations exploring the potential of altmetrics in the context of a sustainability science research center
    2020
    Co-Authors: Omar Kassab, Lutz Bornmann, Robin Haunschild
    Abstract:

    Societal Impact considerations play an increasingly important role in research evaluation. In particular, in the context of publicly funded research, proposal templates commonly include sections to...

  • altmetrics and Societal Impact measurements match or mismatch a literature review
    2020
    Co-Authors: Iman Tahamtan, Lutz Bornmann
    Abstract:

    Can alternative metrics (altmetrics) data be used to measure Societal Impact? We wrote this literature overview of empirical studies in order to find an answer to this question. The overview includes two parts. The first part, “Societal Impact measurements”, explains possible methods and problems in measuring the Societal Impact of research, case studies for Societal Impact measurement, Societal Impact considerations at funding organizations, and the Societal problems that should be solved by science. The second part of the review, “altmetrics”, addresses a major question in research evaluation, which is whether altmetrics are proper indicators for measuring the Societal Impact of research. In the second part we explain the data sources used for altmetrics studies and the importance of field-normalized indicators for Impact measurements. This review indicates that it should be relevant for Impact measurements to be oriented towards pressing Societal problems. Case studies in which Societal Impact of certain pieces of research is explained seem to provide a legitimate method for measuring Societal Impact. In the use of altmetrics, field-specific differences should be considered by applying field normalization (in cross-field comparisons). Altmetrics data such as social media counts might mainly reflect the public interest and discussion of scholarly works rather than their Societal Impact. Altmetrics (Twitter data) might be especially fruitfully employed for research evaluation purposes, if they are used in the context of network approaches. Conclusions based on altmetrics data in research evaluation should be drawn with caution.

  • do altmetrics assess Societal Impact in a comparable way to case studies an empirical test of the convergent validity of altmetrics based on data from the uk research excellence framework ref
    2019
    Co-Authors: Lutz Bornmann, Robin Haunschild, Jonathan Adams
    Abstract:

    Abstract Altmetrics have been proposed as a way to assess the Societal Impact of research. Although altmetrics are already in use as Impact or attention metrics in different contexts, it is still not clear whether they really capture or reflect Societal Impact. This study is based on altmetrics, citation counts, research output and case study data from the UK Research Excellence Framework (REF), and peers’ REF assessments of research output and Societal Impact. We investigated the convergent validity of altmetrics by using two REF datasets: publications submitted as research output (PRO) to the REF and publications referenced in case studies (PCS). Case studies, which are intended to demonstrate Societal Impact, should cite the most relevant research papers. We used the MHq’ indicator for assessing Impact – an indicator which has been introduced for count data with many zeros. The results of the first part of the analysis show that news media as well as mentions on Facebook, in blogs, in Wikipedia, and in policy-related documents have higher MHq’ values for PCS than for PRO. Thus, the altmetric indicators seem to have convergent validity for these data. In the second part of the analysis, altmetrics have been correlated with REF reviewers’ average scores on PCS. The negative or close to zero correlations question the convergent validity of altmetrics in that context. We suggest that they may capture a different aspect of Societal Impact (which can be called unknown attention) to that seen by reviewers (who are interested in the causal link between research and action in society).

  • Societal Impact measurement of research papers
    2019
    Co-Authors: Lutz Bornmann, Robin Haunschild
    Abstract:

    What are the results of public investment in research from which society actually derives a benefit? The scope of research evaluations becomes broader when Societal products (outputs), Societal use (Societal references), and Societal benefits (changes in society) of research are considered. This chapter presents an overview of the literature in the area of Societal Impact measurement of scientific papers. It describes major research projects on Societal Impact measurements. Problems of Societal Impact assessments are discussed as well as proposals to measure Societal Impact. The chapter discusses the role of alternative metrics (altmetrics) in measuring Societal Impact. There is an ongoing debate in scientometrics as to whether altmetrics are able to measure this kind of Impact.

  • does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards Societal Impact
    2017
    Co-Authors: Lutz Bornmann, Robin Haunschild
    Abstract:

    When the meaning of key terms is incompatible in competing taxonomies, a revolution might occur in the field by which the established taxonomy is replaced with another. Since the key term "Impact" in scientometrics seems to undergo a taxonomic change, a revolution might be taking place at present: Impact is no longer defined as Impact on science alone (measured by citations), but on all sectors of society (e.g. economics, culture, or politics). In this Short Communication, we outline that the current revolution in scientometrics does not only imply a broadening of the Impact perspective, but also the devaluation of quality considerations in evaluative contexts. Impact might no longer be seen as a proxy for quality, but in its original sense: the simple resonance in some sectors of society.

Werner Marx - One of the best experts on this subject based on the ideXlab platform.

  • Policy documents as sources for measuring Societal Impact: how often is climate change research mentioned in policy-related documents?
    2016
    Co-Authors: Lutz Bornmann, Robin Haunschild, Werner Marx
    Abstract:

    In the current UK Research Excellence Framework (REF) and the Excellence in Research for Australia (ERA), Societal Impact measurements are inherent parts of the national evaluation systems. In this study, we deal with a relatively new form of Societal Impact measurements. Recently, Altmetric—a start-up providing publication level metrics—started to make data for publications available which have been mentioned in policy documents. We regard this data source as an interesting possibility to specifically measure the (Societal) Impact of research. Using a comprehensive dataset with publications on climate change as an example, we study the usefulness of the new data source for Impact measurement. Only 1.2 % ( n  = 2341) out of 191,276 publications on climate change in the dataset have at least one policy mention. We further reveal that papers published in Nature and Science as well as from the areas “Earth and related environmental sciences” and “Social and economic geography” are especially relevant in the policy context. Given the low coverage of the climate change literature in policy documents, this study can be only a first attempt to study this new source of altmetrics data. Further empirical studies are necessary, because mentions in policy documents are of special interest in the use of altmetrics data for measuring target-oriented the broader Impact of research.

  • policy documents as sources for measuring Societal Impact how often is climate change research mentioned in policy related documents
    2015
    Co-Authors: Lutz Bornmann, Robin Haunschild, Werner Marx
    Abstract:

    In the current UK Research Excellence Framework (REF) and the Excellence in Research for Australia (ERA) Societal Impact measurements are inherent parts of the national evaluation systems. In this study, we deal with a relatively new form of Societal Impact measurements. Recently, Altmetric - a start-up providing publication level metrics - started to make data for publications available which have been mentioned in policy documents. We regard this data source as an interesting possibility to specifically measure the (Societal) Impact of research. Using a comprehensive dataset with publications on climate change as an example, we study the usefulness of the new data source for Impact measurement. Only 1.2% (n=2,341) out of 191,276 publications on climate change in the dataset have at least one policy mention. We further reveal that papers published in Nature and Science as well as from the areas "Earth and related environmental sciences" and "Social and economic geography" are especially relevant in the policy context. Given the low coverage of the climate change literature in policy documents, this study can be only a first attempt to study this new source of altmetric data. Further empirical studies are necessary in upcoming years, because mentions in policy documents are of special interest in the use of altmetric data for measuring target-oriented the broader Impact of research.

  • how should the Societal Impact of research be generated and measured a proposal for a simple and practicable approach to allow interdisciplinary comparisons
    2014
    Co-Authors: Lutz Bornmann, Werner Marx
    Abstract:

    Since the 1990s, the scope of research evaluation has widened to encompass the Societal products (outputs), Societal use (Societal references) and Societal benefits (changes in society) of research. Research evaluation has been extended to include measures of the (1) social, (2) cultural, (3) environmental and (4) economic returns from publicly funded research. Even though no robust or reliable methods for measuring Societal Impact have yet been developed. In this study, we would like to introduce an approach which, unlike the currently common case study approach (and others), is relatively simple, can be used in almost every subject area and delivers results regarding Societal Impact which can be compared between disciplines. Our approach to Societal Impact starts with the actual function of science in society: to generate reliable knowledge. That is why a study (which we would like to refer to as an assessment report) summarising the status of the research on a certain subject represents knowledge which is available for society to access. Societal Impact is given when the content of a report is addressed outside of science (in a government document, for example).

Robin Haunschild - One of the best experts on this subject based on the ideXlab platform.

  • can altmetrics reflect Societal Impact considerations exploring the potential of altmetrics in the context of a sustainability science research center
    2020
    Co-Authors: Omar Kassab, Lutz Bornmann, Robin Haunschild
    Abstract:

    Societal Impact considerations play an increasingly important role in research evaluation. In particular, in the context of publicly funded research, proposal templates commonly include sections to...

  • do altmetrics assess Societal Impact in a comparable way to case studies an empirical test of the convergent validity of altmetrics based on data from the uk research excellence framework ref
    2019
    Co-Authors: Lutz Bornmann, Robin Haunschild, Jonathan Adams
    Abstract:

    Abstract Altmetrics have been proposed as a way to assess the Societal Impact of research. Although altmetrics are already in use as Impact or attention metrics in different contexts, it is still not clear whether they really capture or reflect Societal Impact. This study is based on altmetrics, citation counts, research output and case study data from the UK Research Excellence Framework (REF), and peers’ REF assessments of research output and Societal Impact. We investigated the convergent validity of altmetrics by using two REF datasets: publications submitted as research output (PRO) to the REF and publications referenced in case studies (PCS). Case studies, which are intended to demonstrate Societal Impact, should cite the most relevant research papers. We used the MHq’ indicator for assessing Impact – an indicator which has been introduced for count data with many zeros. The results of the first part of the analysis show that news media as well as mentions on Facebook, in blogs, in Wikipedia, and in policy-related documents have higher MHq’ values for PCS than for PRO. Thus, the altmetric indicators seem to have convergent validity for these data. In the second part of the analysis, altmetrics have been correlated with REF reviewers’ average scores on PCS. The negative or close to zero correlations question the convergent validity of altmetrics in that context. We suggest that they may capture a different aspect of Societal Impact (which can be called unknown attention) to that seen by reviewers (who are interested in the causal link between research and action in society).

  • Societal Impact measurement of research papers
    2019
    Co-Authors: Lutz Bornmann, Robin Haunschild
    Abstract:

    What are the results of public investment in research from which society actually derives a benefit? The scope of research evaluations becomes broader when Societal products (outputs), Societal use (Societal references), and Societal benefits (changes in society) of research are considered. This chapter presents an overview of the literature in the area of Societal Impact measurement of scientific papers. It describes major research projects on Societal Impact measurements. Problems of Societal Impact assessments are discussed as well as proposals to measure Societal Impact. The chapter discusses the role of alternative metrics (altmetrics) in measuring Societal Impact. There is an ongoing debate in scientometrics as to whether altmetrics are able to measure this kind of Impact.

  • does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards Societal Impact
    2017
    Co-Authors: Lutz Bornmann, Robin Haunschild
    Abstract:

    When the meaning of key terms is incompatible in competing taxonomies, a revolution might occur in the field by which the established taxonomy is replaced with another. Since the key term "Impact" in scientometrics seems to undergo a taxonomic change, a revolution might be taking place at present: Impact is no longer defined as Impact on science alone (measured by citations), but on all sectors of society (e.g. economics, culture, or politics). In this Short Communication, we outline that the current revolution in scientometrics does not only imply a broadening of the Impact perspective, but also the devaluation of quality considerations in evaluative contexts. Impact might no longer be seen as a proxy for quality, but in its original sense: the simple resonance in some sectors of society.

  • Policy documents as sources for measuring Societal Impact: how often is climate change research mentioned in policy-related documents?
    2016
    Co-Authors: Lutz Bornmann, Robin Haunschild, Werner Marx
    Abstract:

    In the current UK Research Excellence Framework (REF) and the Excellence in Research for Australia (ERA), Societal Impact measurements are inherent parts of the national evaluation systems. In this study, we deal with a relatively new form of Societal Impact measurements. Recently, Altmetric—a start-up providing publication level metrics—started to make data for publications available which have been mentioned in policy documents. We regard this data source as an interesting possibility to specifically measure the (Societal) Impact of research. Using a comprehensive dataset with publications on climate change as an example, we study the usefulness of the new data source for Impact measurement. Only 1.2 % ( n  = 2341) out of 191,276 publications on climate change in the dataset have at least one policy mention. We further reveal that papers published in Nature and Science as well as from the areas “Earth and related environmental sciences” and “Social and economic geography” are especially relevant in the policy context. Given the low coverage of the climate change literature in policy documents, this study can be only a first attempt to study this new source of altmetrics data. Further empirical studies are necessary, because mentions in policy documents are of special interest in the use of altmetrics data for measuring target-oriented the broader Impact of research.

Ismael Rafols - One of the best experts on this subject based on the ideXlab platform.

  • Using altmetrics for contextualised mapping of Societal Impact: from hits to networks
    2018
    Co-Authors: Nicolás Robinson-garcía, Thed N. Van Leeuwen, Ismael Rafols
    Abstract:

    In this article, we develop a method that uses altmetric data to analyse researchers’ interactions, as a way of mapping the contexts of potential Societal Impact. In the face of an increasing policy demand for quantitative methodologies to assess Societal Impact, social media data (altmetrics) has been presented as a potential method to capture broader forms of Impact. However, current altmetric indicators were extrapolated from traditional citation approaches and are seen as problematic for assessing Societal Impact. In contrast, established qualitative methodologies for Societal Impact assessment are based on interaction approaches. These argue that assessment should focus on mapping the contexts in which engagement among researchers and stakeholders takes place, as a means to understand the pathways to Societal Impact. Following these interaction approaches, we propose to shift the use of altmetric data towards network analysis of researchers and stakeholders. We carry out two case studies, analysing researchers’ networks with Twitter data. The comparison illustrates the potential of Twitter networks to capture disparate degrees of policy engagement. We propose that this mapping method can be used as an input within broader methodologies in case studies of Societal Impact assessment.

  • Using almetrics for contextualised mapping of Societal Impact: From hits to networks
    2018
    Co-Authors: Nicolás Robinson-garcía, Thed N. Van Leeuwen, Ismael Rafols
    Abstract:

    In this article, we develop a method that uses altmetric data to analyse researchers' interactions, as a way of mapping the contexts of potential Societal Impact. In the face of an increasing policy demand for quantitative methodologies to assess Societal Impact, social media data (altmetrics) has been presented as a potential method to capture broader forms of Impact. However, current altmetric indicators were extrapolated from traditional citation approaches and are seen as problematic for assessing Societal Impact. In contrast, established qualitative methodologies for Societal Impact assessment are based on interaction approaches. These argue that assessment should focus on mapping the contexts in which engagement among researchers and stakeholders take place, as a means to understand the pathways to Societal Impact. Following these approaches, we propose to shift the use of altmetric data towards network analysis of researchers and stakeholders. We carry out two case studies, analysing researchers' networks with Twitter data. The comparison illustrates the potential of Twitter networks to capture disparate degrees of policy engagement. We propose that this mapping method can be used as an input within broader methodologies in case studies of Societal Impact assessment.

  • using almetrics for contextualised mapping of Societal Impact from hits to networks
    2017
    Co-Authors: Ismael Rafols, Nicolas Robinsongarcia, Thed N Van Leeuwen
    Abstract:

    In this article, we develop a method that uses altmetric data to analyse researchers’ interactions, as a way of mapping the contexts of potential Societal Impact. In the face of an increasing policy demand for quantitative methodologies to assess Societal Impact, social media data (altmetrics) has been presented as a potential method to capture broader forms of Impact. However, current altmetric indicators were extrapolated from traditional citation approaches and are seen as problematic for assessing Societal Impact. In contrast, established qualitative methodologies for Societal Impact assessment are based on interaction approaches. These argue that assessment should focus on mapping the contexts in which engagement among researchers and stakeholders takes place, as a means to understand the pathways to Societal Impact. Following these interaction approaches, we propose to shift the use of altmetric data towards network analysis of researchers and stakeholders. We carry out two case studies, analysing researchers’ networks with Twitter data. The comparison illustrates the potential of Twitter networks to capture disparate degrees of policy engagement. We propose that this mapping method can be used as an input within broader methodologies in case studies of Societal Impact assessment.

Leona Hakkaartvan Roijen - One of the best experts on this subject based on the ideXlab platform.

  • decreased quality of life and Societal Impact of cryopyrin associated periodic syndrome treated with canakinumab a questionnaire based cohort study
    2018
    Co-Authors: Catharina M Muldersmanders, Tim A Kanters, Paul L A Van Daele, Esther P A H Hoppenreijs, Elizabeth G Legger, Jan A M Van Laar, Anna Simon, Leona Hakkaartvan Roijen
    Abstract:

    Background: Cryopyrin-associated periodic syndrome (CAPS) is a rare disease. Knowledge on the quality of life (QoL) and the disease's Societal Impact is limited. Canakinumab is used in increasing frequency for the treatment of CAPS. Methods: Observational study in Dutch CAPS patients. Patients completed questionnaires regarding treatment with canakinumab at baseline and retrospectively. Quality of life was assessed using the EQ-5D-5L in adults and CHQ-PF50 in children. Impact on work and school was assessed. Caregivers' quality of life was assessed using the CarerQol. Results: Mean quality of life scores during treatment with canakinumab were 0.769 (EQ-5D-5L), 51.1 (CHQ-P) and 57-1 (CHQ-M). Most patients experienced problems on the pain/discomfort dimension. Higher disease activity and the presence of complications negatively influenced QoL. Half of the patients with a paid job reported absenteeism from work due to CAPS, for an average of 8.7 days in a 4-week period. All schoolgoing patients (N = 5) reported absence from school due to CAPS, for an average of 2.9 days. Caregivers reported gaining a lot fulfillment from providing care for their family members. Conclusion: QoL during treatment is lower than in the general Dutch population. CAPS leads to productivity loss and absenteeism from school, and Impacts the quality of life in informal caregivers.

  • decreased quality of life and Societal Impact of cryopyrin associated periodic syndrome treated with canakinumab a questionnaire based cohort study
    2018
    Co-Authors: Catharina M Muldersmanders, Tim A Kanters, Paul L A Van Daele, Esther P A H Hoppenreijs, Elizabeth G Legger, Jan A M Van Laar, Anna Simon, Leona Hakkaartvan Roijen
    Abstract:

    Cryopyrin-associated periodic syndrome (CAPS) is a rare disease. Knowledge on the quality of life (QoL) and the disease’s Societal Impact is limited. Canakinumab is used in increasing frequency for the treatment of CAPS. Observational study in Dutch CAPS patients. Patients completed questionnaires regarding treatment with canakinumab at baseline and retrospectively. Quality of life was assessed using the EQ-5D-5L in adults and CHQ-PF50 in children. Impact on work and school was assessed. Caregivers' quality of life was assessed using the CarerQol. Mean quality of life scores during treatment with canakinumab were 0.769 (EQ-5D-5L), 51.1 (CHQ-P) and 57–1 (CHQ-M). Most patients experienced problems on the pain/discomfort dimension. Higher disease activity and the presence of complications negatively influenced QoL. Half of the patients with a paid job reported absenteeism from work due to CAPS, for an average of 8.7 days in a 4-week period. All schoolgoing patients (N = 5) reported absence from school due to CAPS, for an average of 2.9 days. Caregivers reported gaining a lot fulfillment from providing care for their family members. QoL during treatment is lower than in the general Dutch population. CAPS leads to productivity loss and absenteeism from school, and Impacts the quality of life in informal caregivers.