Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers' Critiques.

Item request has been placed! ×
Item request cannot be made. ×
loading   Processing Request
  • Additional Information
    • Subject Terms:
    • Author-Supplied Keywords:
      gender differences
      NIH funding
      women's career advancement
    • Subject Terms:
    • NAICS/Industry Codes:
      813219 Other Grantmaking and Giving Services
      611310 Colleges, Universities, and Professional Schools
    • Abstract:
      Background: Women are less successful than men in renewing R01 grants from the National Institutes of Health. Continuing to probe text mining as a tool to identify gender bias in peer review, we used algorithmic text mining and qualitative analysis to examine a sample of critiques from men's and women's R01 renewal applications previously analyzed by counting and comparing word categories. Methods: We analyzed 241 critiques from 79 Summary Statements for 51 R01 renewals awarded to 45 investigators (64% male, 89% white, 80% PhD) at the University of Wisconsin-Madison between 2010 and 2014. We used latent Dirichlet allocation to discover evaluative 'topics' ( i.e., words that co-occur with high probability). We then qualitatively examined the context in which evaluative words occurred for male and female investigators. We also examined sex differences in assigned scores controlling for investigator productivity. Results: Text analysis results showed that male investigators were described as 'leaders' and 'pioneers' in their 'fields,' with 'highly innovative' and 'highly significant research.' By comparison, female investigators were characterized as having 'expertise' and working in 'excellent' environments. Applications from men received significantly better priority, approach, and significance scores, which could not be accounted for by differences in productivity. Conclusions: Results confirm our previous analyses suggesting that gender stereotypes operate in R01 grant peer review. Reviewers may more easily view male than female investigators as scientific leaders with significant and innovative research, and score their applications more competitively. Such implicit bias may contribute to sex differences in award rates for R01 renewals. [ABSTRACT FROM AUTHOR]
    • Abstract:
      Copyright of Journal of Women's Health (15409996) is the property of Mary Ann Liebert, Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
    • Author Affiliations:
      1Department of Population Health Sciences, University of Wisconsin-Madison, Madison, Wisconsin.
      2Department of Computer Science, University of Wisconsin-Madison, Madison, Wisconsin.
      3Center for Women's Health Research, University of Wisconsin-Madison, Madison, Wisconsin.
      4Department of Statistics, University of Wisconsin-Madison, Madison, Wisconsin.
      5Health Innovation Program, University of Wisconsin-Madison, Madison, Wisconsin.
      6Wisconsin Center for Education Research, University of Wisconsin-Madison, Madison, Wisconsin.
      7Departments of Medicine, Psychiatry, and Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin.
      8William S. Middleton Veterans Hospital, Madison, Wisconsin.
    • ISSN:
      1540-9996
    • Accession Number:
      10.1089/jwh.2016.6021
    • Accession Number:
      123086777
  • Citations
    • ABNT:
      MAGUA, W. et al. Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers’ Critiques. Journal of Women’s Health (15409996), [s. l.], v. 26, n. 5, p. 560–570, 2017. DOI 10.1089/jwh.2016.6021. Disponível em: http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=a9h&AN=123086777&custid=s6224580. Acesso em: 2 dez. 2020.
    • AMA:
      Magua W, Zhu X, Bhattacharya A, et al. Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers’ Critiques. Journal of Women’s Health (15409996). 2017;26(5):560-570. doi:10.1089/jwh.2016.6021
    • APA:
      Magua, W., Zhu, X., Bhattacharya, A., Filut, A., Potvien, A., Leatherberry, R., Lee, Y.-G., Jens, M., Malikireddy, D., Carnes, M., & Kaatz, A. (2017). Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers’ Critiques. Journal of Women’s Health (15409996), 26(5), 560–570. https://doi.org/10.1089/jwh.2016.6021
    • Chicago/Turabian: Author-Date:
      Magua, Wairimu, Xiaojin Zhu, Anupama Bhattacharya, Amarette Filut, Aaron Potvien, Renee Leatherberry, You-Geon Lee, et al. 2017. “Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers’ Critiques.” Journal of Women’s Health (15409996) 26 (5): 560–70. doi:10.1089/jwh.2016.6021.
    • Harvard:
      Magua, W. et al. (2017) ‘Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers’ Critiques’, Journal of Women’s Health (15409996), 26(5), pp. 560–570. doi: 10.1089/jwh.2016.6021.
    • Harvard: Australian:
      Magua, W, Zhu, X, Bhattacharya, A, Filut, A, Potvien, A, Leatherberry, R, Lee, Y-G, Jens, M, Malikireddy, D, Carnes, M & Kaatz, A 2017, ‘Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers’ Critiques’, Journal of Women’s Health (15409996), vol. 26, no. 5, pp. 560–570, viewed 2 December 2020, .
    • MLA:
      Magua, Wairimu, et al. “Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers’ Critiques.” Journal of Women’s Health (15409996), vol. 26, no. 5, May 2017, pp. 560–570. EBSCOhost, doi:10.1089/jwh.2016.6021.
    • Chicago/Turabian: Humanities:
      Magua, Wairimu, Xiaojin Zhu, Anupama Bhattacharya, Amarette Filut, Aaron Potvien, Renee Leatherberry, You-Geon Lee, et al. “Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers’ Critiques.” Journal of Women’s Health (15409996) 26, no. 5 (May 2017): 560–70. doi:10.1089/jwh.2016.6021.
    • Vancouver/ICMJE:
      Magua W, Zhu X, Bhattacharya A, Filut A, Potvien A, Leatherberry R, et al. Are Female Applicants Disadvantaged in National Institutes of Health Peer Review? Combining Algorithmic Text Mining and Qualitative Methods to Detect Evaluative Differences in R01 Reviewers’ Critiques. Journal of Women’s Health (15409996) [Internet]. 2017 May [cited 2020 Dec 2];26(5):560–70. Available from: http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=a9h&AN=123086777&custid=s6224580