Restricted access

Research article

First published online March 1, 2011

Request permissions

A Systematic Review of the Empirical Literature Evaluating IRBs: What We Know and What We Still Need to Learn

Lura Abbott and Christine Grady [email protected]View all authors and affiliations

Volume 6, Issue 1

https://doi.org/10.1525/jer.2011.6.1.3

Abstract
References
Biographies

Abstract

Institutional review boards (IRBs) are integral to the U.S. system of protection of human research participants. Evaluation of IRBs, although difficult, is essential. To date, no systematic review of IRB studies has been published. We conducted a systematic review of empirical studies of U.S. IRBs to determine what is known about the function of IRBs and to identify gaps in knowledge. A structured search in PubMed identified forty-three empirical studies evaluating U.S. IRBs. Studies were included if they reported an empirical investigation of the structure, process, outcomes, effectiveness, or variation of U.S. IRBs. The authors reviewed each study to extract information about study objectives, sample and methods, study results, and conclusions. Empirical evidence collected in forty-three published studies shows that for review of a wide range of types of research, U.S. IRBs differ in their application of the federal regulations, in the time they take to review studies, and in the decisions made. Existing studies show evidence of variation in multicenter review, inconsistent or ambiguous interpretation of the federal regulations, and inefficiencies in review. Despite recognition of a need to evaluate effectiveness of IRB review, no identified published study included an evaluation of IRB effectiveness. Multiple studies evaluating the structure, process, and outcome of IRB review in the United States have documented inconsistencies and inefficiencies. Efforts should be made to address these concerns. Additional research is needed to understand how IRBs accomplish their objectives, what issues they find important, what quality IRB review is, and how effective IRBs are at protecting human research participants.

Get full access to this article

View all access and purchase options for this article.

References

Association for the Accreditation of Human Research Protection Programs, Inc. (AAHRPP). www.aahrpp.org.

Anderson E. (2006). A qualitative study of non-affiliated, non-scientist institutional review board members. Accountability in Research, 13(2), 135–155.

Bell J. (1998). Evaluation of NIH Implementation of Section 491 of the Public Health Service Act, Mandating a Program of Protection for Research Subjects. Bethesda, MD: National Institutes of Health.

Bramstedt K. & Kassimatis K. (2004). A study of warning letters issued to institutional review boards by the United States Food And Drug Administration. Clinical Investigational Medicine, 27(6), 316–323.

Burman W., Breese P., Weis S., Bock N., Bernardo J., Vernon A., et al. (2003). The effects of local review on informed consent documents from a multicenter clinical trials consortium. Controlled Clinical Trials, 24, 245–255.

Burman W. J., Reves R., Cohn D., & Schooley R. (2001). Breaking the camel's back: Multicenter clinical trials and local institutional review boards. Annals of Internal Medicine, 134(2), 152–157.

Byrne M., Speckman J., Getz K., & Sugarman J. (2006). Variability in the costs of institutional review board oversight. Academic Medicine, 81(8), 708–712.

Campbell E., Weissman J., Clarridge B., Yucel R., Causino N., & Blumenthal N. (2003). Characteristics of medical school faculty members serving on institutional review boards: Results of a national survey. Academic Medicine, 78(8), 831–836.

Campbell E. G., Weissman J. S., Vogeli C., Clarridge B. R., Abraham M., Marder J. E., et al. (2006). Financial relationships between institutional review board members and industry. New England Journal of Medicine, 355(22), 2321–2329.

Candilis P., Lidz C., & Arnold R. (2006). The need to understand IRB deliberations. IRB: Ethics and Human Research, 28(1), 1–5.

Clark S., Pelletier A. J., Brenner B. E., Lang D. M., Strunk R. C., & Camargo C. A. (2006). Feasibility of a national fatal asthma registry: More evidence of IRB variation in evaluation of a standard protocol. Journal of Asthma, 43(1), 19–23.

Coleman C. & Bouesseau M. (2008). How do we know if research ethics committees are really working? The neglected role of outcomes assessment in research ethics review. BMC Medical Ethics, 9, 6.

Cooke R. A., Tannenbaum A. S., & Gray B. (1977). A survey of institutional review boards and research involving human subjects. The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, DHEW.

DeVries R. G. & Forsberg C. P. (2002). What do IRBs look like? What kind of support do they receive? Accountability in Research, 9, 199–216.

Department of Health and Human Services (DHHS). 45 CFR 46.

Dziak K., Anderson R., Sevick M. A., Weisman C., Levine D., & Scholle S. (2005). Variations among institutional review boards reviews in a multisite health services research study. HSR: Health Services Research, 40(1), 279–290.

Edwards S., Ashcroft R., & Kirchin S. (2004). Research ethics committees: Differences and moral judgement. Bioethics, 18(5), 408–427.

Emanuel E. J., Wood A., Fleischman A., Bowen A., Getz K. A., Grady C., et al. (2004). Oversight of human participants research: Identifying problems to evaluate reform proposals. Annals of Internal Medicine, 141(4), 282–291.

Federal Drug Administration (FDA). 21 CFR 50.

Fost N. & Levine R. (2007). The dysregulation of human subjects research. Journal of the American Medical Association, 298(18), 2196–2198.

Government Accountability Office (GAO). (1996). Federal report says protection of human subjects is threatened by numerous factors. Human Research Report, 11(5).

Goldman J. & Katz M. D. (1982). Inconsistency and institutional review boards. Journal of the American Medical Association, 248(2), 197–202.

Gray B. H. (1975). An assessment of Institutional Review Committees in human experimentation. Medical Care, 13(4), 318–328.

Green L., Lowery C., Kowalski C., & Wyszewianski L. (2006). Impact of institutional review board practice variation on observational health services research. HSR: Health Services Research, 41(1), 214–230.

Greene S., Geiger A., & Harris E. (2006). Impact of IRB requirements on a multicenter survey of prophylactic mastectomy outcomes. Annals of Epidemiology, 16(4), 275–278.

Greene S. M. & Geiger A. M. (2006). A review finds that multicenter studies face substantial challenges but strategies exist to achieve Institutional Review Board approval. Journal of Clinical Epidemiology, 59(8), 784–790.

Grodin M. A., Zaharoff B. E., & Kaminow P. V. (1986). A 12-year audit of IRB decisions. Quality Review Bulletin, 12(3), 82–86.

Gunsalus C., Bruner E., Nicholas B., Dash L., Finkin M., Goldberg J., et al. (2006). Mission creep in the IRB world. Science, 312, 1441.

Hayes G. J., Hayes S. C., & Dykstra T. (1995). A survey of university institutional review boards: Characteristics, policies, and procedures. IRB: Ethics and Human Research, 17(3), 1–6.

Helfand B. T., Mongiu A. K., Roehrborn C. G., Donnell R. F., Bruskewitz R., Kaplan S. A., et al. (2009). Variation in institutional review board responses to a standard protocol for a multicenter randomized, controlled surgical trial. Journal of Urology, 181(6), 2674–2679.

Hirshon J. M., Krugman S. D., Witting M. D., Furuno J. P., Limcangco M. R., Perisse A. R., et al. (2002). Variability in institutional review board assessment of minimal-risk research. Academic Emergency Medicine, 9(12), 1417–1420.

Institute of Medicine (IOM). (2002). Responsible research: A systems approach to protecting research participants. Washington, DC: Institute of Medicine.

Jones J. S., White L. J., Pool L. C., & Dougherty J. M. (1996). Structure and practice of institutional review boards in the United States. Academic Emergency Medicine, 3(8), 804–809.

Kimberly M. B., Hoehn K. S., Feudtner C., Nelson R. M., & Schreiner M. (2006). Variation in standards of research compensation and child assent practices: A comparison of 69 institutional review board–approved informed permission and assent forms for 3 multicenter pediatric clinical trials. Pediatrics, 117(5), 1706–1711.

Larson E., Bratts T., Zwanziger J., & Stone P. (2004). A survey of IRB process in 68 U.S. hospitals. Journal of Nursing Scholarship, 36, 260–264.

Lemonick M., Goldstein A., & Park A. (2002). Human guinea pigs. Time Magazine, April 22.

Levine R. J. (1984). Inconsistency and IRBs: Flaws in the Goldman-Katz study. IRB: A Review of Human Subjects Research, 6(1): 4–6.

Loh E. D. & Meyer R. E. (2004). Medical schools' attitudes and perceptions regarding the use of central institutional review boards. Academic Medicine, 79(7), 644–651.

Mansbach J., Acholonu U., Clark S., & Camargo C. (2007). Variation in institutional review board responses to a standard, observational, pediatric research protocol. Academic Emergency Medicine, 14(4), 377–380.

Maschke K. (2008). Human research protections: Time for regulatory reform. Hastings Center Report, 38(2), 19–22.

McCarthy C. (1996). Challenges to IRBs in the coming decades. In Vanderpool Y. (Ed.), The Ethics of Research Involving Human Subjects: Facing the 21st Century (pp. 127–144). Frederick, MA: University Publishing Group, Inc.

McClure K., Delorio N., Schmidt T., Chiodo G., & Gorman P. (2007). A qualitative study of institutional review board members' experience reviewing research protocols using emergency exception informed consent. Journal of Medical Ethics, 33(5), 289–293.

McWilliams R., Hoover-Fong J., Hamosh A., Beck S., Beaty T., & Cutting G. (2003). Problematic variation in local institutional review of a multicenter genetic epidemiology study. Journal of the American Medical Association, 290(3), 360–366.

National Cancer Institute Central IRB Initiative. www.ncicirb.org.

National Institutes of Health (NIH). (2006). National conference on alternative IRB models: Optimizing human subjects protection. Paper presented at the National Conference on Alternative IRB Models: Optimizing Human Subjects Protection.

Office of the Inspector General (OIG). (1998a). Institutional Review Boards: Promising approaches. Washington, DC: US Department of Health and Human Services.

Office of the Inspector General (OIG). (1998b). Institutional Review Boards: A system in jeopardy. Washington, DC: Department of Health and Human Services.

Office of the Inspector General (OIG). (1998c). Institutional Review Boards: A time for reform. Washington, DC: U.S. Department of Health and Human Services.

President's Commission. (1983). Implementing human research regulations. Washington, DC: President's Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research.

Rogers A., Schwartz D., Weissman G., & English A. (1999). A case study in adolescent participation in clinical research: Eleven clinical sites, one common protocol, and eleven IRBs. IRB: Ethics and Human Research, 21(1), 6–10.

Rothstein W. G. & Phuong L. H. (2007). Ethical attitudes of nurse, physician, and unaffiliated members of institutional review boards. Journal of Nursing Scholarship, 39(1), 75–81.

Sengupta S. & Lo B. (2003). The roles and experiences of nonaffiliated and nonscientist members of institutional review boards. Academic Medicine, 78(2), 212–218.

Shah S., Whittle A., Wilfond B., Gensler G., & Wendler D. (2004). How do institutional review boards apply the federal risk and benefit standards for pediatric research? Journal of the American Medical Association, 291(4), 476–482.

Sherwood M. L., Buchinsky F. J., Quigley M. R., Donfack J., Choi S. S., Conley S. F., et al. (2006). Unique challenges of obtaining regulatory approval for a multicenter protocol to study the genetics of RRP and suggested remedies. Otolaryngology–Head and Neck Surgery, 135(2), 189–196.

Silverman H., Hull S. C., & Sugarman J. (2001). Variability among institutional review boards' decisions within the context of a multicenter trial. Critical Care Medicine, 29(2), 235–241.

Sobolski G., Flores L., & Emanuel E. (2007). Institutional review board review of multicenter studies. Annals of Internal Medicine, 146(10), 759.

Stair T. O., Reed C. R., Radeos M. S., Koski G., & Camargo C. A. (2001). Variation in institutional review board responses to a standard protocol for a multicenter clinical trial. Academic Emergency Medicine, 8(6), 636–641.

Stark A., Tyson J., & Hibberd P. (2010). Variation among institutional review boards in evaluating the design of a multicenter randomized trial. Journal of Perinatology, 30, 163–169.

Steinbrook R. (2002). Improving protection for research subjects. New England Journal of Medicine, 346(18), 1425–1430.

Strech D., Synofik M., & Marckmann G. (2008). Systematic reviews of empirical bioethics. Journal of Medical Ethics, 34(6), 472–477.

Taylor H. (2007). Moving beyond compliance: Measuring ethical quality to enhance the oversight of human subjects research. IRB: Ethics and Human Research, 29(5), 9–14.

Veterans Administration Central Institutional Review Board. www.research.va.gov/programs/pride/cirb/default.cfm.

Vick C., Finan K., Kiefe C., Neumayer L., & Hawn M. (2005). Variation in institutional review process for a multisite observational study. The American Journal of Surgery, 190(5), 805–809.

Wagner T., Bhandari A., Chadwick G., & Nelson D. (2003). The cost of operating institutional review boards. Academic Medicine, 78(6), 638–644.

Wagner T., Cruz A. M., & Chadwick G. (2004). Economies of scale among institutional review boards. Medical Care, 42(8), 817–823.

Weil C., Rooney L., McNeilly P., Cooper K., Borror K., & Andreason P. (2010). OHRP compliance oversight letters: An update. IRB: Ethics and Human Research, 32(2), 1–6.

White M. T. & Gamm J. (2002). Informed consent for research on stored blood and tissue samples: A survey of institutional review board practices. Accountability in Research, 9(9), 1–16.

Whitney S. N., Alcser K., Schneider C. E., McCullough L. B., McGuire A. L., & Volk R. J. (2008). Principal investigator views of the IRB system. International Journal of Medical Sciences, 5(2), 68–72.

Whittle A., Shah S., Wilfond B., Gensler G., & Wendler D. (2004). Institutional review board practices regarding assent in pediatric research. Pediatrics, 113(6), 1747–1752.

Wood A., Grady C., & Emanuel E. J. (2004). Regional ethics organizations for protection of human research participants. Nature Medicine, 10(12), 1283–1288.

Biographies

Lura Abbott is now retired, but worked on this article when she was on the staff of the Office of Human Subjects Research in the Division of Intramural Research, National Institutes of Health. Her responsibilities at OHSR involved overseeing a number of NIH Intramural IRBs. Previously, she had been an IRB coordinator for many years. She wrote her doctoral dissertation on the evaluation of IRBs.

Christine Grady is the Acting Chief of the Department of Bioethics, Clinical Center NIH. Her primary research interests are the ethics of clinical research, including interest in research oversight and IRB review. She has been a member of an IRB for over 20 years.

You currently have no access to this content. Visit the access options page to authenticate.