Guidelines to Meet Basic Research Standards for Residents in Emergency Medicine

• Introduction
   

  Purpose  To inform residents and program directors of the research project standards necessary to meet the ACOEP requirement for original research.1  This page intends to define types of acceptable research projects, specify the baseline research components of each type of project, and to provide guidance for program directors and residents for the basic minimum performance required to meet the ACOEP standards for research.  Scope  This page applies to those residents entering the emergency medicine residency.   1 “The resident shall complete a research project during the course of the Emergency Medicine training program. The resident must submit an outline for the project in the first training year, implementation and data collection methods with a second year interim report, and a final product suitable for publication in the third year of residency. The completed research project must be submitted to the ACOEP with the annual report. A permanent copy must be retained in the resident’s file. All research projects must be approved by the program director.” (Taken from ACOEP Basic Standards for Residency Training in Emergency Medicine V. G.)

• Permitted
  Research

  Types of Permitted Research Projects Biomedical Research is divided into primary data investigations or secondary data investigations. Primary data investigations are defined as either the collection of previously unknown data, or the analysis of data previously collected that re-defines a body of knowledge. Primary data investigations are divided into basic science research, clinical research, or epidemiological research. Secondary data investigations refer to re-working pre-existing analyses to sharpen conclusions such as a meta analysis of previously published studies, book chapters, or review articles. Secondary source investigations do not qualify as ACOEP original research projects.  Primary data investigations have three broad categories: basic science, which attempts to define pathophysiological mechanisms; clinical research that attempts to define the efficacy of diagnostics, therapeutics, and devices within individuals that have or may be at risk for the disease studied; and epidemiological research that attempts to define the prevalence of disease and effects of diagnostics or therapeutics within populations.  Basic clinical research is divided into theoretical and applied research. Theoretical research refers to research into biological mechanisms. An example of theoretical research would be the discovery of the double helix structure of DNA by Watson and Crick. Applied basic clinical research refers to the testing of hypothesis of potential mechanisms of action in cell culture, biochemical reactions, or animal models. While important, basic clinical and theoretical research is largely the domain of university-based medical researchers.  Clinical research is divided into experimental and observational research. Experimental clinical research may refer to clinical drug or device efficacy and safety trials such as Food and Drug Administration (FDA) trials (Phase I, II, III, and IV). Also included in experimental research are comparison trials to establish efficacy, superiority, or non-inferiority of diagnostic or therapeutic modalities. An example of comparison trials would include various thrombolytic trials in stroke. Because of the high degree of patient impact, these studies are generally overseen by regulatory organizations.  Observational clinical research is a robust area for resident emergency physician research. The lowest levels of observational clinical research are case reports and case series. Case reports and case series refer to the report of a novel presentation of disease that may indicate either previously unknown pathology or previously unknown efficacy of a treatment. Examples of important case report/series would be the treatment of anthrax victims in 2001-02. A simple case report and review of the literature (secondary data) no longer complies with the ACOEP definition of original research.  Prospective observational studies or retrospective chart reviews to determine the effectiveness of a given set of diagnostic tools or therapeutic strategies for a specific disease are an acceptable form of clinical research. These observational studies may focus upon diagnostic, therapeutic, or prognostic elements of the given condition. The detection of domestic violence by a novel screening tool deployed in the emergency department would be an example of a prospective, diagnostic, observational research project.  Epidemiological studies are divided into either experimental or observational studies. Experimental epidemiological studies refer to testing the efficacy of an intervention on a previously identified population with a given condition. The effect of fluoride on dental caries would be an example of an experimental epidemiological study. Epidemiological observation studies are divided again into prospective and historical studies.  Prospective epidemiological observation studies refer to defining a set of inclusion criteria and collecting descriptive data on the recruited subjects according to the research protocol or survey instrument. The Framingham study would be an example of a prospective epidemiological observational study. A retrospective epidemiological observational study would refer to the analysis of historical records to determine the presence or absence of a given condition or the outcomes of a given treatment. The correlation between traumatic death and motorcycle helmet use using a trauma system data-base would be an example of a retrospective epidemiological observational study.

• Research
  Responsibilities

  Research Responsibilities The key components of a resident research project are:  - Development of the research question or hypothesis  - Development of a research protocol to collect data related to the hypothesis  - Institutional Review Board (IRB) review and support  - Collection of the data  - Analysis of the data for statistical relevance  - Conclusions to support or not support the hypothesis  The development of a research question is the first step in a research project. From this first question, a hypothesis or hypotheses may be developed. This fundamental first step is often stimulated by clinical practice questions and/or an identified gap in the available body of knowledge. The optimal research question reflects a combination of interest of the resident in a given area and the lack of available studies to answer the question. Interest alone is, however, not sufficient. The research question must be supported by a literature review in the area of interest to define; 1) what is known and unknown about the topic, and 2) how previous investigators have attempted to gather data on the topic. Scientific relevance of the research question and support for research methodology are the goals of this phase of the research project. Failure of the resident research project at this stage is often due to a lack of comprehensive reading in the subject matter that leads to a study without scientific merit or a methodology that is not supported by the literature.  Following the literature analysis, the development of a proposed methodology of data acquisition, supported by previous studies, should be developed. The development of a data collection protocol may involve an experimental protocol for basic science research, a protocol for prospective data collection, the development of a survey tool, or a structured retrospective collection of data. This process is a shared resident-faculty process in which an experienced faculty member can guide the resident in the development of a research protocol that is simultaneously ethical, scientifically valid, and able to be accomplished by the resident in the time available. This proposed protocol and literature bibliography should form the basis of the IRB submission.  Following Institutional Review Board (IRB) approval, data collection should begin. The collection of data may be accomplished by primary resident effort, through the use of research assistants, or by the recruitment of fellow residents and staff depending upon the needs of the study and the resources available. For basic science research, a laboratory and experimental protocol should be conducted under the supervision of a competent researcher with experience in the proposed experimental protocol. For prospective research studies, subject selection and data collection must conform to ethical research standards, and Protected Health Information (PHI) must be safe-guarded by using Healthcare Insurance Portability and Accountability (HIPAA) compliant techniques. For retrospective research studies, data collection must be observant of HIPAA restrictions and blinded whenever possible. The work required to collect sufficient data or recruit sufficient subjects is often under-estimated by resident researchers and a common cause of failure at this phase of the research project. Appropriate effort in data collection or subject recruitment is a determination best made by the program director or a designee. A rough estimate for data collection would be that amount of data or chart review that could be reasonably accomplished within the research month.  The data collection period is followed by an analysis of the data, and then by a statement of support or non-support for the initial hypothesis. The data should be analyzed with appropriate statistical techniques to determine: 1) appropriate basic data distribution measures (example: mean, mediation, standard deviation, etc), 2) appropriate statistical validity measures (example: p-value), and 3) the appropriate level support or non-support of the hypothesis. Often, statistical support is needed for higher statistical measures. Statistical support for validity measures is a responsibility of the program, training institution, or Osteopathic Postdoctoral Training Institution (OPTI) and not the responsibility of the individual resident. Common issues at this phase of research are statistical limitations caused by a small data set. If sufficient effort was expended in the data collection phase, then a resident should not be penalized for the lack of statistical validity due to a small data set.

• Research with
  Multiple Residents

  Research with Multiple Residents A resident research project may be shared by two or more residents. Partnership among residents or residency training sites may greatly increase the data collection/subject recruitment, and therefore can increase the statistical validity for a given study. Unfortunately It is also possible for a resident to attempt to avoid or shift work onto a colleague. To prevent such a scenario, the practice of collaborative research should always include a proscriptive written agreement that includes timelines, and apportions the responsibility for explicit aspects of the project to specified individuals or parties. This agreement would be signed by each resident partner and their faculty supervisor or program director. Shared duties must include background reading and development of the research question. If more than one institution is involved, the IRB process need be only engaged in one and reviewed by the other if allowed by institutional rules. Data collection agreements should specify the minimum expected amount of data to review, or the minimum number of subjects to be enrolled by each partner. If one institution has access to statistical analysis, then it is not required that each respective group duplicates the same measurements. It is possible for one resident to fulfill and another to fail the basic research requirement on the same project if the involved participants failed to achieve their predetermined objectives as determined by the faculty supervisor or program director.

• Level of Effort
  Guidelines

  Level of Effort Guidelines Expected levels of effort are best determined by the supervising faculty or program director, but the expected effort should conform to approximately a minimum of one month of full-time work or approximately 26 days of non-contiguous effort. Given that a research project is a non-contiguous endeavor, enforcing guidelines for effort are the domain of the faculty mentor and program director. Further, different research projects may require more or less effort at any given stage. It is expected that the faculty mentor/program director adapt the following effort guidelines for the individual resident and project.  A single resident project should optimally have reviewed 15-30 articles at an approximate pace of one per hour as background to the work. Using a 12 hour work day as a basis, the literature review component may be expected to require two to four days of effort. Development of the research question and bibliography may be estimated to require one to two days of effort, to include meetings with faculty supervisors, bibliography construction, protocol development, and further readings. IRB submission paper work and associated reviews will require approximately one day of effort. Data collection should occur over ten to fourteen days. This is typically accomplished during the research or administrative months of the residency. Residents will require approximately one day of data entry and basic data measurements. Note that advanced statistical analysis is NOT INCLUDED into this estimate, since this detailed level of analysis is typically performed by a contracted individual or other academic resource. Writing the manuscript with conclusions, limitations, and construction of appropriate tables and graphical representations is approximated at four days to include primary writing and revisions prior to final submission.  Multiple resident projects effort levels should roughly multiply by the number of residents involved in the project.

• Suggested 
  Timeline

  Suggested Research Timeline The development of a valid, ethical, and successful resident research project requires distributed effort over several months to years. While many programs allow a month of “research”, “admin”, or “elective” in which the project may be completed, the time and complexity of original data research ensures that the required work will not fit into the designated month of effort. A suggested timeline for research milestones is as follows:  OPTI Resources  To support the proposed project, resources within the affiliated OPTI should be developed or accessed. These resources should include the following:  - Faculty research project review if needed expertise is not present within the residency program.  - Higher level statistical analysis2 if not present within the residency program.  - Paper editing capability if not present within the residency program.  Program Resources  - Designated faculty research resource to help with research question refinement, IRB sponsorship, and project mentoring.  - One month of available time for research project development and processing. This may or may not be contiguous time.  - Work space and computer resources suitable for the conduct of research.  2 Higher level statistical analysis to include power analysis, statistical significance, and validity studies.