HOW TO READ A SCIENTIFIC PAPER

Bill Valenti, MD

Introduction

This article is designed to provide a brief overview of how to critically read a scientific paper. Scientific papers are usually organized using a standard format1 consisting of:

" Summary or abstract

" Introduction

" Materials and methods

" Results section

" Discussion

" Acknowledgements

" Reference list

This format follows The Uniform Requirements for Manuscripts Submitted to Biomedical Journals of the International Committee of Medical Journal Editors (ICMJE), which is used by over 500 medical journals. It is published on the web site of the Journal of the American Medical Association (JAMA) (see Web resource list below). As these statements from the ICMJE are revised and updated, they are reported in major scientific journals.

In general, scientific articles are peer-reviewed. However, not all scientific articles make it to publication in peer-reviewed journals. According to the ICMJE, "a peer-reviewed journal is one that has submitted most of its published articles for review by experts who are not part of the editorial staff. The number and kind of manuscripts sent for review, the number of reviewers, the reviewing procedures, and the use made of the reviewers opinions may vary, and therefore each journal should publicly disclose its policies in its instructions to authors for the benefit of readers and potential authors." Journals are listed in the Index Medicus, a reference work that lists all articles appearing in medical journals, based on a variety of criteria including peer-review (see Web resource list below).

Getting Started

A useful review entitled "How to Read a Scientific Paper," by John W. Little and Roy Parker of the University of Arizona, has been published on the Internet (see Web resource list below). Little and Parker recommend starting by reading the title and abstract first to get an idea of what is being discussed in the paper and as a way of refreshing the readers memory about the subject. They also remind the reader to try to link what is read in the abstract to what the individual reader already knows about the subject being discussed.

These authors also list a series of questions to ask when starting to read a paper. Start by reading the introduction, which usually starts out generally and then becomes more specific, and consider the following questions:

" What questions does the paper address?

" What are the main conclusions of the paper?

" What evidence supports those conclusions?

" Do the data actually support the conclusions?

" What is the quality of that evidence?

" Why are the conclusions important?

Keeping these questions in mind will help you read a paper more analytically and will make the results and conclusions sections more meaningful to you.

Because of the space limitations required by peer-reviewed journals, authors need to write in a clear, concise style, yet provide enough detail to support their conclusions. The additional burden on the reader is to read the paper critically and not overlook important information such as is provided in graphs, charts, tables, and statistical analysis.

Statistical Methods

Most papers that describe study results use statistical analysis to support their conclusions. A review of statistics is beyond the scope of this paper. However, there are many good references that provide this information.

A paperback book by Carr includes a CD-ROM that makes the learning process more interactive.2

In so-called evidence-based medicine, conclusions are validated using scientific methods. These conclusions can then be applied to clinical practice.3 Scientific papers dealing with HIV/AIDS use several important statistical methods to validate their results. Many of these statistical methods are required by the U.S. Food and Drug Administration (FDA) for licensure of new drugs and for new indications for licensed drugs (see the presentation by D. Lin in the resource section below).

Perhaps the most common statistical methods used are tests of significancethat is, tests that compare results among groups to determine if differences between the groups are statistically significant and not due to chance.1 This time-honored statistical evaluation generally sets the limit for statistical significance at a p-value of ¾.05. This means that there is a 5% chance that the results are due to chance. Conversely, there is a 95% chance that the results are due to a significant difference between the groups. A recently published paper looking at discontinuation of prophylaxis for Mycobacterium-avium complex (MAC) uses this statistic.4 Other types of analysis are discussed below.

Observed Data: This kind of analysis, often called an on-treatment analysis, looks at one group and compares it to a control group. Only patients with data available when the analysis is made are counted. Missing information, such as data relating to patients who were randomized but did not participate in the study, or to those who did not complete the study, are not counted in the analysis.

Intent to Treat (ITT): Intent-to-treat analysis is designed to evaluate how drugs perform in the real world. ITT analysis is more conservative than on-treatment analysis. There are two types of ITT: (a) last observation carried forward (LOCF) and (b) non-completer equals failure (NC = F). Of the two, the NC = F method is the most conservative.

In LOCF analysis, data are analyzed based on the last observed measurement of each study participant. If data are not available at the endpoint of the trial, the last observation for subjects is "carried forward" and used in the final analysis. In this type of trial, all patients randomized in the study are included in the evaluation.

In the NC = F analysis, all patients randomized are included in the analysis. In contrast to the LOCF method, patients whose data are missing for any reason are considered failures at the time of analysis. There is one exception in the NC = F analysis. Patients whose data are unavailable at the time of analysis, but who have data available before and after the analysis are excluded or "censored." In other words, in order to be considered a treatment success, the patient must have data available at the time of the analysis.

Time to Treatment Failure (TTF): TTF analysis shows the time after day zero (i.e., the time elapsing between initiation of treatment) for patients who initially achieved a successful response, to the time point where the first day of failure, based on the studys failure criteria, occurs. This allows treatment arms in a study to be compared using tests of statistical significance as described above. The U.S. Food and Drug Administration (FDA) now requires this type of analysis as a part of the approval process for antiretrovirals.

For more information about these statistical methods, visit the duPont Pharmaceuticals web site provides examples of their use.

Summary

It is important to read scientific papers critically in order to apply evidence-based medicine to clinical practice. Reading a scientific paper can be a rewarding experience when it is done using an analytical approach that looks at the paper in a systematic fashion. Some of the resources shown below provide additional details.

Web Resources

The Uniform Requirements for Manuscripts Submitted to Biomedical Journals web site of the Journal of the American Medical Association

<http://jama.ama-assn.org/info/ auinst_req.html>

Journals listed in the Index Medicus

<http://www.nlm.nih.gov/pubs/factsheets/jsel.html>

Parker R. and Little JW. How to Read a Scientific Paper <http://www.biochem.arizona.edu/classes/bioc568/papers.htm#questions>

Lin D. Statistical considerations for

clinical trials in developing antimicrobial drugs. From the FDA Anti-infective Drug Products Advisory Committee (PowerPoint® presentation). July 29, 1998. Available at:

<http://www.fda.gov/cder/present/anti-infective798/biostats/sld001.htm>

Lecture notes for discussion of randomization and intent-to-treat analysis:

<http://www.biostat.washington.edu/biostat/faculty/rossini/talk/rct-itt-html/>

The duPont Pharamceuticals web site discussion of various statistical methods

<http://www.sustiva.com>