This Article Abstract Full Text (PDF) Submit a scholarly reply Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Related articles in CJA Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Hall, R. Articles by Webber, A. Search for Related Content PubMed PubMed Citation Articles by Hall, R. Articles by Webber, A.

Publication bias in the medical literature: a review by a Canadian Research Ethics Board

[Biais de publication dans la littérature médicale : un compte rendu d’un Comité d’éthique de recherche canadien]

Richard Hall, MD FRCPC FCCP^*, Cecilia de Antueno and Adam Webber, MSC^*

^* From the Dalhousie University and the Capital District Health Authority, and the
Capital District Health Authority Research Ethics Board, Halifax, Nova Scotia, Canada.

Address correspondence to: Dr. Richard Hall, Department of Anesthesia, Queen Elizabeth II Health Sciences Centre, 1796 Summer Street, Room 5452 HI, Halifax, Nova Scotia B3H 3A7, Canada. Phone: 902-473-2328; Fax: 902-473-4828; E-mail: rihall{at}dal.ca

	Abstract

TOP Abstract Introduction Methods Results Discussion References

 
Background: We reviewed the publication record of all protocols submitted to the Capital District Health Authority Research Ethics Board (REB) in Halifax, Nova Scotia, for the period 1995-1996. Because of a heightened awareness of the issue, we hypothesized that there would be less publication bias (a failure to report negative results) and a higher publication rate from completed studies, than previously reported.

Methods: Closed studies were identified from the REB database. Publications were identified by the investigators, requests from sponsors, and a literature review. For each publication, we identified authors, title, journal, number of subjects enrolled, and whether or not the publication was a report of a randomized clinical trial. Comparisons were done using a Student’s t test, the Chi-square statistic, or Fisher’s exact test as appropriate.

Results: From the database of closed studies, 106 remained unpublished, while completed investigations resulted in 84 publications (44% publication rate). The median time to publication was 32.5 months. Publication of statistically significant results occurred in 71/84 trials. Publication of protocols submitted by departments ranged from 91% (anesthesia; 10/11) to 25% [nursing; 2/8 (P < 0.05)]. Trials investigating new drugs in Phase 3 or 4 studies were more likely to be published than trials investigating agents in Phase 1 or 2 (P < 0.05), and were less likely to be published if sponsored by a pharmaceutical company (P < 0.05).

Conclusions: Publication bias continues to be a problem, particularly for early phase investigative studies. Our results suggest that a different approach is required to reduce publication bias. The role that REBs and peer-reviewed journals might play requires further exploration.

	Introduction

TOP Abstract Introduction Methods Results Discussion References

 
REPORTING of information obtained from clinical investigations is an important component of the conduct of research. Without publication in peer-reviewed, indexed biomedical journals, there can be no enduring dissemination of the results derived from the trial. A failure to undertake reporting and publication may violate the social contract undertaken by research subjects, as most consent forms list as one of the benefits (perhaps the only benefit) of their participation in the research, is that information provided from the study will be helpful in the management of the disease for others in the future. Because research results reflect the cumulative nature of investigations, failure to publish results of selective scientific investigations (i.e., publication bias)^1–3 leads to misinformation, particularly as it applies to literature reviews including meta-analysis. ^2,4,5 Failure to publish results may lead to ongoing investigation with further wasting of resources, both human and material, and may lead to unnecessary risk for future research subjects. Indeed, failure to publish has been considered a form of scientific misconduct by some commentators.^6,7

As part of their mandate, Research Ethics Boards (REB) (also denoted as Institutional Review Boards) have a responsibility for the oversight of research carried out under their aegis. This mandate is considered by some to include ensuring publication of trial results.^8,9 Because all human clinical research funded by federal granting agencies in Canada, and all investigational drug trials must be submitted to a REB, REBs are uniquely positioned to be able to determine the fate of protocols submitted for their review.^1,3,9–12 We reasoned that if publication bias could be demonstrated for trials submitted for review by a Canadian REB, it would be reasonable to request trial registration in a publicly accessible database prior to REB approval in an attempt to reduce such publication bias. This approach would be similar to that (but more inclusive of early Phase 1 - 2 clinical trials) recently adopted by several leading biomedical journals.^13,14 Because of increased awareness of the problem of publication bias, we hypothesized that there would be less publication bias, and that the publication record of completed clinical trials from a single Canadian centre would be higher than previously reported. If such were the case, more stringent requirements for clinical trial registration (with perhaps increased times to full REB approval) would not be warranted.

	Methods

TOP Abstract Introduction Methods Results Discussion References

 
The Capital District Health Authority (CDHA) REB is the largest REB in Eastern Canada, and reviews protocols for the conduct of all clinical research involving adult human research subjects carried out at the CDHA (encompassing Halifax and the surrounding area, including all adult teaching hospitals affiliated with Dalhousie University in the Halifax area). The REB follows the ethical concepts outlined in the Tri-Council Policy Statement "Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, Social Sciences and Humanities Research Council of Canada, Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans 1998 (with 2000, 2002 and 2005 amendments)¹⁵ (accessed 30 Oct 2006), the principles outlined in The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use Good Clinical Practices guidelines¹⁶ (accessed 30 Oct 2006) and applicable regulations, including those enunciated in Division 5 of the Canadian Food and Drug Act¹⁷ (accessed 30 Oct 2006).

The protocol was reviewed by the Dalhousie University REB (an REB independent of the CDHA REB). As this was a quality assurance initiative with no subject, investigator, or sponsor- specific information being disseminated, and with the published information already in the public domain, the protocol was deemed not to require REB approval. Protocols for studies listed as closed (i.e., no longer following research subjects) by the principal investigator (PI) for the years 1995-1996 were identified from the REB database. For each protocol, the PI was identified and a letter was sent requesting any information concerning publication of the work, the reference citation, and the number of subjects enrolled locally. When the PI did not identify whether a publication had resulted, the appropriate sponsor was notified and a request was made to retrieve the information. If neither initiative identified a publication, the National Library of Medicine PubMed database was searched to identify publications with the methods and treatments identified in the protocol, and the list of authors and contributors was reviewed to determine whether the local PI was listed as a contributor.

For each protocol we identified the local PI, the department performing the research, the sponsor, the phase of clinical investigation (Phase 0 (non-drug trial), 1 (normal volunteers), 2 (safety), 3 (efficacy) or 4 (post marketing)), whether terminated early and if so why, and the date the trial was stopped.

For each publication, we identified the method of publication identification (PI, sponsor, or literature search), publication authors, publication title, date of publication, journal, whether a randomized clinical trial or not and, if so, a Jadad score¹⁸ as a measure of trial quality was assigned. The journal impact factor¹⁹ (accessed 08/03/2005) was determined, as well as total number of subjects enrolled, study start and end time, time to publication, and whether REB approval was mentioned as part of the study methods.

For data analysis, Systat V10 SPSS Inc. 2000 Standard Version Chicago Ill, GraphPad Instat v 3.06 GraphPad Software Inc. 1992–2005 El Camino Real, San Diego, CA, USA; SigmaStat v3.5 Systat Software Inc. Richmond, CA, USA; and SAS V9.1, SAS Institute Inc., Cary, NC, USA were utilized. Continuous data were analyzed using Student’s t test. Categorical data were analyzed using Fisher’s exact test or Chi-square analysis as appropriate. P < 0.05 was taken as statistically significant and corrected for multiple comparisons using a Bonferroni correction. To control for the number of corrected comparisons, only those categories for which an individual cell contribution to the overall Chi-square exceeded 2.706 (P = 0.10) were considered valid for pairwise comparison investigations. Results are expressed as mean ± SD, median ± 95% confidence interval (CI) or ratio ± 95% CI unless otherwise specified.

A multivariate mixed model was constructed to determine which independent variables (from study phase of development, study sponsor, study design, whether a drug study or not, or PI’s department) were useful predictors of publication in the presence of other independent variables. The best model was constructed using Akaike’s information criteria (AIC) using alpha = 2.

	Results

TOP Abstract Introduction Methods Results Discussion References

 
For the years 1995-1996, 190 protocols submitted to the CDHA REB met the criteria for review. Of these, 84 resulted in a publication as identified by the PI (n = 37), sponsor (n = 1), and a literature review (n = 46). The overall publication rate was 44% for all protocols submitted for ethical review. Of the 46 publications which were identified through the literature search, 34 (74%) were multicentre clinical trials. Physicians were the first author in 79/84 publications (94%) and were the PIs in 89% (170/190) of studies. Eight published trials were terminated early in their course. Of these, four showed efficacy at an interim analysis, three were terminated due to poor enrolment, and one was stopped due to the sponsor’s decision to discontinue the development of the drug consequent to a change in corporate structure (merger). Twelve publications (14%) had been forwarded to the REB to complete the file prior to this review. The median number of subjects enrolled per published study was 339 (range 0–117192). Of the 53 publications for which it was possible to determine the time to publication (as identified by the end of recruitment or follow-up as described in the Methods section of the publication until publication occurred), the median time to publication was 32.5 months (975 days) (range 4.5–101 months) (mean ± SD was 33.1 ± 17.8 months).

Unpublished vs published studies
For unpublished studies, subject recruitment by local investigators was reported for 43/106 studies with a median of two subjects recruited per study (range 0–800). Reasons for failure to complete studies which did not get published are presented in Table I. Of note, no reason for termination of the study or failure to publish results was given in 49/70 (70%) industry-sponsored trials and in 21/32 (66%) of studies sponsored by the local health authority.

Influence of phase of development
Publications according to phase of development are presented in Table II. Phase 2 and Phase 3 studies were identified as valid for pairwise comparisons by their contributions to the overall Chi-square, giving a Bonferroni corrected P-value of 0.05/7 = 0.0071. Phase 2 studies were less likely to be published than Phase 3 studies [relative risk (RR) 0.35; 95% CI 0.18–0.68; P = 0.0007]. These findings were not altered when the five studies never initiated [lack of funding (n = 3) or no REB approval (n = 2)] were removed from the analysis (RR 0.39; 95% CI 0.20–0.77; P = 0.0059) nor were they different when Phase 1-2 studies were combined and compared to Phase 3-4 studies combined (RR 0.37; 95% CI 0.20–0.71; P = 0.0002).

Quality of published studies
Quality of published studies described as randomized clinical trials [n = 56 (67%)] using the Jadad score¹⁸ were 1 (Poor Quality (n = 1)), 2 (n = 11), 3 (n = 15), 4 (randomized, blinded) (n = 29). There was no difference in the quality of randomized clinical trials sponsored by industry as compared to trials sponsored by federal granting agencies (P = 0.173) nor was there any difference in the quality of the publication as it related to the phase of drug development (P = 0.586). Mention of REB review was similar in frequency across all quality grades (P = 0.072). Considering only those randomized clinical trials sponsored by the pharmaceutical industry (n = 50), the Jadad scores were: 1 (n = 1), 2 (n = 8), 3 (n = 13), 4 (n = 28).

The distribution of published manuscripts according to journal impact factor (the mean journal citation frequency) is presented in Table IV. Thirty-seven percent (31/84) of publications occurred in journals with an impact factor > 5. There was no difference in publication success in high impact journals i.e., > 5 for trials reporting statistically significant or non-significant results (25/31 for journals with impact factor > 5 vs 46/53 for journals with impact factor < 5) (RR 0.929; 95% CI 0.759–1.137; P = 0.537). For publications sponsored by the pharmaceutical industry 24/60 (40%) were published in journals with an impact factor > 5 as compared to 6/11 (55%) by federal granting agencies (RR 0.911; 95% CI 0.734–1.127; P = 0.509), 1/3 (33%) by a charitable organization (RR 1.013; 95% CI 0.908–1.131; P = 1.00), and 0/10 (0%) of those sponsored by the health authority (RR 1.278; 95% CI 1.097–1.488; P = 0.012). For studies classified as Phase 0–2, 9/25 (36%) publications occurred in journals with impact factors > 5 vs 22/59 (37%) publications for Phase 3–4 studies (RR 0.962; 95% CI 0.484–1.910; P = 1.00). Ethical approval for the conduct of the trial was reported in 64/84 publications (71%). There was no difference in the frequency of mention of ethical approval in journals with impact factors > 5 [27/31 (87%)] vs 37/53 (70%) in journals with impact factors < 5 (RR 1.248; 95% CI 0.998–1.559; P = 0.110).

Publication of studies according to study design are presented in Table V. Randomized clinical trials and non-randomized clinical trials were identified as valid for pairwise comparisons by their contributions to the overall Chi-square, giving a Bonferroni corrected P-value of 0.05/5 = 0.01. Case control was not considered for pairwise comparisons due to the small sample size. As compared to randomized clinical trials, there was no difference in the publication success for observational studies (RR 1.288; 95% CI 1.006–1.649; P = 0.04, not significant when adjusted for multiple comparisons) while non-randomized clinical trials (RR 1.771; 95% CI 1.374–2.283; P < 0.001) and studies of a qualitative nature (RR 1.246; 95% CI 1.048–1.481; P = 0.008) were less likely to be published.

	Discussion

TOP Abstract Introduction Methods Results Discussion References

 
This study examined the publication record for protocols submitted to the REB of a Canadian Academic Health Science Centre for the years 1995–96. We have determined that only 44% of protocols submitted for review by the CDHA REB eventually had their results published in a peer-reviewed journal. Of manuscripts which were published, a greater proportion had positive results for the primary outcome of interest and were studies investigating pharmaceutical agents in the later stages of development (Phase 3 or 4), suggesting that publication bias was evident.

Only 12/84 (14%) publications were identified from reviewing REB files. When queried, investigators identified a further 26 reports (31%), and subsequent contact with the sponsor an additional one report. We were able to identify a further 46 studies (54%) (of which 34 (74%) were multicentre clinical trials) as being published through a careful review of the literature identified by the National Library of Science PubMed database. It would appear that many investigators were unaware of trial publication, particularly if they were a site investigator in a multicentre clinical trial. Alternatively, busy investigators might simply have declined to inform the REB about the publication even if they were aware one had occurred. Sponsor and investigator unwillingness to provide the REB with information concerning publication of trial results (whether published or not) is a significant concern, as it demonstrates a lack of respect for and understanding of the role of REBs in the publication of research. This responsibility has been well articulated by Mann.⁸ Failure of the sponsor or investigator to respond to requests for publication information is also problematic from a moral perspective, given that research subjects have usually given permission to be involved in their studies with the understanding that the information to be derived from the trial will be placed in the public domain so that others may benefit from the results. Failure to publish trial results thus violates this social contract.

We found differences between the phase of development of randomized clinical trials and publication of results. The earlier the study was in the development of a drug or device, the less was the likelihood that the results were published (Table II). In addition, we found differences between the type of sponsor and the likelihood of publication (Table III). When we examined the role of sponsors and publication during phase of development, we found that industry sponsored trials for Phase 2 or earlier were more likely not to have been published, than studies conducted by other funding agencies. While the number of federal/charitable organization trials is small, these results suggest a selection bias by industry as to which studies will be published. This observation is not unique to this study and has been commented upon by Herxheimer.²⁰ When trial results are not published, the contribution of the research subject is ignored and individuals who might have benefited are denied the opportunity to learn from the study results. Any harm that has occurred is not put in the public forum and the potential to avoid making similar errors is lost.^21,22 Tragically, the events of the trial of the monoclonal antibody TGN1412 identified the potential for harm to occur in such studies.^23,24 To reduce publication bias, there is evidence that registration of clinical trials in a publicly accessible database leads to more open disclosure of meaningful trial data.²⁵ Based on our results, it would thus seem reasonable to request clinical trial registration in a publicly accessible database for all clinical trials rather than the more restrictive requirements (primarily Phase 3–4 randomized clinical trials) outlined by the International Committee of Journal Medical Editors.^13,14

Investigators must also share some responsibility to ensure research results are disseminated. The CDHA REB has protected investigators’ rights to publish trial results by stipulating this explicitly in its contract language.²⁶ Publication of significant adverse events might prevent subsequent further harm to patients even if the number of locally recruited patients is small (median of 2 in our study).²⁷

Failure to publish statistically non-significant findings may bias the literature in favour of a treatment effect^28,29 and misinform literature reviews including meta-analysis and public health policy.^4,5,30,31 Only 15% of trials published in this study could be described as reporting statistically non-significant results suggesting a publication bias in favour of statistically significant trial results. Reasons for this have been described by others^32,33 and may include: study design limitations or inadequate power³⁴; failure of investigators to submit statistically non-significant results either for lack of interest, fear of rejection, or lack of time to pursue publication; poor scientific writing (although truly worthy information should be capable of publication with the help of journal editors); the tendency for journals to reject statistically non-significant studies;^35,36 and disinterest in, or deliberate suppression of, statistically non-significant results by the sponsor.^30,37–42

A comparison of publication success by medical specialties has not, to our knowledge, been reported previously. Publication rates for medical specialties varied from a low of 14% for nursing studies to a high of 91% for anesthesia protocols. Nursing protocols (7/7) tended to examine quality of care while anesthesia protocols (9/11) primarily involved pharmaceutical industry sponsored drug trials. Why such discrepancies in publication success occurred is unknown. We caution, however, that the number of publications per specialty is small and our results should be seen as hypothesis-generating rather than as definitive.

There are at least six similar studies examining the fate of research protocols submitted to REBs.^1,3,9–12 These studies have identified many issues similar to ours including: a statistically significant negative relationship between publication and industry sponsorship of the trial;^1,3 publication of studies primarily reporting statistically significant results;^3,12 a lack of funding,^3,9,12 poor recruitment,^9,12 early results showing no difference in outcome,^9,12 sponsor’s decision,⁹ and no reason given⁹ as a reason for failure to initiate or continue a study. Taken together, the results of the current study are consistent with the findings of previous studies of a similar nature, and extend the observations to a different locale (Canada), while providing further insights into the nature of published/unpublished protocols, and observations on publication by medical specialties. Somewhat disappointingly, we must conclude that our original hypothesis (i.e., that there would be less publication bias and more successful publication in our centre) was not supported by the data.

There are limitations with respect to methodology. We report a retrospective analysis of trials submitted to a single REB and it is possible that other REBs would have generated a different result. Because we limited our literature search to the PubMed database, we may have overlooked publications in other databases or in symposiums.⁴³ We were able to determine why publication had not occurred for only 35 trials. Of these, 14 (40%) were never initiated for financial or regulatory reasons. Were this to be the situation for the remaining 71 trials, any suggestions concerning the value of trial registration would have to be significantly tempered. However, for industry-initiated and sponsored studies (70/106; 66%), regulatory approval is required prior to receiving CDHA REB approval, and financial resources to initiate the trial are required prior to commencement of the study. Furthermore, while implicit in our request to investigators and sponsors for publication information, we did not explicitly enquire as to whether a manuscript had been submitted for publication and subsequently rejected for publication. This would of course have a different implication for our results than what we have described. However, it seems highly unlikely that all the unpublished studies resulted from a rejection for publication. Indeed, none of the studies for which a reason for failure to publish results could be ascertained (Table I) had this factor as the explanation. We believe our observations and suggestions concerning publication bias are therefore still valid.

Our study suggests that little has changed in the nearly two decades since publication bias was described, and a different approach to its management is required. Research Ethics Boards could play a role by requiring mandatory trial registration in a publicly accessible database prior to study initiation. At least for local investigators, this would permit the REB to curtail further research by that investigator, if it could be determined that failure to publish or report trial results was a consistent problem. However, REBs - even with a prior trial registration policy – will have little influence on publication bias until journals take a firmer stance on the issue of trial registration prior to publication. What role can the Canadian Journal of Anesthesia play in promoting the ethical dissemination of study results? We examined the instructions for authors for the following journals to determine whether there was a requirement for registration of clinical trials prior to publication: Canadian Journal of Anesthesia; Anesthesiology; Anesthesia & Analgesia; Anaesthesia; British Journal of Anaesthesia; Critical Care Medicine; Journal of Intensive Care Medicine; Intensive Care Medicine; Critical Care; Anaesthesia and Intensive Care. Many of these journals reference the International Committee of Journal Medical Editors requirements for manuscript publication which now includes mandatory trial registration prior to publication for randomized controlled trials. Phase 1 trials designed to assess major unknown toxicity or pharmacokinetics are excluded from this requirement, as are Phase 2 trials designed to investigate the biology of disease. Trials which provide preliminary data that may lead to larger, clinically directed trials may also be excluded if a compelling rationale to do so can be given^13,14 and trials such as meta-analyses, case control studies, and other types of study design other than randomized clinical trials are also excluded. Only Anaesthesia and Intensive Care and Critical Care have explicit statements requiring registration of trials as a condition of publication. The Canadian Journal of Anesthesia and Anesthesia & Analgesia include a statement in their respective Instructions for Authors encouraging authors to register their clinical trial. Based on our results, we believe the Canadian Journal of Anesthesia could provide leadership by mandating that all randomized clinical trials involving research in human subjects be registered in a publicly-accessible database at the time of REB approval, as a condition for acceptance of an author’s work. Public registration of clinical trials at inception is in keeping with the position adopted by the Canadian Institutes of Health Research and, as stated above, for some journals for some studies. There is evidence that mandatory public registration of some clinical trial protocols has already had an impact on the provision of information of relevance to future publication of trial results.²⁵ We therefore suggest that adherence to calls for mandatory registration of all clinical trials involving human subjects research should be considered by all biomedical journals so as to further reduce publication bias.

In conclusion, fewer than half of randomized clinical trials which received REB approval resulted in published manuscripts. Publication was less likely to occur if sponsored by a pharmaceutical company (particularly if in the early phase of drug development), and more likely to occur if reporting statistically significant findings, indicating publication bias (failure to report negative trials). A significant amount of research conducted in Canada never appears in the public forum, thus violating the social contract between research subjects and investigators/sponsors. Our results suggest that while publication bias has been a concern for over a decade, little has changed since it was first recognized. A different approach is necessary to reduce this problem. Based on the results of the current study, in an attempt to improve this situation, the CDHA REB has instituted a policy requiring public registration of all clinical trials it reviews before approval of the study protocol is granted. We encourage other REBs to do likewise.

	Footnotes

 
This study was presented in part at the Annual Meeting of the American College of Chest Physicians in Montreal, October 2005.

Accepted for publication October 23, 2006. Revision accepted December 7, 2006.

This article is accompanied by an editorial. Please see Can J Anesth 2007; 54: 331–5.

	References

TOP Abstract Introduction Methods Results Discussion References

 
1 Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet 1991; 337: 867–72.[Medline]

2 Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA 1990; 263: 1385–9.[Abstract]

3 Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA 1992; 267: 374–8.[Abstract]

4 Sutton AJ, Duval SJ, Tweedie RL, Abrams KR, Jones DR. Empirical assessment of effect of publication bias on meta-analyses. BMJ 2000; 320: 1574–7.[Abstract/Free Full Text]

5 Egger M, Smith GD. Misleading meta-analysis. BMJ 1995; 310: 752–4.[Free Full Text]

6 Antes G, Chalmers I. Under-reporting of clinical trials is unethical. Lancet 2003; 361: 978–9.[Medline]

7 Chalmers I. Underreporting research is scientific misconduct. JAMA 1990; 263: 1405–8.[Abstract]

8 Mann H. Research ethics committees and public dissemination of clinical trial results. Lancet 2002; 360: 406–8.[Medline]

9 Decullier E, Lheritier V, Chapuis F. Fate of biomedical research protocols and publication bias in France: retrospective cohort study. BMJ 2005; 331: 19.[Abstract/Free Full Text]

10 Sune-Martin P, Montoro-Ronsano JB. Role of a research ethics committee in follow-up and publication of results. Lancet 2003; 361: 2245–6.[Medline]

11 Pich J, Carne X, Arnaiz JA, Gomez B, Trilla A, Rodes J. Role of a research ethics committee in follow-up and publication of results. Lancet 2003; 361: 1015–6.[Medline]

12 Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ 1997; 315: 640–5.[Abstract/Free Full Text]

13 DeAngelis CD, Drazen JM, Frizelle FA, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. JAMA 2004; 292: 1363–4.[Free Full Text]

14 DeAngelis CD, Drazen JM, Frizelle FA, et al. Is this clinical trial fully registered? A statement from the International Committee of Medical Journal Editors. JAMA 2005; 293: 2927–9.[Free Full Text]

15 Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, and Social Sciences and Humanities Research Council of Canada. Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, Social Sciences and Humanities Research Council of Canada, Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans. 1998 (with 2000, 2002 and 2005 amendments). Interagency Advisory Panel on Research Ethics (PRE) 2005. Available from URL; www.pre.ethics.gc.ca/english/policystatement.

16 The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). E6(R1): Good Clinical Practice : Consolidated Guideline. The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) 2006. Available from URL; www.ich.org.

17 Government of Canada. Division 5 Drugs For Clinical Trials Involving Human Subjects. Food and Drugs Act ( R.S., 1985, c.F-27 ) 2006. Available from URL; http://laws.justice.gc.ca/en/F-27/C.R.C.-c.870/238148.html.

18 Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996; 17: 1–12.[Medline]

19 Thomson ISI (Institute for Scientific Information). Journal Citation Report. Selected Scientific Journals 2001. Available from URL; www.genebee.msu.su/journals.

20 Herxheimer A. Open access to industry’s clinically relevant data. BMJ 2004; 329: 64–5.[Free Full Text]

21 Psaty BM, Furberg CD. COX-2 inhibitors--lessons in drug safety. N Engl J Med 2005; 352: 1133–5.[Free Full Text]

22 Chopra SS. MSJAMA: industry funding of clinical trials: benefit or bias? JAMA. 2003; 290: 113–4.[Free Full Text]

23 Goodyear M. Learning from the TGN1412 trial. BMJ 2006; 332: 677–8.[Free Full Text]

24 Suntharalingam G, Perry MR, Ward S, et al. Cytokine storm in a phase 1 trial of the anti-CD28 monoclonal antibody TGN1412. N Engl J Med 2006; 355: 1018–28.[Abstract/Free Full Text]

25 Zarin DA, Tse T, Ide NC. Trial Registration at ClinicalTrials.gov between May and October 2005. N Engl J Med 2005; 353: 2779–87.[Abstract/Free Full Text]

26 Steinbrook R. Gag clauses in clinical-trial agreements. N Engl J Med 2005; 352: 2160–2.[Free Full Text]

27 Olivieri NF, Brittenham GM, McLaren CE, et al. Long-term safety and effectiveness of iron-chelation therapy with deferiprone for thalassemia major. N Engl J Med 1998; 339: 417–23.[Abstract/Free Full Text]

28 Chan AW, Krleza-Jeric K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ 2004; 171: 735–40.[Abstract/Free Full Text]

29 Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ 2005; 330: 753.[Abstract/Free Full Text]

30 Garland EJ. Facing the evidence: antidepressant treatment in children and adolescents. CMAJ 2004; 170: 489–91.[Free Full Text]

31 Krzyzanowska MK, Pintilie M, Tannock IF. Factors associated with failure to publish large randomized trials presented at an oncology meeting. JAMA 2003; 290: 495–501.[Abstract/Free Full Text]

32 Thornton A, Lee P. Publication bias in meta-analysis: its causes and consequences. J Clin Epidemiol 2000; 53: 207–16.[Medline]

33 Gilbody SM, Song F, Eastwood AJ, Sutton A. The causes, consequences and detection of publication bias in psychiatry. Acta Psychiatr Scand 2000; 102: 241–9.[Medline]

34 Altman DG. Poor-quality medical research: what can journals do? JAMA 2002; 287: 2765–7.[Abstract/Free Full Text]

35 Massie BM. Negative, neutral, and discordant results: what should an editor do? J Card Fail 2003; 9: 77–9.[Medline]

36 Shields PG. Publication bias is a scientific problem with adverse ethical outcomes: the case for a section for null results. Cancer Epidemiol Biomarkers Prev 2000; 9: 771–2.[Free Full Text]

37 Yaphe J, Edman R, Knishkowy B, Herman J. The association between funding by commercial interests and study outcome in randomized controlled drug trials. Fam Pract 2001; 18: 565–8.[Abstract/Free Full Text]

38 Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine--selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 2003; 326: 1171–3.[Abstract/Free Full Text]

39 Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 2003; 326: 1167–70.[Abstract/Free Full Text]

40 Hailey D. Scientific harassment by pharmaceutical companies: time to stop. CMAJ 2000; 162: 212–3.[Free Full Text]

41 Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA 2003; 290: 921–8.[Abstract/Free Full Text]

42 Rennie D. Thyroid storm. JAMA 1997; 277: 1238–43.[Medline]

43 Cho MK, Bero LA. The quality of drug studies published in symposium proceedings. Ann Intern Med 1996; 124: 485–9.[Abstract/Free Full Text]

Related articles in CJA:

Reporting research results: a moral obligation for all researchers/Présenter les resultants de recherche : une obligation morale pour tous les chercheurs

David Moher
CJA 2007 54: 331-335. [Full Text]  

This article has been cited by other articles:

				J G Z van Uffelen, M J M Chinapaw, W van Mechelen, and M Hopman-Rock Walking or vitamin B for cognition in older adults with mild cognitive impairment? A randomised controlled trial Br. J. Sports Med., May 1, 2008; 42(5): 344 - 351. [Abstract] [Full Text] [PDF]

				D. Moher Reporting research results: a moral obligation for all researchers/Presenter les resultants de recherche : une obligation morale pour tous les chercheurs Can J Anesth, May 1, 2007; 54(5): 331 - 335. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Submit a scholarly reply Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Related articles in CJA Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Hall, R. Articles by Webber, A. Search for Related Content PubMed PubMed Citation Articles by Hall, R. Articles by Webber, A.