Bibliometrics, research quality, and neglected tropical diseases

Increasingly academics and research institutions are being judged on the quality of their research by how frequently their published work is cited by others. Citation rates and H factors are now making or breaking researchers' careers and influencing how much money institutions and universities receive. Although this move towards a more objective assessment of research quality should be applauded, the potential for systematic biases could still exist. Mostly these citation rates are dependent on a single commercial database and search engine (Web of Knowledge), although there is at least one other rival provider (Scopus). I have studied the effect of choice of database on citation rates per publication for several tropical1Hotez PJ Molyneux DH Fenwick A et al.Current concepts—control of neglected tropical diseases.N Engl J Med. 2007; 357: 1018-1027Crossref PubMed Scopus (1154) Google Scholar and non-tropical infectious diseases. I calculated citation rates per disease from citations in 2007 to papers published in the years 2004–06 for the two databases above. I then included citation rate ratios in a meta-analysis done with StatsDirect. Rather more papers were listed in Scopus than Web of Knowledge, reflecting the larger number of medical journals included in the former. However, Scopus listed 55% more papers on the chosen tropical infections than Web of Knowledge, compared with only 35% more for the non-tropical diseases. The citation rate ratios are shown in the figure. For the non-tropical diseases, citation rates were roughly equal between the two databases, whereas citation rates for the tropical disease were significantly higher in Scopus. The reasons for this discrepancy are reasonably obvious. Scopus references many more journals than Web of Knowledge, at least in the medical arena, and journals from developing countries are much more likely to feature in Scopus. This not only makes it less likely that tropical researchers will have their work listed by the market leader, but even if it is, many of the citations to their work will not be detected. It was not possible to compare citation rates for papers published in both databases, although, had this been possible, it is likely that the discrepancy in citation rates would have been even greater because many of the additional papers cited in Scopus are likely to have had lower citations. It could be argued that what matters is that tropical disease researchers are compared with others in the specialty. However, this view would be naive. Many researchers on tropical disease issues also work on diseases of the developed world and compete for jobs and promotion in departments where developed-world researchers also work. It is essential that when using citation rates to make decisions about funding, employment, or promotion of researchers, due account is made of the possible effect of inclusion criteria of the databases used. Otherwise the neglected tropical diseases will become even more neglected. Scopus is a product of Elsevier—the publisher of The Lancet. I declare that I have no conflict of interest.