Considerations on the Impact Factor as a Tool in Scientific Assessment

Authors

  • Katia Denise BrescianiI
  • José Antonio Soares
  • Jorge Ramos
  • Tomás Patrocinio
  • Lucas Shigaki Vinicius
  • Maria Luisa Machado

Keywords:

impact factor, citation database, scientific quality, research outcome evaluation.

Abstract

This exploratory study aims to wide the debate over the impact factor and its application as a tool in the assessment of science. The analysis of scientific activity has been the subject of much debate, especially in the last decade. The impact factor and the citation frequency have been regularly used in the individual analysis of researchers as well verify the performance of journals, investigators, research centers, graduate programs, institutions and countries. The impact factor of a journal can be highly influenced by few of its articles that are very frequently cited. Some researchers are more concerned about publishing in high impact journals than about doing science itself. 

The manner how the scientific research output is assessed by funding agencies, academic institutions, among others, has been the subject of much debate and the current great question is whether the impact factor (IF) is the best indicator to assess the scientific quality or an already obsolete measure. There is one movement of researchers against the impact factor to be used as a qualitative measure of research articles or even as an indicator of the contribution of a scientist for possible hiring, promotion or decision for grant. Then, there is a need of academic community deeply reflect on this important issue 

References

[1] N. SENGUPTA. “Bibliometrics, informetrics, scientometrics and librametrics: an overview.” Libri, vol. 42, n. 2, pp. 99-135, 1992.
[2] J. TAGUE-SUTCKIFFE. “An introduction to informetrics.” Information Processing & Management, vol. 28, n.1, pp. 1-3, 1992.
[3] C.A. MACIAS-CHAPULA. “O papel da informetria e da cienciometria e sua perspectiva nacional e internacional.” Ciência da Informação, vol. 27, n. 2, pp. 134-140, 1998.
[4] L. STREHL. “O fator de impacto do ISI e a avaliação da produção científica: aspectos conceituais e metodológicos.” Revista Ciência da Informação, vol. 34, n. 1, pp. 19-27, jan./apr. 2005.
[5] F. HJOREGAARD CHRISTENSEN; P. INGWERSEN. “Data set isolation for bibliometric online analyses of research publications: fundamental methodological issues.” Journal of the American Society for Information Science, vol. 48, n. 3, pp. 205-217, 1997.
[6] I. WORMELL. Informetria: explorando bases de dados como instrumentos de análise. Ciência da Informação, vol. 27, n. 2, 1998. Available: <www.scielo.br/cgi-bin/fbpe/> [Jun. 16, 2012].
[7] N.A.P. VANTI. “Da bibliometria à webometria: uma exploração conceitual dos mecanismos utilizados para medir o registro da informação e a difusão do conhecimento.” Revista Ciência da Informação, vol. 31, n. 2, pp. 152-162, may/aug, 2002.
[8] Y. OKUBO. “Bibliometric indicators and analysis of research systems: methods and examples.” OECD Science, Technology and Industry Working Papers, 1997.
[9] J.A. SILVA; M.L.P. BIANCHI. “Cientometria: a métrica da ciência.” Paidéia, 2001, vol. 11, n. 21, pp. 5-10. ISSN 0103-863X. Available: <http://www.scielo.br/pdf/paideia/v11n21/02.pdf> [August 20, 2012].
[10] C.A. ARAÚJO. “Bibliometria: evolução histórica e questões atuais.” Em Questão, Porto Alegre, vol. 12, n. 1, pp. 11-32, 2006.
[11] V. ROQUE. “Métricas da Informação: O fator de Impacto na prática.” Egitana Sciencia, vol. 6; n. 10, pp. 177-207, 2012.
[12] R. ABRAHAM. Webometry: Measuring the complexity of the world wide web. World Futures, v.50, n.1, p.785-791, 1997.
[13] L. BJÖRNEBORN; P. INGWERSEN. “Toward a basic framework for webometrics.” Journal of the American Society for Information Science and Technology, vol. 55, n. 14, pp. 1216-1227, 2004.
[14] C.J.C. JABBOUR; A.B.L.S. JABBOUR; J.H.C. OLIVEIRA. “The Perception of Brazilian Researchers concerning the Factors that Influence the Citation of their Articles: A Study in the Field of Sustainability.” Serials Review, vol. 39, pp. 1-4, 2013.
[15] T.S. KHUN. “The structure of scientific revolutions.” The University of Chigaco Press, 1962.
[16] G.L. VOLPATO. Publicação científica. Editora Cultura Acadêmica, 2008.
[17] L. BORNMANN; I. NAST; H.D. DANIEL. “Do editors and referees look for signs of scientific misconduct when reviewing manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting and rejecting manuscripts for publication.” Scientometrics, vol. 77, n.3, pp. 415-432, 2008.
[18] A.F. MAVROGENIS; P. RUGGIERI; P.J. PAPAGELOPOULOS. “Self-citation in Publishing.” Clinical Orthopaedics and Related Research, vol. 468, pp. 2803–2807, 2010.
[19] Psychology, vol.11, pp.95–107, 2011c.
[20] G. BUELA-CASAL; I. ZYCH. What do the scientists think about the impact factor? Scientometrics, vol. 92, pp. 281–292, 2012.
[21] E. GARFIELD. “Citation indexes for science: a new dimension in documentation through association of ideas.” Science, vol. 122, pp. 108-111, 1955.
[22] E. GARFIELD. “Citation analysis as a tool in journal evaluation: journals can be ranked by frequency and impact of citations for science policy studies.” Science, vol. 178, n. 4060, pp. 471-479, 1972.
[23] E. GARFIELD. “How can impact factors be improved?” British Medical Journal, vol. 313, pp. 411-413, 1996.
[24] E. GARFIELD. “Journal impact factor: a brief review.” Canadian Medical Association Journal, vol. 161, pp. 979–980, 1999.
[25] E. GARFIELD. “The meaning of the impact factor.” International Journal of Clinical and Health Psychology, vol. 3, pp. 363–369, 2003.
[26] E. GARFIELD. The history and meaning of the journal impact factor. Journal of the American Medical Association, vol. 295, pp. 90–93, 2006.
[27] R. SANCHO. “Indicadores bibliometricos utilizados en la evaluacion de la ciencia y la tecnologia: revision bibliografica”. Revista Española de Documentación Cientifica, vol. 13, n.3/4, pp. 842-865, 1990.
[28] E. GARFIELD. The evolution of the science citation index. International Microbiology, vol.10, pp.65–69, 2007.
[29] S.M. DHAWAN; S.K. PHULL; S.P. JAIN. “Selection of scientific journal: a model.” Journal of Documentation, vol. 36, n. 1, pp. 24-32, 1980.
[30] B.C. PERITZ. “On the association between journal circulation and impact factor.” Journal of Information Science, vol. 21, n. 1, pp. 63-67, 1995.
[31] M. AMIN; M. MABE. “Impact factors: use and abuse.” Perspectives in publising, vol. 63, n. 41, pp. 347-354, 2003.
[32] D.W. AKSNES. “A macro study of self-citation.” Scientometrics, vol. 56, n. 2, pp. 235-246, 2003.
[33] S. AASE. “What Weight Does Impact Factor Carry?” Journal of the American Dietetic Association, vol.108, n.10, pp.1604-1607, 2008.
[34] E.B. ARRIBALZAGA. “El factor de impacto o el impacto de factores.” Revista Chilena de Cirurg?a, vol.57, pp.269–274, 2005.
[35] E.B. ARRIBALZAGA. “El factor de impacto: su uso, abuso y mal uso.” Revista Argentina de Cirurg?a, vol. 97, pp. 15–21, 2009.
[36] E. BAGATIN; B. GONTIJO. “The expansion of a measure: what is a scientific journal impact factor and how important is it for academic Brazilian dermatologists.” International Journal of Dermatology, vol. 50, pp. 1432–1434, 2011.
[37] G. BUELA-CASAL. “Indices de impacto de las revistas cient?ficas e indicadores para medir el rendimiento de los investigadores.” Revista de Psicodidáctica, vol. 15, pp. 3–19, 2010a.
[38] G. BUELA-CASAL et al. Ranking de 2010 en producción y productividad en investigación de las universidades públicas espanõlas. Psicothema, vol. 23, pp. 527–536, 2011a.
[39] G. BUELA-CASAL et al. Rendimiento en el doctorado en función del área de conocimiento. Revista de Psicodidáctica, vol. 16, pp. 181–192, 2011b.
[40] G. BUELA-CASAL et al. Ranking de 2009 en investigación de las universidades públicas espanõlas. Psicothema, vol. 22, pp. 171–179, 2010b.
[41] G. BUELA-CASAL et al. The h index of the presidents of the American Psychological Association (APA) through journal articles included in the web of science database. International Journal of Clinical and Health, 2011c.
[42] I. ZYCH. Comparacion de los criterios para la acreditacion de profesores contratados y Funcionarios. Aula Abierta, vol. 39, pp. 51–62, 2011.
[43] A. GALAN-GONZALEZ; I. ZYCH. Análisis de los criterios de la comisio´n nacional evaluadora de La actividad investigadora (CNEAI) para la concesión de los tramos de investigación en educación. Revista de pedagogía, vol. 63, n. 2, pp. 117–139, 2011.
[44] A. ANDRADE; R. GONZÁLEZ-JONTE; J.M. CAMPANARIO. “Journals that increase their impact factor at least fourfold in a few years: The role of journal self-citations.” Scientometrics, vol. 80, n. 2, pp. 517–530, 2009.
[45] T.P. KURMIS; A.P. KURMIS, “Self-citation rates among medical imaging journals and a possible association with impact factor.” Radiography, vol. 16, n. 1, p. 21–25, 2010.
[46] B. MINASNY; A.E. HARTEMINK; A. MCBRATNEY. “Individual, country and journal self-citation in soil science.” Geoderma, vol.155, pp. 434-438, 2010.
[47] L. GONZÁLEZ; J.M. CAMPANARIO. “Structure of the impact factor of journals included in the social sciences citation index: citations from documents labeled Editorial Material.” Journal of the American Society for Information Science and Technology, vol. 58, pp. 252–262, 2007.
[48] B. MINASNY; A. MCBRATNEY; A.E. HARTEMINK. “Soil bibliometrics - How much do we love cite ourselves.” Pedometron, vol. 22, pp. 11-13, 2007.
[49] G.L. VOLPATO. Ciência: da filosofia à publicação. 6ª ed. Cultura Acadêmica, 2012.
[50] H.A. BUCHTEL; S. DELLA SALA. “Impact factor: Does the 80/20 rule apply to Cortex?” Cortex, vol. 42, pp. 1064–1065, 2006.
[51] J.A. FOLEY; S. DELLA SALA. “The impact of self-citation.” Cortex, vol. 46, pp. 802-810, 2010.
[52] X. XIAO-DONG; W. YA-WEN. “Journal Self-citation Analysis of Some Chinese Sci-tech Periodicals.” Serials Review, vol. 37, pp. 171–173, 2011.
[53] J.E. HIRSCH. “An index to quantify an individual's scientific research output.” Proceedings of the National Academy of Sciences USA. vol. 102, n. 46, pp. 16569–16572, nov. 2005.
[54] R.N.M. SANTOS; N.Y. KOBASHI. “Bibliometria, cientometria, infometria: conceitos e aplicações.” Revista Pesquisa Brasileira Ciência da Informação, vol. 2, n. 1, pp. 155-172, 2009.
[55] J.A.C. BAUM. “Free-Riding on Power Laws: questioning the validity of the Impact factor as a measure of research quality in organization studies”. Organization, vol.18, pp. 449–466, 2011.
[56] J.M. NIGHTINGALE; G. MARSHALL. “Citation analysis as a measure of article quality, journal influence and individual researcher performance.” Radiography, vol. 18, n.2, pp. 60–67, 2012.
[57] ALBERTS. “Editorial”. Science, vol . 340, 17 may, 2013.
[58] S. CARLEY; A.L. PORTER; J. YOUTIE. “Toward a more precise definition of self-citation.” Scientometrics, vol. 94, pp. 777–780, 2013.

Downloads

Published

2015-11-01

How to Cite

BrescianiI, K. D., Soares, J. A., Ramos, J., Patrocinio, T., Vinicius, L. S., & Machado, M. L. (2015). Considerations on the Impact Factor as a Tool in Scientific Assessment. American Scientific Research Journal for Engineering, Technology, and Sciences, 14(2), 295–303. Retrieved from https://asrjetsjournal.org/index.php/American_Scientific_Journal/article/view/1029