Here is Tolstoy’s opening sentence from his novel Anna Karenina:
“Happy families are all alike; every unhappy family is unhappy in its own way.”
Loosely speaking Bornmann and Marx recently used Tolstoy’s notion of happy families and the Anna Karenina Principle when considering what elements make for successful science. In the bibliometric sense, it’s useful as well to explain how groups of publications which function well and frequently get cited, have similar attributes in common which help them garner citation impact. Poor ‘bibliometric families’ do not always get the citation impact they deserve, perhaps because they lack those features. Recognising and harnessing the key features which can help get good citation impact is an important and useful part of bibliometrics.
On the other hand bibliometrics can be viewed as being a bit like the screenwriter William Goldman’s view of success in Hollywood, here he said ‘Nobody knows anything’ in terms of what consistently makes a film successful, or in this case why something is well cited. So even if you do all the ‘right things’ you are not guaranteed success, but if you do, I would argue that you are improving your chances of doing so.
So in general it’s always important to maximise your chances of being cited by recognising what helps and what doesn’t. But because publishing strategies and subjects vary widely in the way they function, care must be taken in both how you attempt to maximise impact and how you might wisely interpret its result.
What are bibliometrics?
The authors of a recent study on the citation impact of India explained very concisely, without jargon, what bibliometrics are and their usefulness, and so quoting them:
Bibliometrics is the analysis of data derived from publications and their citations. Publication of research outcomes is an integral part of the research process and is a universal activity. Consequently, bibliometric data have a currency across subjects, time and location that is found in few other sources of research-relevant data. The use of bibliometric analysis, allied to informed review by experts, increases the objectivity of and confidence in evaluation.
In the study the authors show the changing scientific impact of India by analysing the research impact of the country through its publications and their citation impact both in comparison to other countries and India’s own relative progress.
So bibliometrics can be used to assess the impact of a country or with some caution even the impact of an individual. But rather than individuals it is better used to look at groups and assess their impact relative to other groups engaged in the same or similar research areas. In doing this you can explore their productivity in terms of scientific output in publications and their impact as measured by how frequently they have been cited. These two features and their attendant characteristics, by subject and publishing culture, form principally, the cornerstone of bibliometrics.
The University of Surrey’s strategic view
The University thinks citations are important and has in its University Strategy: 2012-17 amongst other things committed to:
Continue to grow the volume of our research outputs and build on the improvements we have made to their quality and reputation, as measured by citations in target academic journals
The two key things here ‘…as measured by citations’ and ‘… in target academic journals’ are very important and as we shall see later they more or less go hand in hand. The University also says it wants to
Implement the targeted journal strategy including increasing knowledge and awareness of bibliometric considerations to increase % of academic staff with high quality publications in target journals
So awareness of bibliometrics is something that can be done through these few postings and should help meet this strategic objective.
Why does citation impact matter to individuals and the university?
Having good citation impact enhances the reputation of the university and the individual. It’s broadly a measure of how influential the work of the university is when its work is cited by others. Of course it is individuals that do the research and publish their work and in turn get cited, so collectively, the success of the individual academic becomes the success of the university.
University league tables
University league tables are here to stay. If we think of the plethora of products and services available to us and how we try to make sense of what’s on offer to us, it’s not surprisingly that we attempt to rank and order things in ways that can guide our choices, universities are no different. Hence national league tables are important to the university, they influence the choices made by students and advice given by schools in the UK. It’s a simplistic measure of course, but the higher our ranking the greater is the perception of our relative standing and quality.
Good rankings in international league tables are hard to achieve and to sustain. A key measure used by an increasing number of them is citation impact and where you publish. The Times Higher Education World University Rankings apportions 30% of its ranking score to citation impact. The Academic Ranking of World Universities (Shanghai) apportions 60% to where academics publish their work. The Leiden ranking uses only measures of bibliometric impact; hence rankings are solely dependent on citation counts. Where academics publish their work and how frequently they are cited is key to success in these rankings and of course these tables have international audiences attracting potentially the best students and academics.
The Research Excellence Framework (REF)
The REF allows, in some subject areas, the use of citation data to help inform panel members about the quality of the work they are assessing. Physics, chemistry and computer science are three subjects from amongst the eleven subjects which will use citation data. Peer review will remain the primary method for assessing quality, but nevertheless citation data will be available to panel members.
 Bornmann, L & Marx, 2012.
 Goldman W. 1996. Adventures in the screen trade. Abaccus.
 Bibliometric study of India’s Scientific Publication outputs during 2001-10. 2012. Available at: http://www.dst.gov.in/whats_new/whats_new12/report.pdf [Accessed 18 Feb 2012].
 The University Strategy: 2012-17. Available at: http://www.surrey.ac.uk/about/corporate/documents/university-strategy-2012-17.pdf [Accessed 19 Feb 2013].
 The University Strategy: the next four years. Available at: http://www.surrey.ac.uk/about/corporate/documents/university-strategy-2012-17.pdf [Accessed 19 Feb 2013].
 Times Higher Education World University Rankings 2013. Available at: http://www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=417345&c=1 [Accessed 19 Feb 2013].
 Academic Ranking of World Universities 2013. Available at: http://www.shanghairanking.com/ARWU-Methodology-2012.html [Accessed 19 Feb 2013].