Towards greater context for scholarly metrics
Konkiel, S. (2018). Towards greater context for scholarly metrics. 5th European Congress of Conservation Biology. doi: 10.17011/conference/eccb2018/108206
© the Authors, 2018
This talk will discuss how most quantitative approaches to evaluating research do not yet fully capture the true disciplinary and public influence of science research. I suggest how machines can aid in human judgement and improve context for decision-making, in a scalable manner. Counting citations in the peer-reviewed literature has long been used to understand a paper’s influence on a field, usually as a complement to traditional peer review practices in evaluation scenarios such as promotion and tenure. Recently, field-normalized citation metrics  have been suggested as a more accurate way to understand disciplinary research performance. However, quantitative approaches to understanding research impact are not without their critics. At best, research metrics have been suggested as an evaluation tool that can only “support, not supplant, expert judgement [of research]” . At worst, it has been proposed that altmetrics and “other bibliometric nightmares” are “for people who aren’t prepared to take the time (or lack the mental capacity) to evaluate research by reading about it” . In this talk, I will briefly address some of the benefits and limitations of current attempts to improve quantitative scholarly impact indicators by providing numerical context. Though improvements over the status quo, these metrics by and large still do not: Account for the content of discussions around research (e.g. citations that confirm or refute previous findings); Reflect the stakeholder groups discussing research (e.g. citations from policymakers using research to develop conservation laws); nor Reflect the implicit values that drive research practices (e.g. metrics that accurately capture the impact of open source research software development) However, outright rejection of all impact indicators also misses the mark. With the ever-increasing volume of published research, it is impossible to rely only upon expert peer reviewers to identify influential research. Clearly, evaluation practices must scale using thoughtful, humane machine learning approaches to aid in impact assessment. I will conclude by sharing solutions to the above limitations from across academia and industry. References  Hutchins BI, Yuan X, Anderson JM, Santangelo GM (2016) Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level. PLoS Biol 14(9): e1002541. https://doi.org/10.1371/journal.pbio.1002541  Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., … Johnson, B. (2015). Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management. Higher Education Funding Council for England. London.  Colquhoun, D. (2014). Why you should ignore altmetrics and other bibliometric nightmares. DC’s Improbable Science. 16 January 2014. Retrieved from http://www.dcscience.net/2014/01/16/why-you-should-ignore-altmetrics-and-other-bibliometric-nightmares/ ...
PublisherOpen Science Centre, University of Jyväskylä
ConferenceECCB2018: 5th European Congress of Conservation Biology. 12th - 15th of June 2018, Jyväskylä, Finland
MetadataShow full item record
- ECCB 2018 
Showing items with similar title or keywords.
Singular quasisymmetric mappings in dimensions two and greater Romney, Matthew (Academic Press, 2019)For all n ≥2, we construct a metric space (X, d)and a quasisymmetric mapping f:[0, 1]n→X with the property that f−1 is not absolutely continuous with respect to the Hausdorff n-measure on X. That is, there exists a Borel ...
English language for greater good? : the attitudes of Jyväskylä University students towards English language in face-to-face fundraising Kostiainen, Tommi (2019)F2F-varainkeruu, eli feissaus on hyväntekeväisyysjärjestöjen käyttämä keino hankkia uusia kuukausilahjoittajia hyväntekeväisyystyölle. Feissauksessa varainkerääjät, feissarit, liikkuvat julkisilla paikoilla, kuten keskustassa, ...
Towards a democratic EU online communication ia an organisational and civil context Meriläinen, Niina (2009)
Toward adjustment profiles for lower secondary student-athletes in the Finnish dual career context : A mixed-methods approach Kuokkanen, Joni; Romar, Jan-Erik; Hirvensalo, Mirja (Elsevier BV, 2022)Engagement, burnout, and experiences are central concepts for understanding student-athlete adjustment in a dual career (DC) environment. As part of a larger 3-year, lower secondary sports schools pilot project, the aim ...