Abstract:

Research on academic paper ranking has received great attention in recent years, and various algorithms have been proposed to automatically assess a large number of papers such as those typically required in systematic literature reviews. Similarly, scientific journals have their own classification schemes, based, e.g., on the impact factors, Scopus/WoS quartiles, of subject areas. Numerous indexes for evaluating the authors’ performance and prestige in a given field of research also exists in scientometrics, although there is a continuous discussion in the scientific community about the usage of those indexes and on the interpretation of the resulting ranking of authors.

In this study we propose a new approach for evaluating the authors’, papers’ and journals’ prestige with respect to a given field of research, with the possibility of combining the three single evaluations into a joint performance index. The evaluation follows the same logic for the authors’ and papers’ prestige and specifically, it grounds on a revisited Pareto analysis, which aims at identifying three classes of prestige for both the authors and papers. The starting point of the analysis is obviously a set of papers, as those that can be retrieved from scientific databases when carrying out systematic literature reviews. For those papers, the key metadata and the number of citations should be retrieved. The authors of the same papers also need to be retrieved from the database. Authors are then classified on the basis of the number of papers published, the year of publication of the last paper in the targeted field of research, and the number of citations received; journals are classified using the same parameters, while papers are categorised on the basis of the number of citations received.

The proposed framework, which is illustrated in detail in the paper, can be applied to virtually any research field. An application in the engineering area is currently being planned by some of the authors of this paper.

Keywords:

Bibliometrics, paper ranking, journal ranking, authors ranking, Pareto analysis, research methods.