Exploring the application of machine learning to expert evaluation of research impact.

The objective of this study is to investigate the application of machine learning techniques to the large-scale human expert evaluation of the impact of academic research. Using publicly available impact case study data from the UK's Research Excellence Framework (2014), we trained five machine learning models on a range of qualitative and quantitative features, including institution, discipline, narrative style (explicit and implicit), and bibliometric and policy indicators. Our work makes two key contributions.

Interpreting Bibliometric Data.

Many academic analyses of good practice in the use of bibliometric data address only technical aspects and fail to account for and appreciate user requirements, expectations, and actual practice. Bibliometric indicators are rarely the only evidence put before any user group. In the present state of knowledge, it is more important to consider how quantitative evaluation can be made simple, transparent, and readily understood than it is to focus unduly on precision, accuracy, or scholarly notions of purity.

Evolving Document Patterns in UK Research Assessment Cycles.

Researcher behavior is shown to change under assessment. An unexpected time-skew toward most recent papers in each census period was found among the outputs selected by UK academics for the research assessment cycles of the 1990s. This skew changed to a more even time-based distribution for scientists and engineers in later cycles.

Follow #eHealth2011: Measuring the Role and Effectiveness of Online and Social Media in Increasing the Outreach of a Scientific Conference.

Background: Social media promotion is increasingly adopted by organizers of industry and academic events; however, the success of social media strategies is rarely questioned or the real impact scientifically analyzed.

Objective: We propose a framework that defines and analyses the impact, outreach, and effectiveness of social media for event promotion and research dissemination to participants of a scientific event as well as to the virtual audience through the Web.

Methods: Online communication channels Twitter, Facebook, Flickr, and a Liveblog were trialed and their impact measured on outreach during five phases of an eHealth conference: the setup, active and last-minute promotion phases before the conference, the actual event, and after the conference.