Relevance versus big numbers: Students' criteria for selecting scholarly references online.

This study examined the effect of the number of citations attributed to documents on third year psychology students' selection of bibliographical references. Our main assumption was that students would take high numbers of citations as accessible relevance cues and use them heuristically to facilitate decision making, potentially bypassing deeper relevance assessment based on semantic processing. Experiment 1 presented the students with a reference selection task while manipulating the number of citations attributed to references, and found that the number of citations had a strong impact on reference selection. Moreover, the effect was independent from topic familiarity and even from students' prior knowledge of what the number of citations meant. Experiment 2 used eye-tracking data to show that this "big number" effect was contingent upon the participants fixating the numbers of citations attributed to documents. Experiment 3 manipulated the semantic relevance of references to the search topic, and demonstrated that the less relevant references were 3 times more likely to be selected when they came with a high number of citations. Overall, the study shows that the number of citations significantly influences students' selections, competing with the semantic relevance of references. Implications for the teaching of online search skills are discussed. (PsycINFO Database Record (c) 2018 APA, all rights reserved).