Predicting the replicability of social and behavioural science claims in COVID-19 preprints.

Replications are important for assessing the reliability of published findings. However, they are costly, and it is infeasible to replicate everything. Accurate, fast, lower-cost alternatives such as eliciting predictions could accelerate assessment for rapid policy implementation in a crisis and help guide a more efficient allocation of scarce replication resources.

Predicting and reasoning about replicability using structured groups.

This paper explores judgements about the replicability of social and behavioural sciences research and what drives those judgements. Using a mixed methods approach, it draws on qualitative and quantitative data elicited from groups using a structured approach called the IDEA protocol ('investigate', 'discuss', 'estimate' and 'aggregate'). Five groups of five people with relevant domain expertise evaluated 25 research claims that were subject to at least one replication study.

What's on the horizon for community-based conservation? Emerging threats and opportunities.

Community-based conservation can support livelihoods and biodiversity, while reinforcing local and Indigenous values, cultures, and institutions. Its delivery can help address cross-cutting global challenges, such as climate change, conservation, poverty, and food security. Therefore, understanding trends in community-based conservation is pertinent to setting and implementing global goals.

Predicting reliability through structured expert elicitation with the repliCATS (Collaborative Assessments for Trustworthy Science) process.

As replications of individual studies are resource intensive, techniques for predicting the replicability are required. We introduce the repliCATS (Collaborative Assessments for Trustworthy Science) process, a new method for eliciting expert predictions about the replicability of research. This process is a structured expert elicitation approach based on a modified Delphi technique applied to the evaluation of research claims in social and behavioural sciences.

Reimagining peer review as an expert elicitation process.

Journal peer review regulates the flow of ideas through an academic discipline and thus has the power to shape what a research community knows, actively investigates, and recommends to policymakers and the wider public. We might assume that editors can identify the 'best' experts and rely on them for peer review. But decades of research on both expert decision-making and peer review suggests they cannot.

Towards open, reliable, and transparent ecology and evolutionary biology.

Unreliable research programmes waste funds, time, and even the lives of the organisms we seek to help and understand. Reducing this waste and increasing the value of scientific evidence require changing the actions of both individual researchers and the institutions they depend on for employment and promotion. While ecologists and evolutionary biologists have somewhat improved research transparency over the past decade (e.

Bioengineering horizon scan 2020.

Horizon scanning is intended to identify the opportunities and threats associated with technological, regulatory and social change. In 2017 some of the present authors conducted a horizon scan for bioengineering (Wintle et al., 2017).

Verbal probabilities: Very likely to be somewhat more confusing than numbers.

People interpret verbal expressions of probabilities (e.g. 'very likely') in different ways, yet words are commonly preferred to numbers when communicating uncertainty.

Better Together: Reliable Application of the Post-9/11 and Post-Iraq US Intelligence Tradecraft Standards Requires Collective Analysis.

The events of 9/11 and the October 2002 National Intelligence Estimate on Iraq's Continuing Programs for Weapons of Mass Destruction precipitated fundamental changes within the United States Intelligence Community. As part of the reform, analytic tradecraft standards were revised and codified into a policy document - Intelligence Community Directive (ICD) 203 - and an analytic ombudsman was appointed in the newly created Office for the Director of National Intelligence to ensure compliance across the intelligence community. In this paper we investigate the untested assumption that the ICD203 criteria can facilitate reliable evaluations of analytic products.

Ten-year assessment of the 100 priority questions for global biodiversity conservation.

In 2008, a group of conservation scientists compiled a list of 100 priority questions for the conservation of the world's biodiversity. However, now almost a decade later, no one has yet published a study gauging how much progress has been made in addressing these 100 high-priority questions in the peer-reviewed literature. We took a first step toward reexamining the 100 questions to identify key knowledge gaps that remain.

The Value of Performance Weights and Discussion in Aggregated Expert Judgments.

In risky situations characterized by imminent decisions, scarce resources, and insufficient data, policymakers rely on experts to estimate model parameters and their associated uncertainties. Different elicitation and aggregation methods can vary substantially in their efficacy and robustness. While it is generally agreed that biases in expert judgments can be mitigated using structured elicitations involving groups rather than individuals, there is still some disagreement about how to best elicit and aggregate judgments.

A transatlantic perspective on 20 emerging issues in biological engineering.

Advances in biological engineering are likely to have substantial impacts on global society. To explore these potential impacts we ran a horizon scanning exercise to capture a range of perspectives on the opportunities and risks presented by biological engineering. We first identified 70 potential issues, and then used an iterative process to prioritise 20 issues that we considered to be emerging, to have potential global impact, and to be relatively unknown outside the field of biological engineering.

Metaresearch for Evaluating Reproducibility in Ecology and Evolution.

Recent replication projects in other disciplines have uncovered disturbingly low levels of reproducibility, suggesting that those research literatures may contain unverifiable claims. The conditions contributing to irreproducibility in other disciplines are also present in ecology. These include a large discrepancy between the proportion of "positive" or "significant" results and the average statistical power of empirical research, incomplete reporting of sampling stopping rules and results, journal policies that discourage replication studies, and a prevailing publish-or-perish research culture that encourages questionable research practices.

A 2017 Horizon Scan of Emerging Issues for Global Conservation and Biological Diversity.

We present the results of our eighth annual horizon scan of emerging issues likely to affect global biological diversity, the environment, and conservation efforts in the future. The potential effects of these novel issues might not yet be fully recognized or understood by the global conservation community, and the issues can be regarded as both opportunities and risks. A diverse international team with collective expertise in horizon scanning, science communication, and conservation research, practice, and policy reviewed 100 potential issues and identified 15 that qualified as emerging, with potential substantial global effects.

A Horizon Scan of Global Conservation Issues for 2016.

This paper presents the results of our seventh annual horizon scan, in which we aimed to identify issues that could have substantial effects on global biological diversity in the future, but are not currently widely well known or understood within the conservation community. Fifteen issues were identified by a team that included researchers, practitioners, professional horizon scanners, and journalists. The topics include use of managed bees as transporters of biological control agents, artificial superintelligence, electric pulse trawling, testosterone in the aquatic environment, building artificial oceanic islands, and the incorporation of ecological civilization principles into government policies in China.

A horizon scan of global conservation issues for 2015.

This paper presents the results of our sixth annual horizon scan, which aims to identify phenomena that may have substantial effects on the global environment, but are not widely known or well understood. A group of professional horizon scanners, researchers, practitioners, and a journalist identified 15 topics via an iterative, Delphi-like process. The topics include a novel class of insecticide compounds, legalisation of recreational drugs, and the emergence of a new ecosystem associated with ice retreat in the Antarctic.

Using strategic foresight to assess conservation opportunity.

The nature of conservation challenges can foster a reactive, rather than proactive approach to decision making. Failure to anticipate problems before they escalate results in the need for more costly and time-consuming solutions. Proactive conservation requires forward-looking approaches to decision making that consider possible futures without being overly constrained by the past.

Exploring risk judgments in a trade dispute using Bayesian networks.

Bayesian networks (BNs) are graphical modeling tools that are generally recommended for exploring what-if scenarios, visualizing systems and problems, and for communication between stakeholders during decision making. In this article, we investigate their potential for exploring different perspectives in trade disputes. To do so, we draw on a specific case study that was arbitrated by the World Trade Organization (WTO): the Australia-New Zealand apples dispute.

Expert status and performance.

Expert judgements are essential when time and resources are stretched or we face novel dilemmas requiring fast solutions. Good advice can save lives and large sums of money. Typically, experts are defined by their qualifications, track record and experience.