Publications by authors named "Yoshua Bengio"

One of the most fundamental laws of physics is the principle of least action. Motivated by its predictive power, we introduce a neuronal least-action principle for cortical processing of sensory streams to produce appropriate behavioral outputs in real time. The principle postulates that the voltage dynamics of cortical pyramidal neurons prospectively minimizes the local somato-dendritic mismatch error within individual neurons.

View Article and Find Full Text PDF

An important class of methods for modeling dynamics in open quantum systems is based on the well-known influence functional (IF) approach to solving path-integral equations of motion. Within this paradigm, path-filtering schemes based on the removal of IF elements that fall below a certain threshold aim to reduce the effort needed to calculate and store the influence functional, making very challenging simulations possible. A filtering protocol of this type is considered acceptable as long as the simulation remains mathematically stable.

View Article and Find Full Text PDF

Preparation requires technical research and development, as well as adaptive, proactive governance.

View Article and Find Full Text PDF

Governance frameworks should address the prospect of AI systems that cannot be safely tested.

View Article and Find Full Text PDF

Conscious states-state that there is something it is like to be in-seem both rich or full of detail and ineffable or hard to fully describe or recall. The problem of ineffability, in particular, is a longstanding issue in philosophy that partly motivates the explanatory gap: the belief that consciousness cannot be reduced to underlying physical processes. Here, we provide an information theoretic dynamical systems perspective on the richness and ineffability of consciousness.

View Article and Find Full Text PDF

Scientists have long conjectured that the neocortex learns patterns in sensory data to generate top-down predictions of upcoming stimuli. In line with this conjecture, different responses to pattern-matching vs pattern-violating visual stimuli have been observed in both spiking and somatic calcium imaging data. However, it remains unknown whether these pattern-violation signals are different between the distal apical dendrites, which are heavily targeted by top-down signals, and the somata, where bottom-up information is primarily integrated.

View Article and Find Full Text PDF

The COVID-19 pandemic continues to pose a substantial threat to human lives and is likely to do so for years to come. Despite the availability of vaccines, searching for efficient small-molecule drugs that are widely available, including in low- and middle-income countries, is an ongoing challenge. In this work, we report the results of an open science community effort, the "Billion molecules against COVID-19 challenge", to identify small-molecule inhibitors against SARS-CoV-2 or relevant human receptors.

View Article and Find Full Text PDF

For large libraries of small molecules, exhaustive combinatorial chemical screens become infeasible to perform when considering a range of disease models, assay conditions, and dose ranges. Deep learning models have achieved state-of-the-art results in silico for the prediction of synergy scores. However, databases of drug combinations are biased toward synergistic agents and results do not generalize out of distribution.

View Article and Find Full Text PDF

Background And Aims: Identification and photo-documentation of the ileocecal valve (ICV) and appendiceal orifice (AO) confirm completeness of colonoscopy examinations. We aimed to develop and test a deep convolutional neural network (DCNN) model that can automatically identify ICV and AO, and differentiate these landmarks from normal mucosa and colorectal polyps.

Methods: We prospectively collected annotated full-length colonoscopy videos of 318 patients undergoing outpatient colonoscopies.

View Article and Find Full Text PDF
Article Synopsis
  • Artificial intelligence is revolutionizing scientific discovery by enhancing research processes such as hypothesis generation, experiment design, and data interpretation.
  • Recent advances like self-supervised learning and geometric deep learning are improving model accuracy by utilizing vast amounts of unlabelled data and incorporating the structure of scientific data.
  • While generative AI is helping create innovations like drugs and proteins, challenges like data quality and the need for better understanding among AI developers and users persist, highlighting areas for further progress in AI research.
View Article and Find Full Text PDF

Genomic (DNA) sequences encode an enormous amount of information for gene regulation and protein synthesis. Similar to natural language models, researchers have proposed foundation models in genomics to learn generalizable features from unlabeled genome data that can then be fine-tuned for downstream tasks such as identifying regulatory elements. Due to the quadratic scaling of attention, previous Transformer-based genomic models have used 512 to 4k tokens as context (<0.

View Article and Find Full Text PDF

The apical dendrites of pyramidal neurons in sensory cortex receive primarily top-down signals from associative and motor regions, while cell bodies and nearby dendrites are heavily targeted by locally recurrent or bottom-up inputs from the sensory periphery. Based on these differences, a number of theories in computational neuroscience postulate a unique role for apical dendrites in learning. However, due to technical challenges in data collection, little data is available for comparing the responses of apical dendrites to cell bodies over multiple days.

View Article and Find Full Text PDF

Neuroscience has long been an essential driver of progress in artificial intelligence (AI). We propose that to accelerate progress in AI, we must invest in fundamental research in NeuroAI. A core component of this is the embodied Turing test, which challenges AI animal models to interact with the sensorimotor world at skill levels akin to their living counterparts.

View Article and Find Full Text PDF

The COVID-19 pandemic has spurred an unprecedented demand for interventions that can reduce disease spread without excessively restricting daily activity, given negative impacts on mental health and economic outcomes. Digital contact tracing (DCT) apps have emerged as a component of the epidemic management toolkit. Existing DCT apps typically recommend quarantine to all digitally-recorded contacts of test-confirmed cases.

View Article and Find Full Text PDF

Inverse design of short single-stranded RNA and DNA sequences (aptamers) is the task of finding sequences that satisfy a set of desired criteria. Relevant criteria may be, for example, the presence of specific folding motifs, binding to molecular ligands, sensing properties, and so on. Most practical approaches to aptamer design identify a small set of promising candidate sequences using high-throughput experiments (e.

View Article and Find Full Text PDF
Article Synopsis
  • Adversarial robustness is crucial in deep learning, but methods like adversarial training can harm the model’s performance on normal data, leading some to prioritize accuracy over robustness.
  • The proposed Interpolated Adversarial Training uses new interpolation techniques within adversarial training to maintain robustness while improving performance on conventional test data.
  • In experiments on CIFAR-10, this method significantly reduced the error increase from adversarial training, improving the standard test error from 12.32% to 6.45% while still being mathematically validated for its effectiveness.
View Article and Find Full Text PDF

One aspirational goal of computational chemistry is to predict potent and drug-like binders for any protein, such that only those that bind are synthesized. In this Roadmap, we describe the launch of Critical Assessment of Computational Hit-finding Experiments (CACHE), a public benchmarking project to compare and improve small molecule hit-finding algorithms through cycles of prediction and experimental testing. Participants will predict small molecule binders for new and biologically relevant protein targets representing different prediction scenarios.

View Article and Find Full Text PDF

De novo molecule design algorithms often result in chemically unfeasible or synthetically inaccessible molecules. A natural idea to mitigate this problem is to bias these algorithms toward more easily synthesizable molecules using a proxy score for synthetic accessibility. However, using currently available proxies can still result in highly unrealistic compounds.

View Article and Find Full Text PDF

We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm. ICT encourages the prediction at an interpolation of unlabeled points to be consistent with the interpolation of the predictions at those points. In classification problems, ICT moves the decision boundary to low-density regions of the data distribution.

View Article and Find Full Text PDF

MHC-I associated peptides (MAPs) play a central role in the elimination of virus-infected and neoplastic cells by CD8 T cells. However, accurately predicting the MAP repertoire remains difficult, because only a fraction of the transcriptome generates MAPs. In this study, we investigated whether codon arrangement (usage and placement) regulates MAP biogenesis.

View Article and Find Full Text PDF

Uniquely human cognitive faculties arise from flexible interplay between specific local neural modules, with hemispheric asymmetries in functional specialization. Here, we discuss how these computational design principles provide a scaffold that enables some of the most advanced cognitive operations, such as semantic understanding of world structure, logical reasoning, and communication via language. We draw parallels to dual-processing theories of cognition by placing a focus on Kahneman's System 1 and System 2.

View Article and Find Full Text PDF

Equilibrium Propagation is a biologically-inspired algorithm that trains convergent recurrent neural networks with a local learning rule. This approach constitutes a major lead to allow learning-capable neuromophic systems and comes with strong theoretical guarantees. Equilibrium propagation operates in two phases, during which the network is let to evolve freely and then "nudged" toward a target; the weights of the network are then updated based solely on the states of the neurons that they connect.

View Article and Find Full Text PDF