Light field (LF) cameras record both intensity and directions of light rays, and encode 3D scenes into 4D LF images. Recently, many convolutional neural networks (CNNs) have been proposed for various LF image processing tasks. However, it is challenging for CNNs to effectively process LF images since the spatial and angular information are highly inter-twined with varying disparities. In this paper, we propose a generic mechanism to disentangle these coupled information for LF image processing. Specifically, we first design a class of domain-specific convolutions to disentangle LFs from different dimensions, and then leverage these disentangled features by designing task-specific modules. Our disentangling mechanism can well incorporate the LF structure prior and effectively handle 4D LF data. Based on the proposed mechanism, we develop three networks (i.e., DistgSSR, DistgASR and DistgDisp) for spatial super-resolution, angular super-resolution and disparity estimation. Experimental results show that our networks achieve state-of-the-art performance on all these three tasks, which demonstrates the effectiveness, efficiency, and generality of our disentangling mechanism. Project page: https://yingqianwang.github.io/DistgLF/.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1109/TPAMI.2022.3152488 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Internalization and Cellular Fate of Protein Corona-Coated Nanoparticles by Multimodal Multi-Scale Microscopy.
Small
December 2024
Brazilian Synchrotron Light Laboratory (LNLS), Brazilian Center for Research in Energy and Materials (CNPEM), Campinas, São Paulo, 13083-970, Brazil.
Upon exposure to biological environments, nanoparticles are rapidly coated with biomolecules, predominantly proteins, which alter their colloidal stability, biodistribution, and cell interactions. Despite extensive efforts to investigate the nanoparticles' fate, only a few studies use high-resolution characterization methods that allow in-depth characterization, and the existing methodologies are unable to differentiate particles internalized at the onset of incubation from those taken up toward the end of an incubation period. In this study, these limitations related to incubation disparities are overcame and precisely monitored the spatiotemporal displacement of colloidally stable protein corona-coated nanoparticles within cells.
View Article and Find Full Text PDFSeBIR: Semantic-guided burst image restoration.
Neural Netw
January 2025
School of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China. Electronic address:
Burst image restoration methods offer the possibility of recovering faithful scene details from multiple low-quality snapshots captured by hand-held devices in adverse scenarios, thereby attracting increasing attention in recent years. However, individual frames in a burst typically suffer from inter-frame misalignments, leading to ghosting artifacts. Besides, existing methods indiscriminately handle all burst frames, struggling to seamlessly remove the corrupted information due to the neglect of multi-frame spatio-temporal varying degradation.
View Article and Find Full Text PDFHCMV strain- and cell type-specific alterations in membrane contact sites point to the convergent regulation of organelle remodeling.
J Virol
November 2024
Department of Molecular Biology, Princeton University, Princeton, New Jersey, USA.
Viruses are ubiquitous entities that infect organisms across the kingdoms of life. While viruses can infect a range of cells, tissues, and organisms, this aspect is often not explored in cell culture analyses. There is limited information about which infection-induced changes are shared or distinct in different cellular environments.
View Article and Find Full Text PDF4D Flow Magnetic Resonance Imaging (4D Flow MRI) is a non-invasive measurement technique capable of quantifying blood flow across the cardiovascular system. While practical use is limited by spatial resolution and image noise, incorporation of trained super-resolution (SR) networks has potential to enhance image quality post-scan. However, these efforts have predominantly been restricted to narrowly defined cardiovascular domains, with limited exploration of how SR performance extends across the cardiovascular system; a task aggravated by contrasting hemodynamic conditions apparent across the cardiovasculature.
View Article and Find Full Text PDFThe polarization imaging technique leverages the disparity between target and background polarization information to mitigate the impact of backward scattered light, thereby enhancing image quality. However, the imaging model of this method exhibits limitations in extracting inter-image features, resulting in less-than-optimal outcomes in turbid underwater environments. In recent years, machine learning methodologies, particularly neural networks, have gained traction.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!